New - ISO compression - help testing
#41
Yeah, I know where the xz docs are Wink and I've also been reading many "chatters" before settling on gzip as the first attempt.

As I said, xz is possible but:

- Will generate tons of support requests because it needs special parameters during compression in order to make it viable for random access decompression, and since most people will want the smallest files, their xz files will end up as extremely slow in PCSX2 or not working at all -> "But I have a valid xz file and it's too slowwwwww... mommy...".

- And it needs more coding and integration, which always takes time.

Bottom line, if you're still trying to convince me that xz is an interesting option, you shouldn't, because I thought so too from the beginning, and still do. It just that everything takes time and on top of that, xz has a potential for other issues, as I noted above.
Reply

Sponsored links

#42
I'm completely at your orders man, no struggle intended

A simple drag-and-drop batch file with the recommended parameters may be sufficient even for most newbs (even for the already existing gzip and every other future compression method)
Reply
#43
I didn't say there's no solution, just that much more factors will have to be taken into account, checked for in runtime, and all this adds up to a meaningfully more complex solution.
Reply
#44
(05-08-2014, 03:19 PM)avih Wrote: Yeah, I know where the xz docs are Wink and I've also been reading many "chatters" before settling on gzip as the first attempt.

As I said, xz is possible but:

- Will generate tons of support requests because it needs special parameters during compression in order to make it viable for random access decompression, and since most people will want the smallest files, their xz files will end up as extremely slow in PCSX2 or not working at all -> "But I have a valid xz file and it's too slowwwwww... mommy...".

- And it needs more coding and integration, which always takes time.

Bottom line, if you're still trying to convince me that xz is an interesting option, you shouldn't, because I thought so too from the beginning, and still do. It just that everything takes time and on top of that, xz has a potential for other issues, as I noted above.

As far as xz, I offer this as a potential solution:

1. Advising the user on what settings to use to produce compatible xz
2. Prechecking the xz before launching it, and issuing an error if it's not a compatible one, the error would then also explain how to produce compatible ones

Obviously I'm not pushing the issue, but just suggesting a possible workaround if you ever decide to go this route!
[Image: XTe1j6J.png]
Gaming Rig: Intel i7 6700k @ 4.8Ghz | GTX 1070 TI | 32GB RAM | 960GB(480GB+480GB RAID0) SSD | 2x 1TB HDD
Reply
#45
I think supporting one compression format is enough for quite a while. Every new format brings complexity with it
and the benefit (arguably) is just a different file extension.
Reply
#46
(05-13-2014, 11:48 AM)rama Wrote: I think supporting one compression format is enough for quite a while. Every new format brings complexity with it
and the benefit (arguably) is just a different file extension.

I agree with you!Gzip iso compression works great and i don't know why change to a worst compression for pcsx2.....
Reply
#47
(05-13-2014, 11:34 AM)Blyss Sarania Wrote: As far as xz, I offer this as a potential solution: ...

The problem is not to think of approaches for the solutions. You guys seem to get back to this issue, but this is not a problem - all the "solutions" exist and are easy to come up with.

The issue is that it requires much more attention (and code) once you get to actually coding it, and has more pitfalls along the way. So it was not a good first choice. It's certainly a good second choice though, and it always was - once gzip is stable enough.

Please don't try to convince me anymore that lzma2 based formats can be used. I know they can. If you want to help, don't offer approaches, and instead code it yourself, else, take my word that it's a more complex solution and wait patiently until I maybe get to it.

(05-13-2014, 11:48 AM)rama Wrote: ... and the benefit (arguably) is just a different file extension.

Not necessarily. Some compression formats/algorithms do work better than others, and 7z/lzma(2) certainly beats the cr*p out of most other algorithms. The thing is that usually modern compressions come with higher CPU and memory usage, and much of the advantage they offer come from compressing much bigger blocks of data, which makes them really impractical for the random access which we need. They're great for archiving smaller files, but very bad for random access.

And the thing is, that once you change the compression parameters to be more suitable for random access, the compression ratio deteriorate very quickly. It may still end up better than gzip, but our (limited) experiments so far have shown that most of the time it's not _that_ much better than gzip, and the advantage is nowhere near the advantage you get with modern formats with their default compression parameters.

It would still be great to have them, but right now it looks to me like a lot of more work for not a lot of more benefit.

If we talk in numbers, then if gzip offers about 100% more space saving compared to ntfs compression (about 2x saved space), than my assessment of xz with appropriate compression params compared to gzip is maybe another 10% space saving on average. Certainly nice to have, but also certainly much more work for every extra % save.
Reply
#48
Thanks for writing out the argumentation Smile
Reply
#49
This is a really nice feature. Do you have a plan to add this to the UI in the future? Like right-click > compress.
Reply
#50
(05-13-2014, 01:39 PM)xemnas99 Wrote: This is a really nice feature. Do you have a plan to add this to the UI in the future? Like right-click > compress.

No such plans. The main reason is that PCSX2 is not a compression tool, and there are better dedicated tools out there to compress files. You're free to use any tools you like for compression, like 7z on windows or the gzip command line tool on other platforms. As for opening gzip files, I already added it to "the UI" - when you select to open an ISO file, it now also shows gzip files.
Reply




Users browsing this thread: 1 Guest(s)