Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Emulators & Decompressing ZIP/RAR/7zip
#1
I always thought to myself "If I have all of the ROMs extracted from the archives, it would probably make them run faster" but I just now thought of something. When an emulator decompresses a ROM dom a ZIP, RAR or 7zip file, does it extract it to a temporary folder on the computer or does it continually use the compressed file? Or does it decompress it and keep it in the RAM until you exit the game?
I guess what I realized is that once it's decompressed and loaded the game, there won't be any slow downs after that so it's only the initial loading of the game and then it runs fine meaning it is indeed better to keep ROMs in archives in order to save hard drive space?
OS: Windows 7 64bit
CPU: Intel Core i7 3770K @3.5 GHz
RAM: 16GB DDR3 1600MHz
GPU: Nvidia GTX 680 2GB
Reply

Sponsored links

#2
Depends on the emulator, and the size of the rom.

NES, SNES, Genesis, etc. roms can be decompressed into memory. No problems there.

Other systems with 64-256 meg roms could instead decompress to a temp file, although that can have negative side effects if the emu crashes a lot -- the temp files would get left behind, so the emu should have things in place to check for and clean up old tempfiles on startup.

Emus for CD/DVD-based games are pretty much forced to use "streaming" decompression (PCSX2 included). These emulators use a special block-level compression scheme that allows them to seek out blocks at a pre-defined resolution, typically between 32k to 256k per block. They decompress each block into memory as it's requested, and then discard it later. NTFS compression (built into Windows) works essentially the same way, albeit with some additional filesystem-level optimizations to further improve performance.
Jake Stine (Air) - Programmer - PCSX2 Dev Team
Reply
#3
Regarding modern multi-core CPUs, it should typically be possible to have compressed images that ad no extra overhead to the emulation process. It may not always be without overhead, however, since it's harder for an emulator to do efficient read-ahead prediction.

For example, these days most PC games use fully compressed resources on the hard drive. But they have the luxury of doing read-ahead of resources in a way that's tailored to compressed storage on hard drive, and allows for decompression of the resources on a background thread. Such a system on multicore and Hyprerthreaded CPUs would provide relatively free decompression cost. Smile

But on an emulator like PCSX2, the game is designed with CDVD access in mind, and uses a different system of synchronization and resource locks. Loading the blocks of the cd image on a separate thread is still doable, but generally not as efficient.
Jake Stine (Air) - Programmer - PCSX2 Dev Team
Reply
#4
"Streaming decompression" was the only way I really thought about it up until now. I have no idea why but I just didn't think of temp files or loading directly to RAM, but anyway thanks Air, I got the answer (more helpful than I expected) I was looking for~
OS: Windows 7 64bit
CPU: Intel Core i7 3770K @3.5 GHz
RAM: 16GB DDR3 1600MHz
GPU: Nvidia GTX 680 2GB
Reply
#5
not every compression method allows streaming decompression, just to remind
If you download the PS2 bios illegally, you are breaking the law. Courts around the world have ruled that businesses and individuals can be prosecuted for illegal downloading.
[Image: makemydaynow.jpg]


Reply
#6
(08-25-2009, 02:01 AM)ardhi Wrote: not every compression method allows streaming decompression, just to remind

They do if you manually implement block-level compression. This is precisely how the bzip2 compression of linuzappz cdvdiso works. The end result is, of course, a much lower compression ratio than would be achieved with the usual flat file compression... but yeah, it's pretty much possible with almost any base compression algo.
Jake Stine (Air) - Programmer - PCSX2 Dev Team
Reply
#7
(08-25-2009, 03:08 AM)Air Wrote:
(08-25-2009, 02:01 AM)ardhi Wrote: not every compression method allows streaming decompression, just to remind

They do if you manually implement block-level compression. This is precisely how the bzip2 compression of linuzappz cdvdiso works. The end result is, of course, a much lower compression ratio than would be achieved with the usual flat file compression... but yeah, it's pretty much possible with almost any base compression algo.

yes technically its, but im saying about how a compression software may compress the rom, a user might compress it with a 'solid acrchive' option, which not allows random access to a specific block (if it has more than one block)
If you download the PS2 bios illegally, you are breaking the law. Courts around the world have ruled that businesses and individuals can be prosecuted for illegal downloading.
[Image: makemydaynow.jpg]


Reply
#8
Solid Archive has nothing to do with whether or not the archive is accessible at the block level. For that matter, block level access (aka random access) is nothing to do with linear streaming.

Neither zip, bzip, 7zip, or even PNG file formats can access arbitrary blocks in their default state. Some of them can be effectively streamed once you've got the initial dictionary and headers downloaded, but that doesn't mean anything for the sort of random access an emulator needs for reading image data. In fact, the only block-level streaming algos I know of are multimedia ones: MP3, Ogg/Vorbis, mpg, wmv, flac, etc. Anything else requires special handling to allow for random access, which is why CDVDiso has a "compress/decompress iso" button, and can't just support .gzip/.bz2 images.

In other words: a user compressing their rom/image manually, with any option what-so-ever, is fail. Well, unless the emulator supports images compressed as .flac (should work, it's lossless Tongue2)
Jake Stine (Air) - Programmer - PCSX2 Dev Team
Reply




Users browsing this thread: 1 Guest(s)