Compressing savegame file

[stgatilov]

The problem with savegame compression implementation it not the compression itself. Zip compression of memory buffer is done with ONE line of code. The problem is how to intercept data before it goes to hard drive. I have almost finished this damn piece of code for windows (and I hope for linux it'll be easier).

 

Choosing different compression is easy. Any compression library that can compress from memory buffer to another will be suitable. But remember that speed may become a bottleneck if you choose smth like 7z ultra . Default zip compression is good because it is fast. I've seen bzip2 compresses better that zip, but considerably slower.

 

 

[Aprilsister]

Yeah, just throwing it out there seeing as how I had occasion to run a comparison. I'm sure you are well aware of the wheres and whatnots, just throwing it out there. I really think this could be a great thing. Maybe getting rid of all that "commentary" will decrease the initial "loading" "loaded please wait" time too? I mean if it is unnecessary yet still being loaded somewhere into memory, huge chunks of it...

 

Anyway I'm really glad you have brought this savegame thing to attention.

[7upMan]

Well, there is one thing that speaks for 7zip: multi-core support. Since virtually everyone plays TDM on at least a dual-core machine, the ability to split the workload should increase the speed by a lot. Also, you don't need to use max compression, do you?

[OrbWeaver]

I understand that on Linux, the render image for the light gem is stored in /dev/shm, which is a shared memory mountpoint resulting in no disk access. If the uncompressed savegame could be written here too, it could then be compressed to the real disk in a separate step.

 

Of course, writing the file is easy; getting Doom 3 to recognise it as a savegame file, display it in the menu and pass it to the necessary code for decompression and loading might be a lot harder.

[stgatilov]

I understand that on Linux, the render image for the light gem is stored in /dev/shm, which is a shared memory mountpoint resulting in no disk access. If the uncompressed savegame could be written here too, it could then be compressed to the real disk in a separate step.

Yes, I is the way to handle lightgem on linux and I hope it'll be the way to implement savegame compression.

 

 

Does anyone knows windows equivalent of /dev/shm? It must be possible to open the file by filename with stdio functions. MSDN says that if FILE_ATTRIBUTE_TEMPORARY is set on CreateFile, the file is never flushed to hard drive. Maybe try it?

Edited December 27, 2010 by stgatilov

[i30817]

I wouldn't remove the saves commentary if it is helpful to debug corrupt saves.