Jump to content
The Dark Mod Forums

Texture file size project - reduce loading time and VRAM requirement:


Recommended Posts

Perhaps this is because TDM still uses monstrous 24-bit TGA format for normals and speculars, instead of 8-bit ATI2/BC5. Should TDM cease to use TGA (and all other formats, excluding DXTC/S3TC), it'll drastically reduce its loading time and VRAM requirement.

I assume thus must have been discussed before, what was the outcome? Is it as simple as saving in a better or other file format, that wont have any adverse effects?

 

The DDS folder uncompressed sits at 1GB and the Textures folder is at 1.9GB and the models folder Im guessing at around 500mb.

Link to post
Share on other sites

Perhaps this is because TDM still uses monstrous 24-bit TGA format for normals and speculars, instead of 8-bit ATI2/BC5. Should TDM cease to use TGA (and all other formats, excluding DXTC/S3TC), it'll drastically reduce its loading time and VRAM requirement.

 

This is what we want:

 

https://github.com/eXistence/fhDOOM

 

It has an Intel compatible Normal Map compression standard RGTC compression.

 

That said, I recently ventured a guess that the new GPL Compressonator might have better RxGB compression:

 

http://forums.thedarkmod.com/topic/18220-hmm-new-amd-compressonator/

Please visit TDM's IndieDB site and help promote the mod:

 

http://www.indiedb.com/mods/the-dark-mod

 

(Yeah, shameless promotion... but traffic is traffic folks...)

Link to post
Share on other sites

Did you ever get a chance to look at:

 

http://forums.thedarkmod.com/topic/18220-hmm-new-amd-compressonator/

 

? :)

 

I will get around to it myself once my other projects are settled.

Please visit TDM's IndieDB site and help promote the mod:

 

http://www.indiedb.com/mods/the-dark-mod

 

(Yeah, shameless promotion... but traffic is traffic folks...)

Link to post
Share on other sites

Speculars we can already scrap TGA. We cannot scrap TGA for normal maps until we have a viable replacement.

It's possible that RxGB is good enough now though...

 

One thing that wouldn't help with RAM but would help with loading times would be encoding the TGA files into uncompressed DDS

with mip-maps. Then the engine wouldn't have to "build mipmaps" every time it loads a level...

 

That might even help with malloc errors on level load since the mip-map generation process surely allocates tons of temporary memory

to do the conversion work.

 

Hmm, we could also do a hybrid approach:

 

Low-frequency\low-detal or blurry\smooth normal maps = RxGB compressed DDS

 

Detailed normal map = Uncompressed DDS

 

We might save enough with that effort to make any engine changes moot.

Please visit TDM's IndieDB site and help promote the mod:

 

http://www.indiedb.com/mods/the-dark-mod

 

(Yeah, shameless promotion... but traffic is traffic folks...)

Link to post
Share on other sites

I'll double check.

 

The last time I looked at it, I didn't even see RxGB as an option... it was RGxB or some other order of letters so it might not know

how to swizzle the color channels. It might be possible to swap the colors before the conversion then compress. That would be a pain.

Please visit TDM's IndieDB site and help promote the mod:

 

http://www.indiedb.com/mods/the-dark-mod

 

(Yeah, shameless promotion... but traffic is traffic folks...)

Link to post
Share on other sites

It can be public too. If we discuss it publicly, there's a better chance that an outside expert on texture formats will chime-in.

The rule of the internet is "If you want information, don't ask for it but instead state a bunch of misinformation\speculation\nonsense and someone will get angry enough to give you the good info."

Please visit TDM's IndieDB site and help promote the mod:

 

http://www.indiedb.com/mods/the-dark-mod

 

(Yeah, shameless promotion... but traffic is traffic folks...)

Link to post
Share on other sites

Yeah, they removed RXGB support. So much for improvement.

Still, if I can't get RGTC support working, we can try the tiered approach I mentioned above (RXGB for low detail, Uncompressed DDS for high detail).

Please visit TDM's IndieDB site and help promote the mod:

 

http://www.indiedb.com/mods/the-dark-mod

 

(Yeah, shameless promotion... but traffic is traffic folks...)

Link to post
Share on other sites

Nice :). Guys, what do you think about the Crunch Texture Compression? In a nutshell, the format that compressor provides (CRN) is DDS-like but extremely LZ-encoded: DXT1 (4 bits/texel) and DXT5 (8 bit/texel) can be squeezed to 1-1.25 bits/texel (i.e. up to 4-8 times), and ATI2 (8 bit/texel) to 1.75-2 bits/texel (i.e. up to 4-5 times); the compression is so quality, so you won't even notice the difference. CRN-files are transcoded directly into DDS on the fly, without recompression (i.e. the quality of image doesn't suffer), with speed at 100-250 megatexels/second (that was in 2012, now it's definitely faster). To integrate this format into your engine, you'll need to include just one standalone header file. BTW, Unity 3D already uses this format, so you probably already played some games with it.

Link to post
Share on other sites

Interesting and probably akin to what Serpentine wanted to do but I think this is above my skill level.

 

I'll take a closer look unless someone else gets to it.

Please visit TDM's IndieDB site and help promote the mod:

 

http://www.indiedb.com/mods/the-dark-mod

 

(Yeah, shameless promotion... but traffic is traffic folks...)

Link to post
Share on other sites

I'll take a closer look unless someone else gets to it.

That would be nice :)!

 

Our .DDS files arent much of an issue, but the .TGA normals are.

Agreed. I've found about 1500 TGA normals (*local*.tga) in the vanilla TDM 2.04, with ~2.2 GB in total. If you guys find a way to compress all of them into ATI2/BC5, you'll reduce them to ~0.55 GB in total. Disk space is pretty cheap these days, but loading (and downloading) time is of the essence.

 

BTW guys, do you know why TGA and all other non-DXTC/S3TC formats (BMP, PNG, JPG, etc) are loaded so slow? It's not just because of their size (which is partially true), but AFAIK, also because of the graphics driver that converts them into DXTC/S3TC on the fly, transparently to the application, no matter whether you toggle "image_useNormalCompression" and/or "image_useCompression" options on or off. Now imagine what happens when the game loads, say, PNG or JPG (I've found a plenty of them - over 1700): the DevIL library decompresses it consuming CPU time, then the graphics driver recompresses it, consuming even more CPU time. Perhaps I'm wrong, but if those JPG and PNG files are used in TDM itself, not in the Dark Radiant, it would be nice to convert them into DDS or even CRN.

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...