Textures: The Final Verdict...i Hope. :)

[New Horizon]

I was speaking to Gildoran about using .dds compressed textures a few days ago.

 

Now, before anyone freaks out and jumps in to remind us all how horrible they looked in T3, we have to remember that the T3 textures were compressed to hell and we're not looking to do that. We would be using the highest quality .dds formats.

 

Why am I bringing this up now? Well, it's based on some research I was doing about TGA, TGA RLE compressed (what we use now), and .DDS.

 

At the time, TGA RLE was chosen because it allowed us to have fairly high res textures, but at a reduced size. This kept downloads reasonable and it also kept the mod from ballooning up to a ridiculous size. Uncompressed 1024 X 1024 TGA's are 3 to 4 megs a pop. So, we reduced them to a maximum of 512 x 512 and used lossless TGA RLE. Yay...or so I thought.

 

I have since learned that video cards have no idea what to do with RLE compression, they can only read 'standard' TGA, so the textures has to be reinflated. That means that our 512 x 512, 700 kb texture is reinflated to anywhere from 1 to 1.2 megs in size. The conversion process increases loading times...something we had hoped to lessen with the smaller textures.

 

We are still gaining 'somewhat' of a savings, but the real kicker is this. A 1024 x 1024 DXT5, .dds compressed texture with mip maps, will be roughly the same size as a 512 x 512 reinflated TGA. Video cards handle .dds natively...so there are no conversion issues, and the mip maps allow for better performance since the mip maps get used when the player is further away from a surface.

 

What does this mean for us? Well, we'll be able to use full sized textures now and I think it's pretty easy to force doom 3 to use 'only' .dds textures.

 

Some of the settings we can play with.

 

seta image_downSizeLimit "256"
seta image_ignoreHighQuality "0"
seta image_downSizeBumpLimit "128"
seta image_downSizeSpecularLimit "64"
seta image_downSizeBump "0"
seta image_downSizeSpecular "0"

 

Now, if people chose lower detail settings...doom 3 would reduce the size of the .dds textures and compress them further. To prevent them from looking horrible, we would just set higher limits in the settings above. I would say 512 for imagelimit and bumplimit. Specular limit could be 256.

 

Anyway, I think this is the best way to go after researching the subject.

[Gildoran]

Happily, DDS compression shouldn't have any effect on mappers - we should be able to experiment and find the right level of compression without them having to retexture any maps.

[SneaksieDave]

So, erm... file size on HD stays the same, and resolution in-game goes up? I'll have to see it give this benefit with no hit to performance to believe it, though...

[New Horizon]

So, erm... file size on HD stays the same, and resolution in-game goes up? I'll have to see it give this benefit with no hit to performance to believe it, though...

 

The final size of the files will increase a little bit, as I was saying above, TGA RLE compressed textures aren't recognized natively by video cards, so they must be reinflated to standard TGA, thus ballooning them up to a meg or more for use in video memory. It's a waste since the TGA textures had to be dropped to 512 resolution to reach the smaller size in the first place. It also means that loading times are increased by the reinflation.

 

DDS on the other hand 'is' natively recognized by video cards, so there is no reinflation...they're just used as is. As I said in my original post, DXT5 compression will yield a 1024 texture at roughly 1 to 1.3 megs...with mip maps, and from my understanding..mip maps will help with displaying longer distances since the surfaces will use the smaller images stored in the dds files as you get further away.

 

I haven't worked it all out the standards we should use for everything yet, but I'll post once I do.

[SneaksieDave]

So the textures will be potentially doubled in file size. Or in TDM-currently-still-very-small-texture-repository terms, ~300Mb will grow to ~600Mb, and grow from there. Urk.

 

It's to be expected, sure. Games have on the order of gigs of texture data today. But damn, oh well. A "lite" version might remain viable nevertheless.

[Domarius]

I sure hope your server is a good one Spar...

[sparhawk]

The discspace is certainly no problem. I have about 80% unused right now.

[Dram]

Well this is good news i guess. Lucky I uploaded those hires ones prior to deleting them from my HD

[Redface]

So, what is the final verdict ? Are there going to be new texture guidelines or should I stick with good old 512x512 TGAs at the moment?

[Gildoran]

512x512 or 1024x1024 is fine, whichever you think fits the texture better. I need to get around to making a sticky post about how I'd like people to send in textures though...

[New Horizon]

512x512 or 1024x1024 is fine, whichever you think fits the texture better. We think we can use dds files to decrease the memory usage enough that 1024x1024 is ok. I need to get around to making a sticky post about how I'd like people to send in textures though...

 

Original versions of textures should be maximized to whatever you see fit and in TGA format, but they don't go into the main repository..we upload those to a separate hi-res repository for release as a separate package. As Gildoran was saying, we're going to see if we can do a switch over to .dds files completely. The reasoning for this is quite simple, the lossless compressed TGA files that we initially aimed to use, are not recognized at all by video cards, so Doom 3 has to reinflate the TGA's to their uncompressed size. This adds to loading time, and also decreases ingame performance. DDS textures 'are' recognized by video cards natively, so they are loaded as is and don't increase load times. We can also get away with 1024 X 1024 .dds textures that will give us roughly the same ingame performance as a 512 x 512 TGA.

[OrbWeaver]

Perhaps a bit late to chip in to this discussion, however a couple of points:

 

1) RLE-compressing TGA is a waste of time since the thing is going into a PK4 anyway. Based on some testing I did, zip-compressed uncompressed TGA reaches a smaller final size than zip-compressed RLE TGA, so the RLE-decoding/reinflation need not be an issue.

2) Although I see the point in using DDS in order to increase resolution, one needs to be mindful of using the right tool for the right job. Although DDS can look fine on noisy, detailed textures it can look ghastly on very smooth textures with diagonal lines, for example. In the latter case it might actually be better to use zip-compressed TGA rather than having to add noise before DDS compressing in order to avoid the artifacts, thus defeating the point of compression in the first place.

[Gildoran]

I've heard a similar suggestion to 1) for distributing pk4 files... Since pk4s can't make use of better compression algorithms such as 7zip, they suggest storing files uncompressed in the pk4, and storing that in a compressed archive for distribution.

 

Don't worry, I don't intend to require everything to use dds files... I just intend to use it as a default unless the texture looks bad with DDS images.

[OrbWeaver]

I've heard a similar suggestion to 1) for distributing pk4 files... Since pk4s can't make use of better compression algorithms such as 7zip, they suggest storing files uncompressed in the pk4, and storing that in a compressed archive for distribution.

 

Interesting, although there is obviously a disk space issue there, unless the end user recompresses the uncompressed PK4 on his own machine.

 

Don't worry, I don't intend to require everything to use dds files... I just intend to use it as a default unless the texture looks bad with DDS images.

 

OK, that's what I was nervous about. As long as there is scope for allowing TGA if it works better for a particular texture then I see no problems with using DDS by default.

[New Horizon]

The thing to remember about the issues with DDS in T3 is that the team used 'in most cases' the lowest resolution they could get away with and the highest compression possible. That's why they look so horrible. For our textures, we're talking about going no lower than 512 for diffuse and normals. The majority of textures will no doubt be 1024, and if we're using the correct methods used in D3, they should look fine. Prey is using dds extensively and it looks beautiful, it's just a matter of only compression to the level you have to.