Converting model .tgas to .dds

[Springheel]

It looks like it's about time to take this on. Changing our model .tga textures to .dds files could have a significant impact on loading times/performance for the release map.

 

I have too many other things on my plate to tackle this any time soon, however. Is there anyone else that has some time that wouldn't mind taking this task on? A script that would do it automatically would be fantastic, as there are a few hundred textures.

[New Horizon]

It looks like it's about time to take this on. Changing our model .tga textures to .dds files could have a significant impact on loading times/performance for the release map.

 

I have too many other things on my plate to tackle this any time soon, however. Is there anyone else that has some time that wouldn't mind taking this task on? A script that would do it automatically would be fantastic, as there are a few hundred textures.

 

Just the diffuse and speculars, or normals too?

[Springheel]

The normals need to be left as is, just like the texture files.

[Tels]

The normals need to be left as is, just like the texture files.

 

As I am currently working on the package script and teaching it automatically package textures, one part of it's task will be identifing textures - and for that it needs to figure out DDS vs TGA version.

 

From that it should be only a small step to call ImageMagick and do the conversion automatically and check the DDS version into SVN.

 

Question: Does it hurt if both DDS and TGA are in SVN? (I believe we already have that case, but can't prove it at the moment).

[Springheel]

Question: Does it hurt if both DDS and TGA are in SVN? (I believe we already have that case, but can't prove it at the moment).

 

It won't break anything, if that's what you mean. D3 chooses which one to use based on your video settings. But there's not a lot of point in having both in SVN, since people then have to download both when they first download the mod, and it could create some confusion if people only modify one version.

 

What we will probably do is move the .tga versions to the highres repository.

 

From that it should be only a small step to call ImageMagick and do the conversion automatically and check the DDS version into SVN.

 

Do you mean it would convert the .tgas to .dds? That would be great, if so. How soon would that be available? It would be nice to see what kind of performance impact that would have.

[Tels]

It won't break anything, if that's what you mean. D3 chooses which one to use based on your video settings. But there's not a lot of point in having both in SVN, since people then have to download both when they first download the mod, and it could create some confusion if people only modify one version.

 

What we will probably do is move the .tga versions to the highres repository.

 

Okay.

 

Do you mean it would convert the .tgas to .dds? That would be great, if so.

 

I don't know if I can do the conversion automatically. I know I can do it manually with Gimp, but haven't tried automatically.

 

How soon would that be available? It would be nice to see what kind of performance impact that would have.

 

Dunno, I am currently hacking at the script (see devel/extract-assets.pl and the releated SVN checkins). Hopefully will be done in a few hours.

[Tels]

Okay.

I don't know if I can do the conversion automatically. I know I can do it manually with Gimp, but haven't tried automatically.

Dunno, I am currently hacking at the script (see devel/extract-assets.pl and the releated SVN checkins). Hopefully will be done in a few hours.

 

Btw, the progress is tracked here:

 

http://forums.thedarkmod.com/index.php?showt...st&p=153102

 

Currently I am teaching the Games::D3::Parser to correctly parse material definition files. Once it can do this, it will be easy to extract the diffusemap/specular/bumpmap for each material and then figure out the DDS vs. TGA file from it. (That's the theory, anyway

[AluminumHaste]

You can get the DDS plugin for photoshop made by NVidia, and use photoshops batch tools to automate the process. I'll try it right now.

[Tels]

You can get the DDS plugin for photoshop made by NVidia, and use photoshops batch tools to automate the process. I'll try it right now.

 

I don't think this will work on Linux.

[AluminumHaste]

Alright I just found a tool with the source available (it's really small) that converts tga and bmp to DDS (DXT3 w/Mipmaps), I just converted 27 megs of textures down to 4.5 megs.

 

Get it here: http://www.flipcode.com/archives/Batch_DDS...rter_Tool.shtml

You can see the source for it in there and also the source code comes with the download. Try it it works great. I checked in photoshop and the textures are saved as DXT3 with mipmaps.

[Tels]

Alright I just found a tool with the source available (it's really small) that converts tga and bmp to DDS (DXT3 w/Mipmaps), I just converted 27 megs of textures down to 4.5 megs.

 

Get it here: http://www.flipcode.com/archives/Batch_DDS...rter_Tool.shtml

You can see the source for it in there and also the source code comes with the download. Try it it works great. I checked in photoshop and the textures are saved as DXT3 with mipmaps.

 

Thats good to know, but the first step is to know _which_ TGAs to convert to DDS and do so automatically.

 

Edit: The tool you posted does not compile on Linux, it is for Windows only.

 

It is all great if someone does it manually now, but we will have to redo that everytime manually.

 

If you would just let me finish my script so I can detect the files automatically.........

[AluminumHaste]

If you would just let me finish my script so I can detect the files automatically.........

 

Lol okay okay

[oDDity]

You only need dxt3 if there is an alpha, otherwise dxt 1 will do, which is half the filesize. A 1024x texture should be around 600kb.

[Tels]

You only need dxt3 if there is an alpha, otherwise dxt 1 will do, which is half the filesize.

 

Ah thanx. Hopefully this can be determined automatically, too.

[AluminumHaste]

You only need dxt3 if there is an alpha, otherwise dxt 1 will do, which is half the filesize.

 

I thought also that DXT3 had less compression artifacts?

[oDDity]

NAh, you barely notice the difference.

Also, are mip maps really necessary, since you can shave another 200kb off that by not having them.

Doesn't sound much, but over 1000 textures, that 200mb saved.

[Tels]

NAh, you barely notice the difference.

Also, are mip maps really necessary, since you can shave another 200kb off that by not having them.

 

I think they are, because the graphic card will generate the mipmaps anyway. (which could contribute to even more on-load-stuttering, if sneaksies poor graphic card needs to generate mipmaps for 50+ textures

[AluminumHaste]

I think they are, because the graphic card will generate the mipmaps anyway. (which could contribute to even more on-load-stuttering, if sneaksies poor graphic card needs to generate mipmaps for 50+ textures

 

Doom 3 won't show the textures in game unless you generate the mipmaps before hand, at least that's what happened to me when I tried making them without mipmaps.

[Springheel]

It is all great if someone does it manually now, but we will have to redo that everytime manually.

 

Why? Generate them once manually, upload them to SVN, problem solved.

[AluminumHaste]

NAh, you barely notice the difference.

Also, are mip maps really necessary, since you can shave another 200kb off that by not having them.

Doesn't sound much, but over 1000 textures, that 200mb saved.

 

You're right, I can't tell the difference, the two DXT formats look almost exactly the same.

[Springheel]

If we're going to do this as a batch, we'd have to use DXT3, unless someone wants to manually open each texture to see if it uses any transparency.

[New Horizon]

If we're thinking about compressing the normal maps as well, then they have to be compressed with the compressonator, unless there is some way to get doom 3 reading a standard normal map.

[Springheel]

I thought we'd already established that we can't compress the normalmaps, or else they won't work with addnormals passes?

[Tels]

If we're going to do this as a batch, we'd have to use DXT3, unless someone wants to manually open each texture to see if it uses any transparency.

 

That was my second comment about "automating it", sorry if it was to confusing. I am pretty sure ImageMagick can tell you automatically if analpha channel was used or not.

 

But let me first finish the parser and the script, and then work on the conversion.

 

(Currently the parser comes along nicely, now I just need the script to actually use the parsed information, but my head is like broccoli tonight - damn fever

[SneaksieDave]

For size considerations, it's worth checking. Or at least, erring on the side of making everything DXT1 and then finding the broken ones which need fixing to DXT3. It's just a matter of seeing it screwed up in game and jotting it down, or the manual method of opening it (easier and foolproof, though perhaps more tedious). Anyone can help out doing either; no reason to suddenly accept bloated sizes.

 

if sneaksies poor graphic card needs to generate mipmaps for 50+ textures

I really don't think it's a matter of the card, it's the 1.4GHz processor. The card is a GF6800 - older, but no slouch by any means, and even higher than a few other users' cards. But when the difference in launch time (D3 to main menu only, or DR with no map) between my 1.4GHz CPU and my 3.0GHz CPU is on the order of 3-4x longer at least, that's the key factor. Or, compiling a map on the 3.0 in <6 mins, but >20 mins on the 1.4. So the card is probably sitting there waiting to be handed the next set of textures. And once they're all cached, and as long as they don't go beyond some magic number in size, it runs without issue. The loading is slow, the swapping is slow. Live rendering is fine.

 

Edit:

I am pretty sure ImageMagick can tell you automatically if analpha channel was used or not.

It might be the fault of PS6.0 (known to have some bugs/limitations with alpha channels), but I see at least a blank alpha for what seems to be every darkmod DDS I open. Just so you're aware.