DDS Texture discussion

[sparhawk]

OK. I get started on it. Didn't have time, but hopefully over the weekend.

[greebo]

@sparhawk, I just noticed that the actual shadernames did get changed by the script as well:

 

textures/darkmod/nature/grass/thick_straw01_nontiling

 

which should be:

 

textures/darkmod/nature/grass/nontiling/thick_straw01 (i.e. unchanged, same as before)

 

edit: Not all shader names got changed though, but quite a bunch.

[sparhawk]

Uh! How many are there? If it's only a few can you fix them manually in the meantime?

 

I'll fix the script, later.

[greebo]

I think there are 20 or 30, I can't say exactly, but I can of course fix them manually.

[sparhawk]

That woulod be great. Otherwise I will have to fix the script. Rollback the changes, and then apply them. I can do that if you want, but 20-30 sounds in the range of being acceptable for manual processing.

[greebo]

It's already done, it was not too much and I could quickly locate them. I just finished syncing the textures, so I can start checking the textures and rename them to DDS.

[sparhawk]

Great! Thanks!

[greebo]

Can please somebody try to combine two DDS bumpmaps via addnormals() and see if it works? This is rather important.

 

It doesn't work on our end, and I got a bad feeling in my stomach about this. It's possible that DDS textures can't be used in MapExpressions.

[Springheel]

I'll test it if you spell out how to do it.

[greebo]

Please take two DDS normalmaps and create a new material containing an addnormals() expression as bumpmap, like this:

test_shader
{
 diffusemap textures/darkmod/any_diffusemap
 bumpmap   addnormals( FIRST_DDS_HERE,  SECOND_DDS_HERE )
}

 

Then test this shader in any map and see if Doom3 can load it.

 

The second test is to use just one of the two DDS files as bumpmap (without the addnormals() thingy) and see if it works.

[Springheel]

Hmm, I tried it and it appeared to work, but I can't really tell if both normal-maps are actually showing up. I think I'll need to try it with a more distinctive normalmap.

[greebo]

I think I've converted all of our diffuse, specular and normalmap textures to DDS now, which should have cut the size of our regular textures down to roughly a third of the space the TGA textures have been using.

 

Now the only problem is the one with the addnormals() which mostly affects all of the stone_sculpted textures from angua and me, as addnormals() hasn't been extensively used by the rest of the shaders. I will try to do some more tests and get this to work.

 

Springheel, could you test this again, please? I want to make sure if this is caused by a different configuration or is another Doom 3 limitation.

 

Plan B would be to bake the two normalmaps into each other and create separate images.

[Springheel]

Ok, I confirmed it. Addnormals does not work with dds normalmaps for some reason.

[greebo]

Hooray. We seem to frequently run into these limitations lately...

 

For our sculpted textures it probably makes sense to leave them as TGA, as this takes less space than the various combinations of baked sculpted+roughness images. *sigh*

[Springheel]

Oh Christ, I goofed. The test I just did wasn't with .dds normals, it was just with regular .tgas (which leaves me wondering why it didn't work...).

 

But now I'm having a new problem. Is anyone else having trouble getting .dds normals to open in photoshop? At the moment they just open as solid black images. Can't confirm right now whether they work in game or not, but it's going to be pretty difficult to modify them if they don't show up in PS.

 

I had trouble with the normals when I was doing the reorg as well...they always opened as weird distorted colours, but I thought NH said something solving the way they had to be compressed.

[New Horizon]

Oh Christ, I goofed. The test I just did wasn't with .dds normals, it was just with regular .tgas (which leaves me wondering why it didn't work...).

 

But now I'm having a new problem. Is anyone else having trouble getting .dds normals to open in photoshop? At the moment they just open as solid black images. Can't confirm right now whether they work in game or not, but it's going to be pretty difficult to modify them if they don't show up in PS.

 

I had trouble with the normals when I was doing the reorg as well...they always opened as weird distorted colours, but I thought NH said something solving the way they had to be compressed.

 

The .dds normal maps have to be created and opened with the compressonator, but that's really only for viewing them. If you want to edit a dds file, you would have to use the original uncompressed tga, which is what you would want to do even with the diffuse, because saving the dds again in PS would further compress it and you would end up losing a lot of the quality each time you resave it.

 

I do wish there was a proper Photoshop plugin to create .dds normals for doom 3.

[greebo]

I just requested the Photoshop SDK from their website (those people are a bit anal retentive and make you actually fill out a written request the SDK and want to know what you intend to to with it). I don't think that it's easy to write a plugin, but perhaps I can be bothered to play with it. Probably not, but I'm curious about the SDK anyway.

[sparhawk]

Did you create a label before you deleted all these TGAs? I have the feeling that we might need to reinvoke them, considering how problematic these DDS's seem to be.

[greebo]

No, I did not create a tag. But a look into the log makes it easy to see when I started to convert the textures.

 

I created a mirror of the current folder structure in the hi_res repository and uploaded all TGAs as backup to it, so retrieving any source TGA files from before the conversion is easy.

[Springheel]

So wait, if you want to make any modifications you have to go to the hires repository to find the originals? Wow, that's inconvenient. It seems like every time we turn around there's another hoop to jump through for the sake of these .dds files....

 

I hope we're not planning on using .dds for the model textures, as I'm constantly referencing and making new versions based on the old.

[greebo]

For the DDS normalmaps, this is the way to go, yes. These are compressed using the RXGB format, which isn't supported by the nVidia plugins. The other ones (diffuse and specular) can be opened just fine in Photoshop, although the uncompressed textures are of course better to start from for quality's sake.

 

It's not that bad though. You can also keep your own copy of the hires repository on your hard drive to have the files handy in case you need them.

 

I actually wanted to convert the model diffusemaps as well, but I won't do that of course if there are objections.

[Springheel]

Let's hold off doing the model textures for now and live with the brush textures for a while. Who knows what other hoops we might find.

[Springheel]

Ok, great. Now that I've finished updating, all the normalmaps for textures are screwed up.

 

 

I remember this being a problem in the past with dds textures. Did we find what the solution was?

[greebo]

These are my relevant DoomConfig.cfg settings, be sure the have normalCompression set to 2, this might be the reason (it enables the use of RXGB DDS compression for normalmaps):

seta image_ignoreHighQuality "0"
seta image_usePrecompressedTextures "1"
seta image_useNormalCompression "2"
seta image_useAllFormats "1"
seta image_useCompression "1"

[sparhawk]

I noticed that you copied the tga's to the hires, and I'm not really happy at the moment. The repository is already pretty big and now we have duplicated the tga's into two repositories. I don't know if it would be possible to delete the history of the files while retaining the history itself. this would shrink the main mod repository quite a lot.