TDM file format support info

[stgatilov]

On 1/9/2025 at 1:43 AM, datiswous said:

From info here it seems they both support transparency:
http://wiki.polycount.com/wiki/DXT

Yes, both DXT3 and DXT5 support alpha transparency.
I made an error there.

12 hours ago, datiswous said:

I don't know what you mean with this. There are whole wikipages about this and it's implemented in multiple missions.
You can also display video's as textures directly.

If you speak about fullscreen cutscene in the middle of the mission, then it is OK. We don't have some built-in support for this, but this can be done with full-screen overlay.

I'm talking about Doom 3 style videos on computer screen. This is not going to work well with FFmpeg videos, they are only designed for FMV case.

Quote

Do you have an example of this in a mission?

It is used for water in NHAT campaign, I think at the beginning of the second mission.

[datiswous]

12 hours ago, stgatilov said:

I'm talking about Doom 3 style videos on computer screen. This is not going to work well with FFmpeg videos, they are only designed for FMV case.

Isn't this the method described in this wiki tutorial?

https://wiki.thedarkmod.com/index.php?title=Full-Screen_Video_Cutscenes#Movie_Theatre_Method_2

This seems like a video as a texture on a surface.

[nbohr1more]

The lava in "The Gatehouse" uses videomap textures... Time to test I guess.

[stgatilov]

10 hours ago, datiswous said:

Isn't this the method described in this wiki tutorial?
https://wiki.thedarkmod.com/index.php?title=Full-Screen_Video_Cutscenes#Movie_Theatre_Method_2

Yeah, and then FFmpeg integration came.

I don't think FFmpeg is good enough to decode many videos at once. Maybe it is a bit more lightweight if you use ROQ, I don't know.

[OrbWeaver]

On 1/9/2025 at 2:46 AM, HMart said:

Afaik both are correct, I was taught to use dxt5 for normal maps 

On 1/9/2025 at 2:58 AM, datiswous said:

I see there's also support for BC5 (for normalmaps, since 2.10), but not BC7 (probably because it requieres a DX11 graphic card).

DXT5 for normal maps is obsolete. Both game and DR support BC5 (RGTC) normal maps, which are better quality and more efficient than DXT5 for this purpose.

On 1/7/2025 at 6:32 PM, peter_spy said:

Btw. IMO it would be useful to use .dds images in regular myFM/textures folder structure, so you don't have to replicate that in dedicated dds folder. TDM doesn't use texture quality settings anyway, it's a deprecated Doom3 thing.

I fully agree with this. Having to maintain two separate trees mirroring each other is an awkward hold-over from vanilla D3. It would be so much more ergonomic to just dump DDS files into the main textures tree along with TGAs or any other format. Unfortunately when I looked at the code to see if this could be improved, it seemed very non-trivial because of the way that the scanning of the DDS tree was bound up with the compression code.

[HMart]

1 hour ago, OrbWeaver said:

DXT5 for normal maps is obsolete. Both game and DR support BC5 (RGTC) normal maps, which are better quality and more efficient than DXT5 for this purpose.

Yep shows how old some of my knowhow is, thanks for the update.

Btw I did read something about AMD 3Dc format for normal maps being better than dxt5 ages ago, but was a AMD thing so I mostly ignored it and used normal DDS dxt5.

Also reading this, it seems 3Dc, is BC5 and is not very different from dxt5!  

http://wiki.polycount.com/wiki/3Dc

Is that correct? 

Edited January 19, 2025 by HMart

[OrbWeaver]

"3Dc", "BC5", "RGTC", "ATI2" are texture compression equivalents of "panther", "puma", "cougar" and "mountain lion": all different names for the same thing.

The article is correct, but you need to look carefully at the wording: "Each channel uses the same compression technique as DXT5 alpha". DXT5 has smooth alpha but blocky RGB, that's why vanilla D3 swapped the red and alpha channels for normal maps (so at least the red channel would be smooth, even though the green channel would still be blocky).

The idea behind BC5 is to use the same technique for both red and green channels, while leaving out the blue entirely (because it contains no additional information in a normal map, and can be reconstructed in the shader). This gives an effective compression ratio of 50% compared to the uncompressed image, without introducing the blocky artifacts of DXT1/3/5 normal maps.

[datiswous]

On 1/24/2025 at 9:59 PM, OrbWeaver said:

"3Dc", "BC5", "RGTC", "ATI2" are texture compression equivalents of "panther", "puma", "cougar" and "mountain lion": all different names for the same thing.

Source? I try to find info that directly specifies that. I can only find that "3Dc" and "BC5" and "ATI2" are the same.

I read that RGTC is BC4 and BC5

 

I guess @duzenko talks about it in this topic:

I actually don't really want to dig to deep into it (although it's kind of interesting), I just want to make sure the info on this page is correct:

https://wiki.thedarkmod.com/index.php?title=File_formats#DDS

Now it seems there's partly double info. I think using BC# is most useful, because it's easiest to find good info about it online.

 

 

Reading at the info here: https://www.reedbeta.com/blog/understanding-bcn-texture-compression-formats/#bc4 I would think that BC4 could be good for specular maps (or black and white diffusemaps), although maybe it doesn't matter?

Although if people normally use DXT5 (BC3) for specular maps, then BC4 is better, because it's the same size as  DXT1.

If there's full support for RGTC (which supposedly is the same as BC4 and BC5) then is BC4 also supported?

Edited January 28, 2025 by datiswous
it's easy to make mistakes with these format names

[datiswous]

So @stgatilov/ @nbohr1more does tdm support BC4 now as well, as part of RGTC ?

[nbohr1more]

2 hours ago, datiswous said:

So @stgatilov/ @nbohr1more does tdm support BC4 now as well, as part of RGTC ?

Only BC5 

 

[datiswous]

4 minutes ago, nbohr1more said:

Only BC5 

Ah that's a bummer, because I think it could be beneficial for texture size. Well ok

[nbohr1more]

BC4 is single channel. It would require another branch in the normal shaders and would probably look awful because the Y slope would need to be reconstructed from the x slope data.

Might be useful for grayscale heightmap input data though ( both for POM and Doom 3 formats ).

@OrbWeaver @stgatilov ?

[datiswous]

4 hours ago, nbohr1more said:

BC4 is single channel. It would require another branch in the normal shaders and would probably look awful because the Y slope would need to be reconstructed from the x slope data.

Might be useful for grayscale heightmap input data though ( both for POM and Doom 3 formats ).

Maybe I'm not understanding it entirally, but I thought they could be useful for specular and black and white diffusemaps. So only black and white use.

I think it's good if tdm supports these formats becouse missions are much bigger latelly and size of new missions will only increase.

Edited January 28, 2025 by datiswous

[stgatilov]

Yes, it seems that BC4 is just half of BC5.

It should not be too hard to implement it.
Maybe even use it automatically... e.g. for heightmaps. Or for all single-channel and grayscale images.

[OrbWeaver]

But aren't heightmaps only used by the heightmap() expression, which generates a new normal map? If the heightmap isn't directly rendered by a shader on the GPU, there's no benefit in using DDS compression.

I suppose there's no harm in supporting BC4 if it's easy to implement, but I doubt it's going to make much of a difference. How many actual black-and-white images are there? Specular maps aren't needed for most materials, they can be lower resolution than the diffuse and normal maps, and they're not necessarily monochrome. And wouldn't you need special handling in the shader to duplicate the single channel into all three RGB channels for BC4 images (but not do this for full-color images)?

Update: I just did a test with GIMP, and a DXT1 and a BC4 image are exactly the same size. So using BC4 doesn't achieve anything in terms of file size. What it gives you is smoother gradients for image types which are necessarily single-channel (e.g. heightmaps).

[stgatilov]

1 hour ago, OrbWeaver said:

But aren't heightmaps only used by the heightmap() expression, which generates a new normal map? If the heightmap isn't directly rendered by a shader on the GPU, there's no benefit in using DDS compression.

Automatic BC4 compression can be enabled for the new parallax maps.

Quote

I suppose there's no harm in supporting BC4 if it's easy to implement, but I doubt it's going to make much of a difference.

This is most likely true.

Quote

How many actual black-and-white images are there? Specular maps aren't needed for most materials, they can be lower resolution than the diffuse and normal maps, and they're not necessarily monochrome.

As far as I understand, most of the specular maps should be grayscale.
It does not mean we can enable the compression for all maps. But the loading code already checks whether all channels are the same and switches to single-channel format automatically if it is true, so it is possible to use BC4 instead in this case.

Also there are light falloff textures. But they are few, small, and they'd better be uncompressed.
Also there are light projection textures. Sometimes they are single-channel, in which case it makes sense to compress them it mapper wants to do it explicitly.

Quote

And wouldn't you need special handling in the shader to duplicate the single channel into all three RGB channels for BC4 images (but not do this for full-color images)?

Luckily, no. It is possible to set texture swizzle on the texture during creation, then it will automatically broadcast red channel to RGB on all fetches. That's how grayscale images work right now.

[OrbWeaver]

Oh, I was assuming we were talking about pre-compression of the images on disk. If it's just a case of dynamically compressing images at runtime which are known to be single-channel, then BC4 is a reasonable choice.

I still suspect there will be very few cases where it is actually useful, though. Dynamic compression of specular maps will only be useful if the source images are uncompressed — re-compressing DXT1 to BC4 will be worse than uploading the DXT1 directly. I haven't been following the development of parallax maps so if there are a lot of new dynamically-generated or uncompressed images, perhaps enabling BC4 for these will be more of a win.

[Guest]

On 1/28/2025 at 10:10 PM, stgatilov said:

As far as I understand, most of the specular maps should be grayscale.

Unless idtech4 works in linear space instead of gamma space (which I don't think it does), then not really. Grayscale works well for selected surfaces only, like silver, water, glass, and perhaps a few others. For conductive surfaces (e.g. raw metals), you need specular color map similar to your diffuse, to fake the effect of the specular hotspot being colored by that surface. For dielectric surfaces, you can use the trick with inverting the diffuse and playing with saturation to get both proper specular power amount and white specular hotspot. Unfortunately this is rather difficult to pull off due to unnecessary math added to specular calculations (like pseudo fresnel term).

Using greyscale speculars on all surfaces regardless of their purpose and color values makes them look wet or like plastic.

Edited January 30, 2025 by Guest

[HMart]

peter_spy is right specular maps don't necessarily need to be gray scale, Doom 3 has plenty of specular maps with color on it, see in the base/textures/hell folder for examples.  And I have used specular maps with color before as well.

A Quake 4 tutorial that imo shows how color in specular maps can be usefull. (hope the link works for all...)

https://web.archive.org/web/20160321000657/https://www.iddevnet.com/quake4/ArtReference_CreatingModels#head-4152af2ebdcdf51e21eaf84fd7ea3f511a7a1fab

Edited January 31, 2025 by HMart

[Guest]

I think this matches what idtech4 uses pretty well, and it's been a fairly common knowledge among asset creators from mid 2000s:

http://wiki.polycount.com/wiki/Specular_color_map

This is not exactly correct, as e.g. the wood grain shouldn't really be white, but the base color for the wood can be inverted for that purpose.

I got plenty examples of materials I made for TDM over the years, so I can give you more specific examples and comparisons if you want to

[datiswous]

On 1/28/2025 at 8:29 PM, OrbWeaver said:

Update: I just did a test with GIMP, and a DXT1 and a BC4 image are exactly the same size.

Yeah you can read that in the documentation:

https://www.reedbeta.com/blog/understanding-bcn-texture-compression-formats/#bc4

And there you can also read what the benefits are of the format.

 

On 1/28/2025 at 8:29 PM, OrbWeaver said:

So using BC4 doesn't achieve anything in terms of file size.

Quote

BC4 is the same size as BC1, but it gives much better quality than BC1 when storing a grayscale image

 

Edited February 2, 2025 by datiswous