Should we consider using detail textures?

[MirceaKitsune]

There's been talk over the years on how we could improve texture quality, often to no avail as it requires new high-resolution replacements that need to be created and will look different and add a strain on system resources. The sharpness post-process filter was supposed to improve that, but even with it you see ugly blurry pixels on any nearby surface. Yet there is a way, a highly efficient technique used by some engines in the 90's notably the first Unreal engine, and as it did wonders then it can still do so today: Detail textures.

Base concept: You have a grayscale pattern for various surfaces, such as metal scratches or the waves of polished wood or the stucco of a rough rock, usually only a few highly generic patterns are needed. Each pattern is overlayed on top of corresponding textures several times, every iteration at a smaller... as with model LOD smaller iterations fade with camera distance as to not waste resources, the closer you get the more detail you see. This does wonders in making any texture look much sharper without changing the resolution of the original image, and because the final mixture is unique you don't perceive any repetitiveness! Here's a good resource from UE5 which seems to support them to this day:

https://dev.epicgames.com/documentation/en-us/unreal-engine/adding-detail-textures-to-unreal-engine-materials

Who else agrees this is something we can use and would greatly improve graphical fidelity? No one's ever going to replace every texture with a higher resolution version in vanilla TDM; Without this technique we'll always be stuck with early 2000's graphics, with it we have a magic way of making it look close to AAA games today! Imagine being able to see all those fine scratches on a guard's helmet as light shines on it, the thousands of little holes on a brick, the waves of wood as you lean into a table... all without even losing much performance nor a considerable increase in the size of game data. It's like the best deal one could hope for!

The idTech 4 material system should already have what we need, namely the ability to mix any textures at independent sizes; Unlike the old days when only a diffuse texture was used, the pattern would now need to be applied to both albedo / specular / normal maps, to my knowledge there are shader keywords to combine each. Needless to say it would require editing every single material to specify its detail texture with a base scale and rotation: It would be painful but doable with a text injection script... I made a bash script to add cubemap reflections once, if it were worth it I could try adapting it to inject the base notation for details.

A few changes will be needed of course: Details must be controlled by a main menu setting activating this system and specifying the level of detail, materials properties can't be controlled by cvars. Ultimately we may need to overlay them in realtime, rather than permanently modifying every material at load time which may have a bigger performance impact; We want each iteration to fade with distance and only appear a certain length from the camera, the effect will cause per-pixel lighting to have to render more detail per light - surface interaction so we'll need to control the pixel density.

[nbohr1more]

@Mortem Desino already implemented detail textures into No Honor Among Thieves and his tech demo:

https://wiki.thedarkmod.com/index.php?title=Detail_Textures

[MirceaKitsune]

10 hours ago, nbohr1more said:

@Mortem Desino already implemented detail textures into No Honor Among Thieves and his tech demo:

https://wiki.thedarkmod.com/index.php?title=Detail_Textures

Interesting! Does it update all default textures so it's used on everything in the world? I should replay it and check that out: It would give us a good view of how the effect will feel in practice.

Looking at the page, they seem to do it the conventional way I was thinking of trying out, which is currently supported by the engine but more limited than a proper implementation. It also looks like they're only doing it for the albedo channel, to be effective detail should be applied to all maps... the normal map is where the improvement should be most noticeable as it responds to lighting and modifies everything else.

The implementation I'm thinking of should be universal like all effects and work on any FM new and old. It would be controlled by a menu setting, no one needs to enable it if they don't like how it looks or it impacts performance. Each detail pass should fade and be hidden with distance, we don't want to stress pixel lighting by having it compute thousands of dots on distant surfaces each frame. Just like the TDM ambient method, we'll likely need a special segment for materials meant to indicate what kind of detail each texture wants, then based on settings and camera position the renderer must modify each surface accordingly.

[The Black Arrow]

Alright, so, I'm a Texture Artist myself for more than 20 years, which means I know what I'm talking about, but my word isn't law at all, remember that. I've worked (mostly as mods, I am a professional but I much prefer being a freelance) with old DX8 games up to DX12.

When it comes to Detail Textures, for my workflow, I never ever use it except rarely when it's actually good (which, I emphasize on "rarely"). This is one reason I thought mentioning that I worked with DX8 was logical.

One of the few times it's good is when you make a game that can't have textures higher than what would be average today, such as, World Textures at 1024x1024. Making detail textures for ANY (World, Model) textures that are lower than 128x128 is generally appealable.
Another is when the game has no other, much better options for texturing, such as Normal Maps and Parallax Mapping.

Personally, I think having Detail Textures for The Dark Mod is arguably pointless. I know TDM never had a model and texture update since 2010 or so, but most textures do seem to at least be 1024x1024, if there's any world texture that's lower than 256x256, I might understand the need of Detail Textures.

Now, if this was a game meant to be made in 2024 with 2020+ standards, I would say that we should not care about the "strain" high resolution textures add, however, I do have a better proposition: Mipmaps.
There are many games, mostly old than new ones, that use mipmaps not just for its general purpose but also to act as a "downscaler".
With that in mind, you boys can add a "Texture Resolution" option that goes from Low to High, or even Lowest to Highest.
As an example, we can add a 2048x2048 (or even 4096x4096) world texture that, if set to Lowest, it would use the smallest Mipmap the texture was made with, which depends on how the artist did it, could be a multiplication of 1x1 or 4x4.
One problem with this is that, while it will help in the game with people who have less VRAM than usual these days, it won't help with the size. 4096x4096 is 4096x4096, that's about 32mb compressed with DXT1 (which is not something TDM can use, DXT is for DirectX, sadly I do not know how OpenGL compresses its textures).

I would much rather prefer the option to have better, baked Normal Maps as well as Parallax Mapping for the World Textures.

I'm still okay with Detail Textures, I doubt this will add anything negative to the game or engine, very sure the code will also be simple enough it will probably only add 0.001ms for the loading times, or even none at all.
But I would also like it as an option, just like how Half-Life has it, so I'm glad you mentioned that.

But yet again, I much prefer better Normal Maps and Parallax Mapping than any Detail Textures.

On another note...Wasn't Doom 3, also, one of the first games that started using Baked Normal Maps?

[MirceaKitsune]

@The Black Arrow That's a good analysis. I don't disagree but we're referring to different time periods with different quality aims: In the early days of 3D and low-res CRT screens when we had 256x256 textures, detail textures were used to make surfaces appear as if they have 1024x1024 textures... today in the age of 1080p monitors such texture can appear blurry from up close, we want to make 1024x1024 textures appear of 4096x4096 quality. Back then the goal was to get at least a little bit of perceived sharpness, today the goal is to see those microscopic details on every surface as if everything is real... while the concept of detail textures is old it scales to cover both aims.

As you correctly pointed out, the ideal solution would be upgrading the actual textures themselves. Sadly there are two big problems with this that will likely never be possible to overcome:

• Someone must create or find identical textures to replace existing ones, which have to retroactively fit every old FM. That would be a huge effort for so many images, and will not look exactly the same way so people would complain how "this wall used to be made of small red bricks which are now larger and yellower which isn't what I intended and no longer line up". An advanced upscaling filter may be able to bump the resolution with good results, this would be a lot less effort and retain the exact appearance of textures.
• The even greater issue is storage and memory use would go through the roof. Imagine all our textures (from surfaces to entity skins) being 4096x4096 which would be the aim for decent quality today: TDM could take over 100 GB of drive space, you'd need at least 16 GB of RAM to run it, and the loading time of a FM will be 5 minutes.

Detail textures are a magic solution for both problems: They're overlayed in realtime on top of the standard textures without changing their base appearance. This means you see pixels several times the scale of the image without requiring any image to actually be at that resolution, no vRAM or loading time increase. And if detail layers are disabled with distance you also don't lose FPS in per-pixels calculations when distant lights update.

Edited April 15 by MirceaKitsune

[The Black Arrow]

I always say that...The only thing that beats a fail is a try.

I'm actually interested in these Detail Textures now that you further talked about its history and how today we'd use it on 1080p for 512 and 1024 textures.

If anything, it would be a fun experiment just to see how it would look and how it would go.

Also, yeah, I wouldn't want the game to be bigger than 100GB and have extreme loading times, all in the name of modern textures at 4k, I have a very hefty computer but that doesn't mean poor ol' but venerable IdTech 4 wants to handle it.

[nbohr1more]

I think that this discussion is probably similar to discussions that idSoftware themselves had about the challenges of texture storage in engines that heavily rely on Normal Maps for real-time lighting. The conclusion was Megatextures ( later known as partially resident textures ) but the suggestion was a little too ahead of its time. Heck, early Megatexture games would probably benefit from detail textures more than idTech4 because they capped the pixel density to allow larger map-sized textures. Many modern games have caught up and use partially resident textures but do so in a more conservative way thus making them part of a hierarchy of texture usage methods that includes texture atlases and traditional tiled textures.

[The Black Arrow]

25 minutes ago, nbohr1more said:

I think that this discussion is probably similar to discussions that idSoftware themselves had about the challenges of texture storage in engines that heavily rely on Normal Maps for real-time lighting. The conclusion was Megatextures ( later known as partially resident textures ) but the suggestion was a little too ahead of its time. Heck, early Megatexture games would probably benefit from detail textures more than idTech4 because they capped the pixel density to allow larger map-sized textures. Many modern games have caught up and use partially resident textures but do so in a more conservative way thus making them part of a hierarchy of texture usage methods that includes texture atlases and traditional tiled textures.

Wasn't "Megatextures" the reason Doom 2016 (IdTech 6) takes years to load though, even on a high-class SSD?

I've then heard that Megatextures were scrapped out in Doom Eternal (IdTech 7), which apparently could add to the fact it loads incredibly fast.

[MirceaKitsune]

3 hours ago, The Black Arrow said:

Wasn't "Megatextures" the reason Doom 2016 (IdTech 6) takes years to load though, even on a high-class SSD?

I've then heard that Megatextures were scrapped out in Doom Eternal (IdTech 7), which apparently could add to the fact it loads incredibly fast.

Megatextures were a horrible idea for obvious reasons, not sure why ID chose to learn that the hard way. The concept from what I remember is the whole map uses a single gigantic texture... instead of how we independently pick a couple of 1024 px brick materials for a few brushes and surfaces, the whole map acts as one model with one material and a single texture which probably needs to be 1 million x 1 million pixels even for a small level.

This is ridiculous from a perspective of system resources with 100's of GB's of storage and huge (v)RAM requirements and hours of loading time, as well as raising the skills required for level editing since you now need mappers to also be texture artists and sculpt / paint their levels instead of just placing stuff. The only thinkable benefit is there's no repetition since every pixel on every part of the world is unique, but who notices any similarity with independent texturing if it's done right anyway?

Detail textures have yet another advantage there: Since you scale the pattern independently on top of the original texture, you can make every surface appear as if it has unique pixels like megatextures. Hence why I'd advice having the details be very high-res, 4k or 8k even 16k if we can take it: Yes that's enormous, but remember we'd only have a few patterns probably no more than 15 in total, and can store them as grayscale then use a single image to modify both albedo / specular / normal (heightmap to normalmap): Map the detail in world space rather than the brush or model UV map, and the resulting pattern on every surface in the world will always be unique since the original and detail textures will be out of sync.

[nbohr1more]

Rage didn’t have terrible loading times but people complained that it took awhile for the texture fidelity to catch up. Doom 2016 tried to fix that by front loading more but they went overboard with that.

[nbohr1more]

19 minutes ago, MirceaKitsune said:

Megatextures were a horrible idea for obvious reasons, not sure why ID chose to learn that the hard way. The concept from what I remember is the whole map uses a single gigantic texture... instead of how we independently pick a couple of 1024 px brick materials for a few brushes and surfaces, the whole map acts as one model with one material and a single texture which probably needs to be 1 million x 1 million pixels even for a small level.

This is ridiculous from a perspective of system resources with 100's of GB's of storage and huge (v)RAM requirements and hours of loading time, as well as raising the skills required for level editing since you now need mappers to also be texture artists and sculpt / paint their levels instead of just placing stuff. The only thinkable benefit is there's no repetition since every pixel on every part of the world is unique, but who notices any similarity with independent texturing if it's done right anyway?

Detail textures have yet another advantage there: Since you scale the pattern independently on top of the original texture, you can make every surface appear as if it has unique pixels like megatextures. Hence why I'd advice having the details be very high-res, 4k or 8k even 16k if we can take it: Yes that's enormous, but remember we'd only have a few patterns probably no more than 15 in total, and can store them as grayscale then use a single image to modify both albedo / specular / normal (heightmap to normalmap): Map the detail in world space rather than the brush or model UV map, and the resulting pattern on every surface in the world will always be unique since the original and detail textures will be out of sync.

The Megatextures concept predates idSoftware. It even predates hardware accelerated 3D in consumer graphics. The problem it solves is managing image diversity. It is well suited to photogrammetry where almost nothing is repeated \ tiled.

It is not well suited to human texture artists because even the most perfectionist ones are going to use some sort of template that they repeat and thus it is a waste to store the resultant work as unique pixels rather than tiles.

[HMart]

1 hour ago, MirceaKitsune said:

Megatextures were a horrible idea ...

Completely disagree it was a very good idea that unfortunately came too early with idSoftware RAGE 1 because hardware was still and is not truly ready for it. 

Megatexture is for textures, what UE5 nanite is for geometry, it permits huge amounts of texture data in a level and even totally unique textures per each surface, no repetition unlike older methods.

MT isn't simple a giant texture slapped unto a level, it is a type of virtual texturing system, the MT is a huge "texture" but it's data, is not all drawn on a level at the same time, like a normal one is, it is broken into smaller squared peace's and those peace's into even smaller ones for LOD aka mipmapping, then those peace's are streamed in, in real time has the player moves around. 

Again IMO It is a very cool idea and if the hardware was fast enough, to stream it fast and it was possible to compress the MT data, to very small amounts and still keep good quality, it would be a fantastic system for very unique levels. Rage 1 world may look bad at close distance, but at medium to large distances imo it looks fantastic and totally unique.

 

Edited April 27 by HMart

[wesp5]

2 hours ago, nbohr1more said:

Rage didn’t have terrible loading times but people complained that it took awhile for the texture fidelity to catch up.

Yes, if you turned too fast, you could see the textures pop in. Most Unreal engine games still do this when you start...

Edited April 27 by wesp5

[MirceaKitsune]

23 minutes ago, HMart said:

Completely disagree it was a very good idea that unfortunately came too early with idSoftware RAGE 1 because hardware was still and is not truly ready for it. 

Megatexture is for textures, what UE5 nanite is for geometry, it permits huge amounts of texture data in a level and even totally unique textures per each surface, no repetition unlike older methods.

MT isn't simple a giant texture slapped unto a level, it is a type of virtual texturing system, the MT is a huge "texture" but it's data, is not all drawn on a level at the same time, like a normal one is, it is broken into smaller squared peace's and those peace's into even smaller ones for LOD aka mipmapping, then those peace's are streamed in, in real time has the player moves around. 

Again IMO It is a very cool idea and if the hardware was fast enough, to stream it fast and it was possible to compress the MT data, to very small amounts and still keep good quality, it would be a fantastic system for very unique levels. Rage 1 world may look bad at close distance, but at medium to large distances imo it looks fantastic and totally unique.

 

Oh, some implementations might work a little differently from what I remember the term megatexture referring to. From what I used to know, it meant turning the entire level into a single model or set that uses a single enormous texture.

While the concept may have its upsides, there are two major issues that negate any benefit in my view: The first is system resources, you don't benefit from any reuse as every pixel is unique, the only way to do it at scale is with a gigantic image thus a huge performance drop in pretty much every department. The second issue is that level design becomes far harder and more specialized... while here in TDM we only need to draw a bunch of brushes and place some modules to make a level, an engine based on megatextures would require level designers to sculpt and paint the entire world in software like Blender which is far more difficult and we likely wouldn't have even half of the FM creators we do today, even for those that know how to do it imagine the task of manually painting every brick on every home and so on.

[HMart]

37 minutes ago, MirceaKitsune said:

Oh, some implementations might work a little differently from what I remember the term megatexture referring to. From what I used to know, it meant turning the entire level into a single model or set that uses a single enormous texture.

R: The level does use a single enormous texture but is not a texture in the usual sense per se, afaik a MT is a buffer of pixels arranged and optimized in a way to be streamed in fast possible.  

While the concept may have its upsides, there are two major issues that negate any benefit in my view: The first is system resources, you don't benefit from any reuse as every pixel is unique, the only way to do it at scale is with a gigantic image thus a huge performance drop in pretty much every department.

R: Where do you got that idea? Afaik performance is exactly the same as for normal textures?! You only render the necessary tiles, for the scene you see on screen at any giving time as use the necessary vram, you can even see on the video I posted that a Nintendo 64 can handled it fine!

If you are talking of the blurriness when moving fast, that IMO is not performance, is more speed of streaming in the textures, it only affects visuals not movement performance, but this was worse on the HDD days thou, today with SSD's and M2 drives, IMO things have improved a ton. 

Also I played most of all modern Wolfenstein games, Dishonored 2 and the first modern Doom game, all of them use MT and are imo, very performant games. 

The second issue is that level design becomes far harder and more specialized...

R: This is true indeed.

while here in TDM we only need to draw a bunch of brushes and place some modules to make a level, an engine based on megatextures would require level designers to sculpt and paint the entire world in software like Blender which is far more difficult and we likely wouldn't have even half of the FM creators we do today, even for those that know how to do it imagine the task of manually painting every brick on every home and so on.

R: Yes you are right but IdSoftware remade their own editor to be able to paint in real time unto surfaces and slap overlays/decals very easily all around, they add no need to force artists to do it on other tools like Blender, thou based on what I know about Seneca (one of the id's artists) for example he wouldn't mind working on a external tool, he already did it for Rage. 

But anyway in reality, this tech was made for pros, for big teams, not single developers like TDM mission makers, even John Carmack (the guy that invented MT) said that when Rage 1 came out, he said it would be very hard to mod Rage, if not impossible, because of MT.

Btw you can slap repeated textures unto surfaces with MT, there's nothing preventing that, it just gave artist's the option to make every surface unique, by being able to easily paint detail all over with no fear of texture limits, if they wanted too, and in some ways they did that in other games and engines as well, even without MT, for example Fallout 3 world, was "painted" all over using a special decal system. 

I concede that there's problems with MT, but IMO the real problem with it, is not on the user side, or was performance, quality or being a bad made tech, it was just speed of streaming in, size and was hard to handle. Specially for indie developers but later even for AAA teams, it was found to hard to handle for PBR and real time lighting.

Spoiler

MT was good for games with baked lighting, because most surface data/info could be baked into a single huge texture, but modern games, more and more require real time lighting and also require, a bunch of more texture data, the data to make a single surface looks, on many modern AAA games, is mind blowing, there's the diffuse/albedo map, the normal map, the metal/specular map, the roughness map, the AO map, the height map, the gloss map, the subsurface map, the reflection/cube map, etc, etc, with MT, many of this textures, would need their own separate MT, this to work with real time lighting. This would be extra GB of texture data on top of GB and so, even more compression needed. And baking a Rage 1 MT, was already a time consuming process and needed a PC farm to do it, imagine today.

This is why even idSoftware ended abandoning some of the MT ideas, like the baking huge textures workflow but not all of it, the virtual texturing itself and texture streaming the most important part of the tech anyway, is still used, afaik UE5 still uses VT and texture streaming as well, and this can work with individual textures as well.

But just saying that MT was a horrible idea because its limitations, is also imo not fare or true, because it also brought some good ideas to the table, it was a evolution process and like all such things the final incarnation can end very different but it needed the initial state to start somewhere. Enough said.

Ps. sorry for the huge wall of text....