PBR update of the Darkmod's character model

[Arcturus]

I tried this once years ago with a guard NPC. I noticed there have been ongoing discussions about PBR in Darkmod. So I put the builder priest on Sketchfab:

https://skfb.ly/pqrrC

I enabled model inspector, so you can take a look at all the texture maps. I haven't remade anything from scratch, mostly I brightened the metal parts of the diffusemaps and made metalness and roughness maps.

And here's rendered in Cycles.

 

[Arcturus]

If someone's interested, here's the .blend file with textures packed. I put in short comments in the nodes that describe what they're doing.

When in 'render' or in 'material preview' modes, you can preview each node on the 3d model by connecting it to the material output. Ctrl+Shift+LeftMouse should do that automatically in newer versions of the program.

At first I thought I would use Substance Painter, as this is the most popular app for texture maps creations. I have old version I bought on Steam, but I have some problem running it. For this purpose painting app isn't really required. In principle you could prepare all textures in an image editor. I made the masks needed in Gimp. I made one mask for the boot in Blender, because it was more convenient. However in case of complex models it is useful to have a render preview.

In vanilla Blender, exporting texture maps is a bit cumbersome. There's a "TexTools" plugin that makes this a bit faster. There are also paid plugins that try to emulate the Substance Painter interface.

You can see that some materials have additional stages: clear coat, sheen and subsurface scattering for skin. I didn't make any special textures for those.

I had to invert green channel of the normalmaps, as Blender uses different convention from Darkmod. All maps except "color" maps use "Non-color" space. Something worth remembering if one wants to work in Blender.

I used compressed .dds textures. There used to be a separate repository for original uncompressed Darkmod textures. Some may even have original sources with layers? That would help editing them.

I think that ordinary environment textures like wooden floors, or stone walls wouldn't be a problem when transitioning to a PBR model. More problematic are textures with multiple different surfaces, like wooden doors that have metal hinges for example, because they require manual masking.

It took me a moment to realize that the boots are made of leather bottom and metal top. There's a certain level of artistic interpretation. What material is the robe exactly. Is it very rough, or a little shiny? And how shiny should the metal parts be, etc.

[Arcturus]

Priest model's skinning was horribly mangled so I fixed it in the repository and on Sketchfab. It's still bad, but at least not fubar. The collar should be stiffer now and I glued the stole to the legs. It doesn't look great in idle animations but better when walking.

[Jetrell]

Very nice work. I'm glad you addressed the stole clipping problem, it was bugging me. It also looks like you fixed the hammer necklace clipping problem. Thanks for taking on this task.

[Arcturus]

@JetrellThanks. He still has shadowmesh sticking from his ass as he moves that needs fixing.

I've been thinking, since we're now living in the future we might use AI to create masks for us. I tried the 'Segment Anything' extension in Stable Diffusion using the AI model developed by Meta. There are also models from other laboratories to choose from. It has a range of settings, some of which didn't work for me at all, some worked poorly, and some that work quite well. Here's a best case scenario where it does a pretty good job at isolating different parts of the image:

There's also an option called 'Image Layout' that spits out separate image files for every detected feature. Those could be loaded for example into Krita as selection layers.

Of course it's no use for a more intricate details, especially that I'm using compressed .dds files which have additional noise.

 

[STiFU]

PBR sure would be a major upgrade for our game. Problem is, all our assets would have to be converted and that includes custom assets from FMs.

I do wonder, whether there is any research on automatically converting traditional materials to "good-enough" PBR-materials, or if we should experiment ourselves with such a process. I imagine a heightmap could be "approximated" from Normalmap (I know this is not accurate, but would maybe be good-enough for starting), metalness could be derived from thresholded specular map (or material name), glossiness could be derived from normalmap roughness and maybe specular map and as albedo we could simply use the existing diffusemap (maybe try to get rid of any brightnessvariations on that map).

@stgatilov care to comment?

[Arcturus]

@STiFUFully automatic conversion won't be possible. Closest you can get are the generic, tiling sets like the textures/darkmod/stone or wood. And that's because roughness and metalness in those cases can be defined by a single number. Just slap a metalness as a zero and roughness as something like 0.7 and it will be good enough. Later if you want to be fancy you can think about tweaking it further. Bigger problem are textures that combine different types of materials, e.g. metals with non metals, like many of the prop and character textures do.

Using existing specularmaps might be helpful, but not always. I was for example able to utilize the priestbottom_s.dds specularmap:

But I had to crank up the contrast in order to make the metalness and roughness maps:

In the case of priesttop_s.dds specularmap however, the original texture has this noise that makes it difficult to use:

That's why some manual work was involved in order to make the metalness nad roughness maps:

That's why I wonder if there are original PSD files in the repository somewhere.

Colors in general will need to be tweaked but metals especially. In Darkmod metals are generally dark with a bright specularmap added on top. Here's original diffusemap (I'm compressing to .jpg for the forum, btw):

Here's the map in my demo. Gold is much brighter and more saturated:

Parts of those textures are missing in my version, because different character models share some of the textures and here you see only parts used by the priest model.

We probably should take a look how the big boys in the industry are doing it. Some basic palette can be created as reference for things like shiny steel, gold, cast iron, copper, skin.

1 hour ago, STiFU said:

I imagine a heightmap could be "approximated" from Normalmap

Modern engines still use normalmaps. You would need heightmaps if you wanted to implement displacement or parallax occlusion mapping, but those are nice to have not mandatory.

Sketchfab has 'model inspector' where all the textures can be seen:

[STiFU]

Yeah, I was not very optimistic about my idea to begin with. So, we'd have to painfully go through all materials (and custom FM materials) and convert them manually.

Makes me wonder if it would be feasible to support both PBR and traditional materials in a somewhat efficient way in our renderer, so we could still have a functioning game while not all assets have been converted. @stgatilov?

[nbohr1more]

RBDoom3BFG solved this by figuring out by determining the approximate roughness and metalness that Doom 3 specular is supposed to convey so old materials would render with a similar characteristic but integrate well while new materials use new roughness and metal maps.

[HMart]

People can dream...  

But is not as simple as just updating a few materials or textures. 

TDM lighting interaction shader will have to be changed to be fully physical accurate, because afaik right now is not, then the engine will have to support some kind of automatic global reflection system (screen space or cube maps with probes) all to support the metalness system, something it still doesn't have, and that are jobs for one or more good graphics programmers, plus if cube maps, will the probes be automatic or manually placed? If manually (that i doubt...), who will place them on all existing missions? What about those missions where original makers are MIA? 

Then there's the unescapable fact that, it will take a huge amount of work to change all TDM materials (the material code as well) and textures to PBR and I mean all, down to particles, overlays, flares, the all shebang (unless done automatically like you guys are saying....).

I would love, to not be a doubter but unfortunately, I don't think this will ever happen, not unless a number of people on the TDM community, with sufficient knowhow, get serious about it and do it. 

Edited September 3 by HMart

[Arcturus]

@STiFU

Here's Unreal Engine 5 with just normalmaps plus diffusemaps connected to 'color'. Metalness is set to zero and roughness is set to 0.7 on all materials. It would look pretty similar to current Darkmod, except for the missing additional specularity on metal parts.

[datiswous]

On 9/3/2024 at 12:13 PM, STiFU said:

Makes me wonder if it would be feasible to support both PBR and traditional materials in a somewhat efficient way in our renderer, so we could still have a functioning game while not all assets have been converted.

Yeah that's what I think as well. Otherwise a new tdm has to be started. TDMpbr. Possibly a seperate branch in the installer.

Edited September 4 by datiswous

[nbohr1more]

3 hours ago, Arcturus said:

@STiFU

Here's Unreal Engine 5 with just normalmaps plus diffusemaps connected to 'color'. Metalness is set to zero and roughness is set to 0.7 on all materials. It would look pretty similar to current Darkmod, except for the missing additional specularity on metal parts.

can you test setting the roughness to the specular map?

[STiFU]

The main issue seems to be to derive metalness.

What does our current specular map do? It basically encodes reflection intensity at a constant roughness. Isn't reflection intensity basically metalness? 

[nbohr1more]

Metalness is specular color and reflection intensity

[nbohr1more]

I guess more accurate is that "reflection intensity" is more akin to "reflection blur amount". The more metal the less blur. ( And the more metal the more specular color, of course. )

[Arcturus]

@nbohr1more@STiFU

In real world the difference in appearance comes from the difference in electric conductivity. Polished metals are mirrors:

Modern mirrors are made by coating glass surface with aluminum or silver. James Webb space telescope uses gold.

Whereas if you set metalness to zero you get a ceramic looking material:

In principle specularmaps in Dark Mod should be equivalent to inverted roughness. The more white the spularmap values are the less rough the surface should be. In practice the specular effect in Darkmod was considered so strong, that most specularmaps were kept in the 0-0.5 range. Typically it's mostly metals that have specularmaps in Darkmod, but there are exeptions like the skin of NPCs.

In the priesttop_s.dds for example, only metal parts have specularity, leather belt is black. All values are kept from zero to about 33% of white.

Demonstration with a different lighting that shows the difference better.

All materials with roughness 0:

With specularmaps connected to roughness it looks practically the same:

With inverted (in GIMP) specularmaps it's very rough looking:

And with all materials set to 0.7:

[peter_spy]

IMO switching to PBR doesn't help here; if anything, it only emphasizes how dated the modelling and texturing is.

[Arcturus]

One possible path would be to invert all the specularmaps, change them to greyscale (some are in color), and crank the levels almost to the maximum:

This way you end up with roughness levels closer to what you would expect.

With the inversion of those maps connected to metalness sockets:

Skin has a specularmap so it ends up being slightly metallic. There are material parameters in Darkmod like "skin" that perhaps could be excluded. Another problem is as I said that color of metals in diffusemaps is a bit too dark.

[peter_spy]

Texturing has to be redone from scratch, if you want to make anything 'PBR' out of PBR. For starters, diffuse maps are base color textures in PBR, need to use certain range and have no AO information in them. Color range is like 50 to 240.

Edit: I see in the sketchfab model inspector, that you brightened the color map there. It looks pretty dark in the screenshots though, and as a final result too. Maybe using other some generic PBR materials would help instead? Working with existing TDM textures seem like a very wobbly foundation for any effort like this.

Edited September 5 by peter_spy

[Arcturus]

We can think of adhering to some standardized material table or we can take a look at textures in some latest games for reference when it comes to color. My assumption though is that we can't remake the assets and have to reuse as much as possible.

[peter_spy]

This should still be relevant: https://creativecloud.adobe.com/learn/substance-3d-designer/web/the-pbr-guide-part-2

[STiFU]

9 hours ago, peter_spy said:

IMO switching to PBR doesn't help here; if anything, it only emphasizes how dated the modelling and texturing is.

If we were able to convert our existing materials to "good enough" PBR materials, we could make the switch to a PBR renderer. This would allow us to successively improve our materials and new contributions could be made directly with PBR. 

If we had to first fix ALL materials manually up for PBR before making the switch, I don't think this small project would ever take off. That's the motivation for the investigation if the materials can be converted in a sensible way.

[Wellingtoncrab]

9 hours ago, STiFU said:

If we were able to convert our existing materials to "good enough" PBR materials, we could make the switch to a PBR renderer. This would allow us to successively improve our materials and new contributions could be made directly with PBR. 

I would just agree and add this would be a pretty huge efficiency gain for new stuff. There is always a lot of understandable worry about the amount of time it’d take to get old assets updated to work in PBR - but man does it also take a lot of time to convert the output from any modern tools from PBR into bump/spec for the game. That also means we are already making PBR assets for the game - we just can’t use them.

So yeah - if the time comes to go down that road the standard for converting old stuff should not be “perfectly physically correct” - it should be is it a close enough approximation of the original material - ie like @STiFUputs it - “good enough”.

This is pretty much what RBDoom3’s PBR conversion math does - if you look at screenshots it doesn’t suddenly turn Doom 3 into a game that looks like it was *authored* with PBR materials - it just gets the doom 3 materials into the PBR render within some approximation of the original so that everything will render whether they were authored with PBR in mind or not.

It is literally described as a “hack” in the code which uses magic math to convert the specular map into a roughness map. As far as I know it doesn’t even attempt to do something like make a metalness map. And hey - that actually looks pretty good!

https://github.com/RobertBeckebans/RBDOOM-3-BFG/commit/5854242d7b882d29c7e20253b29284072d6a7f74#diff-b2a7b195ed7c2c983755815d54e2f8e4fe2641b69ce37cb229fd52e545a9c409R2128

Of course Doom 3 is the product of a single art team - so things like specular maps are going to be utilized to much more of a standard than we will find in all our stuff. Not sure our math would be quite so magic.

But then almost any image to material software from the last decade could take a diffuse map input and provide a tweakable PBR result approaching “good enough”: bitmap2material, materialize, etc.

[peter_spy]

2 hours ago, Wellingtoncrab said:

This is pretty much what RBDoom3’s PBR conversion math does - if you look at screenshots it doesn’t suddenly turn Doom 3 into a game that looks like it was *authored* with PBR materials - it just gets the doom 3 materials into the PBR render within some approximation of the original so that everything will render whether they were authored with PBR in mind or not.

Speaking of which, did you take a closer look at those screenshots? It's not like the switch to PBR, even with the same legacy materials, left the levels unchanged. They're brighter due to different material response I guess. Second thing is either some generic cubemap fallbacks, or generated env probes for the materials – Robert did the latter in RBD3BFG. This will increase mission package size substantially. This is not just change in the aesthetics, you'd also have to change existing missions.

Edited September 6 by peter_spy