Messin with shaders

[totallytubular 45]

3 days w/o power in texas. Froze my nuts off. But, power's back on.

Upgraded to 2.09, then messed with shaders some more. Mainly the glprogs/stages/interaction/ ... common & ambient, b/c they control bulk of lighting.

• broke out diffuse & specular lighting from each other again.. and figured out why specular was getting cut-off seams last time. It's because I didn't mul specular by light source, which takes attenuation into account. Did that this time.
• ambient.fs specular is generated from world light shining straight down, so was creating a spotlight under player on shiney floors (eg: gamma room floor on Training Mission). So, applied a 1-NdotL to have it use the light dot and not apply to surfaces perpendicular to the world light source (IE: ground).
• common & ambient .vs files didn't look like they were normalizing T,B,N before loading them into TBN matrix, so normalized them. (I think one was using a clamp, but switched it to normalize). Not sure it makes a difference or not.
• common.fs generated the normalized & TBN'ed "N" variable, but then used RawN in the NdotX calculations. Switched it over to using N instead. Not sure it makes a difference or not.
• ambient.fs was using AmbientRimColor to apply a fresnel highlight to final color, but after testing, didn't seem AmbientRimColor.a was ever > 0, so no fresnel effect was occuring. So, added an "else" branch to fallback on a generic fresnel highlight. (The velour-style fresnel I used in previous shader). Tweaked it to try not to look gaudy.
• Got rid of most of the fresnel effects I slapped in, or shifted them around a bit. The "eye adapt" was just half-arsed specular effect, so ditched it. The backlighting I just use on ambient.fs, but gets washed out a bit in darker levels and because specular is negated on floors in ambient.fs now. Still, it blends with fresnel to help out some. The velour fresnel was moved as alt fresnel effect in ambient.fs when AmbientRimColor not set or pitch black.
• switched ambient.fs to use halfdir Blinn-Phong specular like common.fs does.
• instead of just using generic default specular amount for things w/o specular texture maps, decided to BS specular maps for them by using the diffuse texture map.. did a dot(diffuse.rgb,0.33) to get diffuse lum and create a faux specular grayscale map which I then mul the default specular amount by to tone it.
• Also did that to create faux AO maps... initially used diffuse to do all AO maps, but noticed pillars in Training Mission looked awkward, b/c the AO was following the dark seams in the marble instead of the carved lines. So, if anything has a real specular texture map, used that for the AO map, and if it didn't, used the diffuse map as both faux specular and faux AO map. The faux AO blocks the ambient shadow light, but a final AOcolor is added to keep AO areas from being pitch black. This helps darken areas down some. Might be tacky in some places.

I've attached a .zip that contains interaction.ambient.vs, interaction.ambient.fs, interaction.common.vs & interaction.common.fs ... main files I farted around with.

Here's some screen shots.. noticed some issues...not sure if my messing around caused them or what...

ambient world light seems to always come straight down regardless of level designers placement of moon in sky.

This puts the ambient specular always underfoot like a spotlight. Without some kind of occlusion factor for it to know when player is indoors or not, dark dungeon floors get lit up with ambient world light. That's why I used a 1-NdotL to kill specular on floors. The ambient world light is still nice to make speculars shine on walls and objects. Just had to kill it on floors, like the one here in gamma room on Training mission. (This was done before I clamped specular color down to be a dull shine).

Some windows have a weird rectangle shape that doesn't blend well with the scene. It's like something that isn't controlled by ambient or common is not taking light attenuation into account. I experimented to see what ambient and common control, and there's some window textures that would stick out, which meant there's another shader being used to draw some textures. Couldn't seem to find it. I didn't spend much time digging.. but just looking in teh stages shaders, I didn't see anything else that could be doing that. The manylights shader doesn't seem to be used (?), so skipped messing with it. Not sure what's causing this.

Seeing how Faux AO map will impact things. It uses either specular map (if object has specular texture) or diffuse map otherwise. Basically just getting the dot luminance of either to create a single float value gray-scale to mul light dot by to add more darkness to things. A final AO color is added in to keep things from being pitch black.

Certain diffuse textures are missing .. something.. or processing is casuing this chunky issue. I noticed it intiially on some tile floors and windows on some levels.. in the distance the textures seemed to have a weird "acne" on them. But, when closing in, it would disappear. I thought it was specular. But, in testing fake AO map using diffuse, I noticed this. I have no clue what's causing it. A lot of these things blend into shadows, so don't notice them.. only reeally notice them in light (which is why I noticed them on windows and such). This one stuck out, b/c it's using the diffuse map as a fake AO grayscale map. Not sure what's causing it.

Fallback fresnel used when AmbientRimColor is 0 or not set. Tried to keep it subtle. It's more noticeable when moving around. Have to keep it subtle, b/c too much will make things in light appear ashy, and things in shadows look glowy. Too little, and it's as if it's not even there. Has to be just a little to help add depth to shadows. I figure enhancements are "death of a thousand" cuts... lots of little enhancements add up instead of one big honking one looking gaudy.

Fake AO. Most things don't have specular texture maps, so using diffuse map as fake AO means darker things darken up more.. hence the wall and grass are darker. Stone path is darkened in shadows more, too, though. With fresnel highlights and such going, the darkened areas still have depth.

 

This is a WIP. Just farting around, and trying to apply stuff I learned when dinking around with other game shaders, and experimenting to see what happens when I try different things.

glprogs.states.interaction.011.zip

[nbohr1more 2239]

Hmm, those artifacts may be due to bindless texture driver issues. Does r_useBindlessTextures 0 fix them?

I will test this tonight.

[totallytubular 45]

Got a pm from a user asking me how to install the shaders. I took it for granted folks staring at this part of the forums would know the source code break down and where to stuff them. So, here's some instructs for folks that don't know what to do with the files...

For Dark Mod v2.09, do the following...

1) go to your dark mod game folder (where you installed TDM, and where the .exe executables are)

2) make the following folders / folder path (this is important, b/c it changed for 2.09)

• glprogs\stages\interaction\

3) if you have those folders, and there's stuff in the "interaction" folder, then either zip up the original files there as a backup archive, or make a sub-folder like "\original" or something to put the following files...

• interaction.ambient.fs
• interaction.ambient.vs
• interaction.common.fs
• interaction.common.vs

If you don't have those folders or those files, then no big deal. It just means you don't have them (which means the game has been using whatever's in the pk3 resource files that come with it's install).

4) after making that folder structure and/or backing up your original 4 files, unzip the files from the zip in this thread into that "interaction" folder

5) start the game

The game should override pk3 resources with whatever loose files it finds in same folder structure and file name. So, firing up the game with those glprogs sitting out in that folder structure, it'll use those instead of what it has in the pk3's.

You should be good to go.

If you notice the torches don't emit light (pitch black) or the shadows are pitch black, it means something's borked up in the shader files and they're not compiling. (When the game notices a borked up shader file, instead of just crashing like a lot of DX HLSL stuff does, it skips it instead. Since those shaders handle lighting, pitch-black means they're being skipped and not doing the lighting.)

[totallytubular 45]

4 hours ago, nbohr1more said:

Hmm, those artifacts may be due to bindless texture driver issues. Does r_useBindlessTextures 0 fix them?

I will test this tonight.

I don't think it's bindless textures.

I modified the diffuse texture pull functions to make anything in bindless one bright color, and anything not using bindless another bright color... Basically...

• vec4 diff = (pull texture);
• diff.r *= 10.;
• return diff;

That sort of thing. Nothing seemed to use bindless textures. i went and commented out the normalization I added to the TBN matrix in the vertex shaders to see if maybe that was doing something, but nope.

The more I play, the more I notice it. I'm running through The Heart of Saint Mattis right now (jesus christ, great level!), and noticed very detailed textures look "spazzy", then I zoom in and notice it's because the texture has very distinct cut-offs between squares. EG: in the cemetary, the dirt plots over the graves. They're detailed grass or dirt. But, zooming in with spyglass I see stark lines where some pixels of texture seem to be missing or something.

Again, I don't know if it's something I did. Maybe using diffuse as fake AO is causing this. Not sure.

Another thing I'm noticing is that some textures don't have normals.. or, rather.. probably using the normal from the TBN matrix piped in from vertex shader. I notice it via the fresnel. Look down a wall at an angle, and it's like a sheet of glass with smooth fresnel even though it's brick. There's a sub-routine that does either normals * TBN, or just pulls normal from TBN to return as normal. I'm gonna experiment with that branch. If the normal from the vertex is being used, we might be able to branch and skip some of the processing that the full-bodied normal might use. I don't know. Gonna stare at them more later. Wife just came down with strep throat or something, so kind of busy being nurse maid.

[peter_spy 1615]

These artifacts also look similar to what you can get with either .dds or ingame compression. You can change these to 0 to make sure it's not that:

image_useNormalCompression "0"
image_useCompression "0"

[nbohr1more 2239]

I played around with this a little last night.

The ambient improves the appearance of some assets but makes the AI too shiny.

Full disclosure: my interest in this is because your original screens resembled the original TDM 1.03 ambient by JC Denton.

I am hoping that this work can lead to a "legacy mode" where missions like "Inn Business" that rely heavily on

the original Ambient Fresnel can look more like they were intended to.

For forward development, the TDM team is speculating about a PDR pipe-line similar to what RBDoom3BFG is currently working on:

https://github.com/RobertBeckebans/RBDOOM-3-BFG/commit/7f3208adb41d91c41879e2ed3f8eecd2a510ef93

Doom 3 Vanilla:

 

Doom 3 with PBR materials mixed in:

 

We already sorta have part of this up and working via the ambientCubicLight and our parallax corrected cubemaps can approximate the reflections:

But we really need to create a "proof of concept" branch with a proper PBR pipeline to determine how feasible it will be.

All this said, if nothing comes of this but slightly improved shaders compared to the current TDM design then we can still offer them

in the legacy part of the light pipeline and also put it up on Moddb where 2.09 (etc) players can enjoy them prior to whenever we

"possibly" move to a PBR model.

[VanishedOne 550]

Is AmbientRimColor here related to the ambientRimColor material keyword? I'm not aware of that having been used yet except in experiments.

 

[peter_spy 1615]

Moving to PBR will require making roughness maps for all materials. All the base color textures will have to be reauthored too, as they need to use correct intensity range. Since non-pbr TDM stock assets use specular maps in like 1% of the cases, changing from non-pbr to pbr will in essence change the art style of TDM.

Edited February 23 by peter_spy

[nbohr1more 2239]

Robert Beckebans reconciled this in Doom 3 BFG by assigning a default PBR behavior to all non-pbr materials so that the difference

in lighting would not be so jarring. It seems to have worked pretty well. I am not sure whether he also toned down the pbr response of

pbr materials to sorta "meet in the middle".

 

I think we would be OK but we would need to add a few artificial bias factors:

1) Probe lighting in the ambient would need to be limited in legacy maps to retain the darker look

2) Standard lighting in legacy maps would also need to multiply the falloff against the PBR method to limit the light radius closer to light sources

3) Probe based specular \ metal behaviors in legacy maps would be reduced and multiplied with a standard specular constant

 

That is, of course if we go for a hybrid model. We could just offer PBR as a new mapping mode and let mission authors choose to create assets

and missions in that new mode.

 

Whatever happens it needs to be studied for feasibility but Robert's approach shows that it can look great and mixing

asset types doesn't have to be catastrophic.

[nbohr1more 2239]

55 minutes ago, VanishedOne said:

Is AmbientRimColor here related to the ambientRimColor material keyword? I'm not aware of that having been used yet except in experiments.

 

Thanks for that! I guess I lost the plot thinking this was related to r_newFrob or for other special features. I'll tinker with Inn Business and see if this feature

is good enough to revive the look!

[cabalistic 861]

18 minutes ago, nbohr1more said:

Whatever happens it needs to be studied for feasibility but Robert's approach shows that it can look great and mixing

asset types doesn't have to be catastrophic.

A potentially significant difference, though, is that Doom 3 assets all have a coherent source and style. That makes it potentially easier to apply some kind of hacks to make them look decent when combined with PBR. This might be a lot harder to do for TDM, where assets come from various authors with different creation process and style.

Ultimately, though, all of this is pure speculation. We won't know anything until we actually experiment with it in a controlled fashion. Then, and only then, will we see what may or may not be possible and how best to move forward. But either way, before such experiments can happen, there is some ground work to be done, so I feel this is a discussion for another time

[nbohr1more 2239]

Right, I mostly added this to the topic since the author is doing lots of ambitious shader experiments and I wanted to

temper expectations about whether any of them would be included or part of our road-map.

 

Again, even if totallytubular's shaders never get merged they might still be a great add-on pack for players

to experiment with just like my torch glare mod.

[peter_spy 1615]

Don't get me wrong, it would make my model and material workflow like 10x easier I already own Substance tools, so with e.g. proper metallic-roughness workflow, I'd export all textures directly from Painter to TDM.

I was just thinking, since TDM doesn't use specular workflow much at all (most surfaces have kind of chalk or powdery properties), switching to PBR makes little sense if you plan to maintain current approach, i.e. mostly no roughness maps. Otherwise, TDM will change style in a way that my custom project differs from current stock TDM assets.

Edited February 23 by peter_spy

[VanishedOne 550]

2 hours ago, cabalistic said:

A potentially significant difference, though, is that Doom 3 assets all have a coherent source and style. That makes it potentially easier to apply some kind of hacks to make them look decent when combined with PBR. This might be a lot harder to do for TDM, where assets come from various authors with different creation process and style.

To add to that, even 'PBR' texture types aren't an assurance that you're actually getting something created with numbers based on physics. On Sketchfab I've seen models with 'metallic' window glass (for reflectivity, I suppose) and even metallic meat. (Well, we do have a quasi-mediaeval setting; maybe we could say it's adulterated...) I think it's just in the nature of the beast that TDM will never have as consistent an art style as projects with dedicated art teams.

[cabalistic 861]

Oh absolutely. And just to be clear, my interest in PBR isn't really in giving TDM a stylistic overhaul, "next-level graphics" or any such nonsense I'm more interested in if it could make our artists' lives easier, as peter_spy hinted. As an "industry standard" in some sense, it has better tooling and potentially a more consistent look across different scenes, which might need less tinkering. But again, that's all speculative

[totallytubular 45]

Replying to various points I see made...

The backup Fresnel would def look better with materials lighting toning it, b/c it's too monochromatic and even-looking. I thought AmbientRimColor was for materials, so when I noticed many things didn't use it I just figured they didn't have materials-based lighting implemented or their textures weren't prepped for it. Hence I chucked in the fallback fresnel. But, since the fallback fresnel is very monotone (monochromatic.. blue/gray haze on everything) and everything looks the same in shadows b/c no variations based on texture or materials I then tried applying the fake AO from dot(diffuse, 0.33) to add in variation to the fresnel (darker parts blocked more). But, lots of stuff just got their fresnel cancelled out when I did that. (EG: there's this dark brick texture on Saint Mattis FM in the underground .. it's so dark it just cancels out the fresenl if I mul it by fake AO). Tried doing a "if AO < 0.5, use 1-AO" to flip it, but it just went back to looking very monotone.

That brick texture was also the one I I thought had no real normal map applied, but when I just piped normals through I saw it did have a normal.. just a very flat-looking normal without much variation. (IE: not as bumpy as, say, a round-rock stone wall texture).

Looking down some walls, it looked like fresnel on a sheet of glass, but that was b/c the normal map on the texture was very flat. EG: wallpaper with the raised pattern..just looking down that wall the fresnel looks flat. But, zoom in with spyglass, and notice the fresnel is hugging the normal, it's just so faint that you don't see it unless zoomed in.

Since missions can be a hodge-podge of assets, the goal was to use the shaders as a "butter knife" smoothing frosting on a cake. If textures can do PBR / materials and other stuff, then they take those branches. If not, then they use fallback methods to try to still add some detail. In a perfect world, every object/texture in the game would have PBR / Materials, specular maps, AO maps, etc. But, it's not perfect. The great part about shaders is we can do hacky fallback stuff if the cool stuff is missing.

My idea is also to try to automatically offload some cool effects onto the shaders, so the FM designers don't have to think about it.

EG: a shader that controls water should always add some default reflection to the surface, or, better yet, do calculations to determine transmission vs. reflection when looking down at the water vs looking parallel to it. And, do some depth calculations to determine where water edge is for edge softening / blending. Maybe also add a default water color value to blend in, so all water isn't crystal clear. That way FM designer just has to slap some water in their level without having to worry about messing with it to make it reflective or realistic-looking.

Or, things like floors where the "tap tap" walking sound occurs, just have the game engine automatically use a shader that applies a slightly reflective cubemap on those textures to make tappy tile floors look polished and slightly reflective.

Just build in some default enhancements that the game engine & shader logic handles, so the FM designer can focus more on making a level instead of trying to mess around with making textures and objects look good.

I am still very much a nub when it comes to shaders. I hit my level of incompetence pretty quick when I start reading some of the white papers nvidia/amd publish on how to implement things in shaders. But, I like messing around with them, and like the idea of the shaders being able to make things look nicer automatically without level designers having to worry about micro-managing details like that.

The shaders I chucked up here are just a WIP of hacks to see if the shaders could help smooth out more enhancement. There's def better / official ways to do things (like PBR). But, also hack alternatives to fallback on that the shaders can do when that's missing.

[peter_spy 1615]

Having more controls over how water looks might be interesting, but automating tile floor reflections isn't a good idea IMO. As with most materials in non-pbr setup, you have to author specularity and cubemaps per your lighting setup. E.g. cubemap reflections look very pronounced in dark areas and washed out in intensely lit places. There's no one-size-fits-all solution to that.

[HMart 343]

On that Doom 3 shots that nbohr1more posted, can someone explain to me how PBR can do such a stark diference in light levels?! Is that really pure material PBR alone or there's some extra fake shader based ambient being done there? Is almost like with PBR on the scene gets radiosity bouces.

About PBR in TDM I know is just a "what if" but imo that would be fantastic.

And btw this is my idea how it could work, sorry if is stupid don't know much about this stuff. OK TDM would have two render paths, one with PBR support and the other vanilla, the PBR one is off by default and not used, unless, mission makers explicitly turn it on, somehow, perhaps by setting something on a config file.

They would be the ones responsible to make sure their missions assets, really support PBR, perhaps using a few converted default materials or making their own.

Why I recommend this, because IMO this way the engine team can, create a pure PBR render and not have to marry/couple the non-PBR and the PBR render together, IMO making the render more complex, more error/bug prone and perhaps making PBR less than what it can be. But this is only a uninformed assumption on my part, perhaps I'm just sub-estimating the abilities of the TDM engine team.  sorry guys.  

Edited February 23 by HMart

[totallytubular 45]

6 minutes ago, peter_spy said:

Having more controls over how water looks might be interesting, but automating tile floor reflections isn't a good idea IMO. As with most materials in non-pbr setup, you have to author specularity and cubemaps per your lighting setup. E.g. cubemap reflections look very pronounced in dark areas and washed out in intensely lit places. There's no one-size-fits-all solution to that.

Yeah, in the FUEL shaders, it was a b**** and a half getting the chrome/clear coat reflections to blend properly in light/shadow. Had to process the final diffuse with lighting / shadow first, then mix/lerp the cubemap in via fresnel that was impacted by shadow to make the lerp use more diffuse in shadows and more clear coat/chrome in light.

I noticed in TDM every now and then there'd be a glass object that's super-bright in ambient shadow light. Thought I'd take a crack at evening that out, but haven't found a shader that controls that yet, though.

I'm not sure what shaders are used / aren't used, or what they impact. So, I've just been dinking around with what I know impacts things so far.

Is there documentation somewhere that says what files control what, and are/aren't used?

[HMart 343]

Do you know c++? IMO you would gain much by looking at the engine source, to see how the render backend does the shader stuff, even if you have no intention of messing with the c++ code but if there's some docs better. 

Edited February 23 by HMart

[nbohr1more 2239]

The "lit" glass is probably a "blend add" or "blend blend" stage in the glass "material".

The material files are mostly in tdm_base01.pk4 under the materials folder with the extension mtr.

[peter_spy 1615]

Blend add mode is TDMs equivalent of emissive maps. I understood that totallytubular meant glass with cubemap reflection that is fairly bright, when put in area with ambient world only.

[peter_spy 1615]

1 hour ago, HMart said:

On that Doom 3 shots that nbohr1more posted, can someone explain to me how PBR can do such a stark diference in light levels?!

I can only assume the lighting setup was corrected to match textures that were adjusted for PBR. While non-pbr can use full 0-255 intensity range for diffuse (although it shouldn't), PBR base color needs to be between something like RGB 50 and 235.

[nbohr1more 2239]

3 hours ago, HMart said:

On that Doom 3 shots that nbohr1more posted, can someone explain to me how PBR can do such a stark diference in light levels?! Is that really pure material PBR alone or there's some extra fake shader based ambient being done there? Is almost like with PBR on the scene gets radiosity bouces.

RBDoom3BFG is capturing "probes" (cubemaps) of the "area" which are fed to the ambient light shader. A technique as old as HL2 at least.

[peter_spy 1615]

That technique didn't become widespread until parallax-corrected cubemaps and then Disney's BRDF solution became available. There were custom solutions, like for UE3 there was Remember Me game, but it only around UE4 we're talking about standards. Plus I guess hardware had to catch up as well.

Edited February 24 by peter_spy