[2.11] Stencil shadows optimizations

[stgatilov]

Historically, there were several inefficiences in the way stencil shadows were combined with antialiasing.

1. The first improvement landed in 2.10, and fixed most of the slowdown (not even tracked, I suppose).
2. The second improvement landed in the just released dev16617-10107 (5851), and further reduced bandwidth.

Now turning antialiasing on increases stencil bandwidth by constant multiplier.
So regardless of scene, the different between antialiasing Off and 2x and between 2x and 4x should be more or less the same.

Another improvement is about soft stencil shadows (6076), also landed in dev16617-10107.
It can be turned on and off by cvar r_softShadowsMipmaps (turned on by default).
Unfortunately, this cvar can improve performance, but sometimes it can worsen it too.

It would be great if players could test it in various scenes and various hardware and report their results.

Here are the gory details about the new optimization:

Spoiler

I know that everyone would be more excited about shadow maps, but I decided to start from stencil because it is way easier to implement for them Similar approach can be applied to shadow maps too.

As you probably know, with soft shadows on, GPU samples some pixels in a vicinity of the rendered one in order to determine its shadowed ratio. This sampling costs quite a lot of performance, which you can easily notice if you crank soft quality up. This sampling is done for all pixels on screen, even the ones which are very far from penumbra regions, and even if there is no penumbra at all.

dev16617-10107 includes tile-based culling, which reduces amount of sampling in typical cases.
Basically, it splits the screen into small "tiles", then precomputes for each tile whether the whole tile is shadowed, the whole tile is lit, of tile is of mixed type. When soft shadows are rendered, we first fetch a few tiles and check if they are all fully lit or fully shadowed. If that's the case, then we don't do any full-resolution sampling.

Basically, it removes the cost of sampling in the regions which are far from penumbra... but with the cost of having to precompute tiles information. And this precomputation is not free, especially when there are many lights in the scene.

 

[AluminumHaste]

Sounds fantastic, checking it out now.

[datiswous]

@Dragofer should be added to

?

Maybe then more people see it. It seems a bit forgotten currently.

[MirceaKitsune]

This improvement is only for stencil shadows correct? I believe I used shadow maps previously... likely because they were faster at the time, and I think they're the only mode that allowed shadows to grow softer with distance from the caster rather than having a constant softness? If stencil supports the same features as maps now and are expected to be faster or as fast, I may use it with this optimization! I'll likely test at least if it helps, probably not tonight as I'm first getting re-accustomed with the 2.10 release.

[stgatilov]

44 minutes ago, datiswous said:

should be added to [feature discussion threads]

I think not, because it is not a feature, it is optimization.
Neither players nor mappers really have to know about its existence.

The reason for this thread was to check what beta-testing players would say about performance: it is better/worse and how much.

25 minutes ago, MirceaKitsune said:

This improvement is only for stencil shadows correct? I believe I used shadow maps previously... likely because they were faster at the time, and I think they're the only mode that allowed shadows to grow softer with distance from the caster rather than having a constant softness? If stencil supports the same features as maps now and are expected to be faster or as fast, I may use it with this optimization! I'll likely test at least if it helps, probably not tonight as I'm first getting re-accustomed with the 2.10 release.

This is for stencil shadows only, not applicable to shadow maps.

Stencil shadows do have larger penumbra at larger distance from light: it is distance to obstacle which is ignored by stencil shadows but used by shadow maps. This technical limitation is hard to workaround without shadow maps.

[datiswous]

1 hour ago, stgatilov said:

I think not, because it is not a feature, it is optimization.

I think optimization is a feature, like faster load times (2.10) is a feature as well, but can also be seen as optimization.

Edited October 23, 2022 by datiswous

[MirceaKitsune]

I upgraded to latest dev earlier due to what I wrongly perceived as a bug with 2.10. Decided to test this and I'm glad I did: Enabling r_softShadowsMipmaps breaks stencil shadows and causes them to stop working! Stencil shadows work once setting the cvar to 0, whereas map shadows always work fine. I'm on Linux (Manjaro / KDE) using the default amdgpu driver, I know some issues are specific to it so it may be why.

Here's two comparison screenshots, first being normal and the other broken. Note that when I pressed F12 to take it, I was in reality seeing the the scene just with no shadows... only now noticed everything came out black with only the HUD and entities showing! Will share it here anyway in case it helps.

[MirceaKitsune]

Here's a proper screenshot of the broken version, managed to catch it using system printscreen instead.

[nbohr1more]

Did you reloadModels after changing shadow modes? Did you restart TDM after the change?

[MirceaKitsune]

9 minutes ago, nbohr1more said:

Did you reloadModels after changing shadow modes? Did you restart TDM after the change?

Just set the option to stencil and did a full restart: Same issue, no shadows with stencil and the new setting turned on.

[nbohr1more]

Did you re-run tdm_installer ?

Did you try deleting tdm_base01.pk4 and re-running tdm_installer to ensure you have the latest glprogs?

Did you make sure there is no glprogs folder in your darkmod root directory?

What GPU driver do you have installed ?

[MirceaKitsune]

16 minutes ago, nbohr1more said:

Did you re-run tdm_installer ?

Did you try deleting tdm_base01.pk4 and re-running tdm_installer to ensure you have the latest glprogs?

Did you make sure there is no glprogs folder in your darkmod root directory?

What GPU driver do you have installed ?

A forgotten glprogs directory was causing colors to go crazy, removed that before I started testing. The update went well but just to be extra safe I did that cleaning as well, same issue after reinstalling.

GPU driver is amdgpu, the default installed and used by the OS (not pro version). This is my system info from the KDE panel, Mesa is 22.1.7 with Amdgpu 22.0.0:

Operating System: Manjaro Linux
KDE Plasma Version: 5.25.5
KDE Frameworks Version: 5.98.0
Qt Version: 5.15.6
Kernel Version: 6.0.2-2-MANJARO (64-bit)
Graphics Platform: X11
Processors: 16 × AMD Ryzen 7 3700X 8-Core Processor
Memory: 31.3 GiB of RAM
Graphics Processor: AMD Radeon RX 570 Series

[stgatilov]

Post darkmod.cfg and condump then.

[SeriousToni]

10 hours ago, MirceaKitsune said:

This improvement is only for stencil shadows correct? I believe I used shadow maps previously... likely because they were faster at the time, and I think they're the only mode that allowed shadows to grow softer with distance from the caster rather than having a constant softness? If stencil supports the same features as maps now and are expected to be faster or as fast, I may use it with this optimization! I'll likely test at least if it helps, probably not tonight as I'm first getting re-accustomed with the 2.10 release.

Im always confused which one is better. Stencil or mapped shadows. I can't even tell which I chose right now in the game. Could this be renamed for 2.11 to just shadow quality high and low maybe?

[datiswous]

1 hour ago, SeriousToni said:

shadow quality high and low maybe?

Which one is high and which one is low? I like stencil shadows more visually, but Volumetric Lights usually don't work with them.

For me performance with Stencil shadows is faster, but for others it is (or was) shadow maps.

[datiswous]

6 hours ago, MirceaKitsune said:

Note that when I pressed F12 to take it, I was in reality seeing the the scene just with no shadows... only now noticed everything came out black with only the HUD and entities showing! Will share it here anyway in case it helps.

This happens when you have set fullscreen to Borderless. Typically after an update you forget to change this setting to Fullscreen.

Maybe the other problem is related to this as well?

I think this should be set to Fullscreen by default on Linux systems (and maybe not have the borderless option at all?). I don't know if that's possible. In the description for this setting it should say it is a Windows-only setting.

Btw. what does the borderless functionality under Windows do better? I just tested this on a Windows 10 computer and I didn't see a difference in functionality, so I think it should be Fullscreen by default on both systems.

Edited October 24, 2022 by datiswous

[Obsttorte]

5 hours ago, SeriousToni said:

Im always confused which one is better. Stencil or mapped shadows. I can't even tell which I chose right now in the game. Could this be renamed for 2.11 to just shadow quality high and low maybe?

I don't think one is better than the other. As far as I understand it, but I haven't really dived into the materia, it's just two different approaches to calculating the shadows. Depending on your hardware and the specific mission one might gain you better performance than the other.

But if you don't notice any difference both visually and performance wise, it doesn't play a role which one you've choosen anyway. So I would suggest: Forget about it

[MirceaKitsune]

7 hours ago, stgatilov said:

Post darkmod.cfg and condump then.

Darkmod.cfg

con.txt

As my config is ancient I went for a fresh one only changing some menu settings: Same issue by default.

6 hours ago, SeriousToni said:

Im always confused which one is better. Stencil or mapped shadows. I can't even tell which I chose right now in the game. Could this be renamed for 2.11 to just shadow quality high and low maybe?

They're two different implementations I believe: One isn't necessarily better than the other, it's whichever renders fastest and / or looks best, which until now changed with each release. Yesterday I got the impression stencil feels smoother in terms of FPS, but I don't get realistic shadows that soften with distance over surfaces so I wanted maps to defeat that limitation.

Also I use fullscreen not borderless. Borderless helps with some issues... but IIRC it means the OS still renders the desktop and wastes resources, I use fullscreen to ensure maximum FPS.

Edited October 24, 2022 by MirceaKitsune

[nbohr1more]

Yep. It seems that the associated shader won't compile via your GPU driver:

Quote

WARNING:Compiling shader file tiled_custom_mipmap.frag.glsl failed:             
0:46(2): error: `return' with wrong type uvec4, in function `sample' returning  
vec4                                                                            
                                                                                                                                                
File indexes:                                                                   
  0 - tiled_custom_mipmap.frag.glsl                                             
                                                                                
WARNING:Failed to attach shader tiled_custom_mipmap.frag.glsl to program        
tiled_mipmap_first. 

  You could try replacing all vec4 references with uvec4 in tiled_custom_mipmap.frag.glsl ?

 

 

 

[datiswous]

1 hour ago, MirceaKitsune said:

Also I use fullscreen not borderless. Borderless helps with some issues... but IIRC it means the OS still renders the desktop and wastes resources, I use fullscreen to ensure maximum FPS.

Ah sorry. Last time I had this issue I had it set on borderless by accident, changing it to fullscreen fixed it for me. I use almost the same os as you.

[MirceaKitsune]

2 hours ago, nbohr1more said:

Yep. It seems that the associated shader won't compile via your GPU driver:

  You could try replacing all vec4 references with uvec4 in tiled_custom_mipmap.frag.glsl ?

Replaced every vec4 with uvec4 (left ivec4 untouched) in tdm_base01.pk4/glprogs/tiled_custom_mipmap.frag.glsl: Still would not work. Here is another condump with this change included:

con.txt

Quote

Checking for changed model files...
Linking GLSL program tiled_mipmap_first ...
WARNING:Compiling shader file tiled_custom_mipmap.frag.glsl failed:
0:53(2): error: `return' with wrong type vec4, in function `sample' returning  uvec4

File indexes:
  0 - tiled_custom_mipmap.frag.glsl

WARNING:Failed to attach shader tiled_custom_mipmap.frag.glsl to program
tiled_mipmap_subsequent.

Linking GLSL program tiled_mipmap_subsequent ...

 

Edited October 24, 2022 by MirceaKitsune

[stgatilov]

Yes, the shader does not compile:

    uniform usampler2D u_sourceTexture;
    vec4 sample(ivec2 pixelCoords) {
        // fetch from base LOD level
        return texelFetch(u_sourceTexture, clampPixelCoords(pixelCoords), 0);
    }

Since texture is usampler2D, texelFetch returns uvec4, i.e. vector unsigned, while the function should return vec4, i.e. vector of floats.

However, according to official docs , it should work:

Quote

Certain values can be implicitly converted to certain types. This means that an explicit construction operation is not necessary.

Signed integers can be implicitly converted to unsigned integers, but the reverse is not true. Either integer type can be converted into floats, and integers and floats can be converted into doubles.

Vector and matrix values are implicitly converted if the basic type they contain is implicitly convertible.

So it looks like an issue in the driver.

Try to put the attached file into glprogs directory in TDM root.
It will override the file you have in pk4 archive.

I added an explicit cast to vec4 there.

 

tiled_custom_mipmap.frag.glsl

[MirceaKitsune]

20 minutes ago, stgatilov said:

Yes, the shader does not compile:

    uniform usampler2D u_sourceTexture;
    vec4 sample(ivec2 pixelCoords) {
        // fetch from base LOD level
        return texelFetch(u_sourceTexture, clampPixelCoords(pixelCoords), 0);
    }

Since texture is usampler2D, texelFetch returns uvec4, i.e. vector unsigned, while the function should return vec4, i.e. vector of floats.

However, according to official docs , it should work:

So it looks like an issue in the driver.

Try to put the attached file into glprogs directory in TDM root.
It will override the file you have in pk4 archive.

I added an explicit cast to vec4 there.

 

tiled_custom_mipmap.frag.glsl 4.77 kB · 0 downloads

Wonderful, that seems to be working: I can now switch to stencil shadows with r_softShadowsMipmaps and see them just fine. If that doesn't break anything on other platforms it's good on my end.

[HMart]

11 hours ago, SeriousToni said:

Im always confused which one is better. Stencil or mapped shadows. I can't even tell which I chose right now in the game. Could this be renamed for 2.11 to just shadow quality high and low maybe?

IMO that is not a good idea, stencil shadows and shadow maps, are two totally different things, technically and visually, putting them both under a single shadow quality option, IMO is not correct and may lead to way more confusion by mission makers.

Spoiler

Stencil shadows and shadow maps, have visual diferences and IMO you cannot say one, is higher quality than the other, depends on preference and implementation.

Stencil shadows look very sharp (if you disable soft shadows), no matter the quality level, maps can look very pixelated if you use lower shadow map sizes.

Stencil shadows is extruded geometry (triangle mesh's).

Shadow maps is textures simulating shadows (there its perfect support for alpha mapped materials). 

The only reason some may think, stencil and maps are the same, is because afaik, in TDM the team is forced to limit shadow maps to what stencil shadows can do. What I mean by this is, for example, transparent textures, grass, tree branches and other stuff using alpha maps, shadow maps support those perfectly, stencil shadows don't, so because TDM started historically with stencil shadows alone, afaik no "perforated" materials (alpha mapped materials) in TDM cast shadows.

This is a historical limitation and one that TDM may never be able to change, because has the potencial to break shadows on many older missions, also saves performance...

 

[MirceaKitsune]

Right, that's another limitation: Stencil shadows can't produce shadows from alpha textures, alongside shadows not softening with length. So if you want maximum quality and realism you need maps, stencil may be faster in some cases though.

Speaking of shadow maps: Is there a reason why "r_shadowMapSinglePass" and "r_shadowMapCullFront" aren't defaulted to 1? I almost lost them from my config and when I turned them on I noticed a significant performance boost, at least +10 FPS.