Jump to content
The Dark Mod Forums
Sign in to follow this  
cabalistic

Testers and reviewers wanted: BFG-style vertex cache

Recommended Posts

Your thinking of baking shadows like the dark engine does and as far as I know idtech4 isnt able to do that.

 

Yup. Judging by the MSI Afterburner readouts on my end, it seems like right now shadowmaps lower CPU load only a bit, while increasing GPU load quite much. If that load reaches or exceeds 99%, FPS go down. I'm not sure if hardware like my laptop is worth supporting any longer, as it's pretty old. Both its CPU and GPU have quite a bit to do with stencil shadows, but at least it's kind of balanced work (like 50% CPU, 65% GPU). Upsetting this balance only makes things worse (e.g. 45% CPU, 99% GPU). Maybe that also has something to do with soft shadows being quite expensive, regardless of r_shadows value? Getting back to hard shadows, as difficult as it is to swallow, makes things much easier for the whole system.

Edited by Judith
  • Like 1

Share this post


Link to post
Share on other sites

Both problems I listed persist under update 15.

Thanks for testing.

 

I didn't notice widespread issues with fog but one wall in TD3: Heart of Lone Salvation definitely flickers.

 

As for the "projected lights"... I presume you are referring to 1D+2D style projected lights?

 

(GLSL Cubemap lights don't work in projected mode as far as I know, but there was a discussion about maybe getting them back that sorta trailed off...)

  • Like 1

Please visit TDM's IndieDB site and help promote the mod:

 

http://www.indiedb.com/mods/the-dark-mod

 

(Yeah, shameless promotion... but traffic is traffic folks...)

Share this post


Link to post
Share on other sites

Yes, I mean the 1D+2D spotlights. Tried the single pass mode and some misc settings like FBO but it didn't change anything.

 

Try renaming this file to remove the txt extension and placing it into the glprogs folder (in the tdm_base01.pk4 ).

 

shadowMapN.gs.txt

 

This has a fix by Duzenko for divide by zero issues:

 

 

It turned out to be zero-division due to a surface vertex being the same as light origin.

It should have been fixed now that I switched to hardware W-division instead of doing it in the shader.

 

It might be more friendly to your hardware. I held off on an immediate package update because there is another

issue with "noshadows_lit" that might be addressed shortly.

  • Like 1

Please visit TDM's IndieDB site and help promote the mod:

 

http://www.indiedb.com/mods/the-dark-mod

 

(Yeah, shameless promotion... but traffic is traffic folks...)

Share this post


Link to post
Share on other sites

No dice, I'm afraid. Still flickering, still no lights when SoftShadows=0 and lights w/o shadows when SoftShadows=/=0.

 

quick edit: Ah, check this one out, though. On a lark, I launched DarkModNoTools.exe, the executable duzenko provided in the GLSL skybox testing thread. With it, shadows on spotlights show, but they are very low-res (I can't quantify how it looks, but let's say a magnitude lower than whatever the MapSize is set to). Shadow maps from omni lights seem fine, but both omni and spotlights are unaffected by SoftShadows. Fog flickering still persists.

Edited by Spooks
  • Like 1

My FMs: The King of Diamonds (2016)



| Visit my Mapbook thread sometimes! | Read my tutorial on Image-Based Lighting Workflows for TDM! |

Share this post


Link to post
Share on other sites

Hmm.

 

I apologize if I missed that earlier.

 

That sounds like broken GLSL compliance.

 

Please post a condump for each mode:

 

1) Open the console

2) reloadGLSLprograms

3) condump badshaders1.txt

  • Like 1

Please visit TDM's IndieDB site and help promote the mod:

 

http://www.indiedb.com/mods/the-dark-mod

 

(Yeah, shameless promotion... but traffic is traffic folks...)

Share this post


Link to post
Share on other sites

Please post a condump for each mode:

 

Excuse me, could you be more specific?

 

r_shadows 1 vs 2? The output from DarkModNoTools.exe vs regular DarkMod.exe?

 

edit: looking over the condump, perhaps this is the salient point here:

OpenGL vendor: NVIDIA Corporation
OpenGL renderer: GeForce GTX 950/PCIe/SSE2
OpenGL version: 4.6.0 NVIDIA 390.77

[yadda yadda yadda]

---------- R_ReloadGLSLPrograms_f -----------
interaction VF
ambientInteraction VF
interactionN VF
stencilShadow VF
shadowMap VGFWARNING:shaderCompileFromFile(glprogs/shadowMap.fs) validation
0(9) : error C7566: in blocks require #version 150 or later
0(9) : error C0000: ... or #extension GL_ARB_gpu_shader5 : enable
0(17) : warning C7533: global variable gl_FragColor is deprecated after version
 120

with DarkModNoTools.exe, which has somewhat working shadow maps.

 

No such warning with the DarkMod.exe from update 15:

---------- R_ReloadGLSLPrograms_f -----------
interactionA VF
ambientInteraction VF
stencilShadow VF
shadowMapA VGF
oldStage VF
depthAlpha VF
fog VF
blend VF
cubeMap VF
interactionN VF
shadowMapN VGF
Edited by Spooks
  • Like 1

My FMs: The King of Diamonds (2016)



| Visit my Mapbook thread sometimes! | Read my tutorial on Image-Based Lighting Workflows for TDM! |

Share this post


Link to post
Share on other sites

 

Excuse me, could you be more specific?

 

r_shadows 1 vs 2? The output from DarkModNoTools.exe vs regular DarkMod.exe?

How old was the notools exe? It's most likely too old.

The best way is to upload a quick save for us to try.

And try the console commands from @nb's post above.

  • Like 1

Share this post


Link to post
Share on other sites

The file signature is ‎Saturday, ‎September ‎22, ‎2018, ‏‎6:50:07 PM, so yes, I agree it's too old :D

 

I really can't think of an FM with a spotlight off the top of my head... Just a minute...

 

Okay, King of the Mountain by Spoonman has a corridor with a bunch of spotlights. Here is the save file:

 

https://drive.google.com/file/d/14U4pyOk8tMwIhpngLd0_vj1DGfnJdpiX/view?usp=sharing

 

noshadowshere.txt (if you need to create it) says:

"noshadowshere"
"kotm"
""

This was saved with TheDarkModx64.exe from update 15, in case that's important. I recommend g_showPlayerShadow 1 because there aren't any shadowcasting objects besides the patrolling guard.

Edited by Spooks
  • Like 1

My FMs: The King of Diamonds (2016)



| Visit my Mapbook thread sometimes! | Read my tutorial on Image-Based Lighting Workflows for TDM! |

Share this post


Link to post
Share on other sites

Double-post for another bug report:

 

With r_shadowMapSinglePass 1, the shadow maps on alpha tested materials don't work anymore (it's back to solid shadows, like with stencils).

  • Like 1

My FMs: The King of Diamonds (2016)



| Visit my Mapbook thread sometimes! | Read my tutorial on Image-Based Lighting Workflows for TDM! |

Share this post


Link to post
Share on other sites

One very peculiar bug: when the light radius exceeds ~550, such light will add to the shadow tris, despite using r_shadows 2.

It's a 'feature'

Try the new r_maxShadowMapLight cvar

  • Like 1

Share this post


Link to post
Share on other sites

Hi. I just started Dark Mod today and all of a sudden shadow maps work with spotlights??

 

¯\_(ツ)_/¯¯\_(ツ)_/¯¯\_(ツ)_/¯¯\_(ツ)_/¯¯\_(ツ)_/¯¯\_(ツ)_/¯¯\_(ツ)_/¯¯\_(ツ)_/¯¯\_(ツ)_/¯

 

I have no idea why it's working now, I changed no settings whatsoever. I'll point out that the fog flickering is still present and I'll be keeping an eye out for the spotlight issue reappearing in later updates, but for now... I guess we're cool?

 

No! Wait! I literally just found out why this happens! Agh. OK, check this out:

 

If r_softShadowsRadius is equal or greater than 0 the bug as I've described it appears.

If r_softShadowsRadius is less than 0 - a negative value - the bug disappears.

 

It doesn't matter if r_softShadowsQuality is at 0 or not (!), although the bug manifests differently when it's at 0, like I described in my original post. I must also report that at negative values of the r_softShadowsQuality cvar the shadows also disappear - but this happens with both r_shadows 1 and 2 and I don't think you're supposed to be inputting negative values for this one anyway.

 

OK! With this mystery being solved I want to complain about these now:

 

gyGMxdy.png

 

This is shadow maps with soft shadows on. The opaque splotches appear near thin shadows close together, like grates, or around the player's shadow and they're worse the lower the soft shadow quality value is. Having it higher, then, does help but I don't think it eliminates them entirely. I'm wondering if there's any way to lessen the issue, or if it's just a kink of the soft shadow implementation. Some scenes are bound to have these if a player has soft shadow quality on low settings, around 16 or so.

  • Like 1

My FMs: The King of Diamonds (2016)



| Visit my Mapbook thread sometimes! | Read my tutorial on Image-Based Lighting Workflows for TDM! |

Share this post


Link to post
Share on other sites

If r_softShadowsRadius is equal or greater than 0 the bug as I've described it appears.

If r_softShadowsRadius is less than 0 - a negative value - the bug disappears.

In the latest code, positive radius means "deduce radius from light source".

The negative radius means just "use this value always" and is much safer.

 

The code for positive radius is:

if ( !args->GetFloat( "size", "-1", renderLight->radius ) )
    renderLight->radius = 1e-2 * idMath::Fmin( renderLight->lightRadius[0], renderLight->lightRadius[1], renderLight->lightRadius[2] );

In my opinion, this is wrong. Moreover, given the number of different lights in existing TDM missions, this is a dangerous idea which may sometimes produce completely wrong results.

Spooks, do you have problems with positive value only with r_shadows 2 ? Or they are present with r_shadows 1 (stencil shadows) also?

This question is very serious, because we need to decide now whether we save this behavior for 2.07 or not.

 

This is shadow maps with soft shadows on. The opaque splotches appear near thin shadows close together, like grates, or around the player's shadow and they're worse the lower the soft shadow quality value is. Having it higher, then, does help but I don't think it eliminates them entirely. I'm wondering if there's any way to lessen the issue, or if it's just a kink of the soft shadow implementation. Some scenes are bound to have these if a player has soft shadow quality on low settings, around 16 or so.

Most likely a problem in implementation.

I have r_shadowMapSize 1024, r_softShadowsQuality 24, r_softShadowsRadius -2. I did not notice such problems.

Could you write about some specific place (you can post coordinates which getviewpos console command prints) ?

 

 

P.S. I have also noticed the "aas out of date" message in Betrayal mission on the current SVN.

This is very strange, I think someone has to bisect over SVN history to find out when it started to happen.

By the way, dmap has the ability to precompute shadow volumes of static objects, and by default it is turned on.

 

Share this post


Link to post
Share on other sites

Spooks, do you have problems with positive value only with r_shadows 2 ? Or they are present with r_shadows 1 (stencil shadows) also?

This question is very serious, because we need to decide now whether we save this behavior for 2.07 or not.

 

In 2.06 it used to be positive values would just work with r_shadows 1 (the only shadow mode). I was going to say this extends to here and only shadow maps have a problem, but... On further testing positive values also break soft shadows for r_shadows 1! The shadows are there, they're just not softened. Again, it is specifically in spotlights, omni lights look fine. Setting the cvar to a negative value softens the shadows properly.

 

I had to comb through the FM list for a while before I found a pre-existing place where the opaque blotches show up. You can see it in Sir Tallbot's Collateral: map kiss.map, g_showPlayerShadow 1 and notarget just so the guard doesn't bother you.

The getviewpos is: -53.56 1057.18 172.25 40.4 179.3 0.0

Strafe left to right and look at the kitchen grate to see the blotches appear as your shadow intersects with the grate shadows. At SoftShadowsRadius -1 it's kind of minimal, but put it to something like -5 and it really shows. The SoftShadowsQuality can be conservative values, something between 16 and 32. My ShadowMapSize is 1024 as well.

 

edit:

 

yeWx7IL.jpg

 

Here's a screenshot of what it looks like and where the place is exactly in the map. It's in the kitchen area, ground floor, the unlit one of two pairs of fireplaces.

Edited by Spooks

My FMs: The King of Diamonds (2016)



| Visit my Mapbook thread sometimes! | Read my tutorial on Image-Based Lighting Workflows for TDM! |

Share this post


Link to post
Share on other sites

I had this kind of artifacts some time ago, but they are gone now. I'm not sure what I did to make it go away, but I suspect it's either using AA x 2 (it helps to get rid the HeatHaze program artifacts, for some reason), or disabling in-engine compression for images (image_useNormalCompression "0", image_useCompression "0"). Try to see whether it does something on your end.

Edited by Judith

Share this post


Link to post
Share on other sites

I had to comb through the FM list for a while before I found a pre-existing place where the opaque blotches show up. You can see it in Sir Tallbot's Collateral: map kiss.map, g_showPlayerShadow 1 and notarget just so the guard doesn't bother you.

The getviewpos is: -53.56 1057.18 172.25 40.4 179.3 0.0

Strafe left to right and look at the kitchen grate to see the blotches appear as your shadow intersects with the grate shadows. At SoftShadowsRadius -1 it's kind of minimal, but put it to something like -5 and it really shows. The SoftShadowsQuality can be conservative values, something between 16 and 32. My ShadowMapSize is 1024 as well.

Yes, this is the problem of the algorithm.

 

In the first pass it tries to find closest blocker in a rather large cone going from light source to the fragment being rendered.

The blocker is searched for by sampling the shadow map texture in some vicinity. In this particular case it seems that all samples are shadowed, so the algorithm declares the fragment as fully shadowed.

 

Here is the guilty optimization:

//shortcut: all blockers in search angle => fully occluded
if (blockerCnt == u_softShadowsQuality) {
  gl_FragColor.rgb *= 0.0f;
  return;
}

It seems that it is not robust enough. Often the search code is very large and has all samples shadowed, but if we then compute the penumbra radius, it is much smaller, and some samples during real evaluation are lit.

This is sad because it disables optimization for fully shadowed surfaces :unsure:

 

Share this post


Link to post
Share on other sites

Yes, a tough cookie.

 

Increasing the shadow map size or quality can reduce or eliminate these but

that can be a pretty high price to pay.

 

One thing we might consider is a history buffer:

 

https://www.cg.tuwien.ac.at/research/publications/2007/Scherzer-2007-PCS/Scherzer-2007-PCS-Preprint.pdf

 

which would also cure temporal artifacts on moving shadows far from the light center.


Please visit TDM's IndieDB site and help promote the mod:

 

http://www.indiedb.com/mods/the-dark-mod

 

(Yeah, shameless promotion... but traffic is traffic folks...)

Share this post


Link to post
Share on other sites

Yes, this is the problem of the algorithm.

 

In the first pass it tries to find closest blocker in a rather large cone going from light source to the fragment being rendered.

The blocker is searched for by sampling the shadow map texture in some vicinity. In this particular case it seems that all samples are shadowed, so the algorithm declares the fragment as fully shadowed.

 

Here is the guilty optimization:

//shortcut: all blockers in search angle => fully occluded
if (blockerCnt == u_softShadowsQuality) {
  gl_FragColor.rgb *= 0.0f;
  return;
}

It seems that it is not robust enough. Often the search code is very large and has all samples shadowed, but if we then compute the penumbra radius, it is much smaller, and some samples during real evaluation are lit.

This is sad because it disables optimization for fully shadowed surfaces :unsure:

 

Try textureGather - that would quadruple the sample count

Share this post


Link to post
Share on other sites

Further experimenting with lights. It seems parallel lights with shadow maps also have problems, the shadows do not display, or do only in part. This one's a bit harder to pin, no soft shadows are glitchy in a different way than with soft shadows, and it seems the higher the soft shadow quality the more diffused and weak the parallel light is.


My FMs: The King of Diamonds (2016)



| Visit my Mapbook thread sometimes! | Read my tutorial on Image-Based Lighting Workflows for TDM! |

Share this post


Link to post
Share on other sites

Hmm. Parallel lights use a math trick to move the light center to a very far location.

It is meant to simulate moon light. This trick is in place no matter how large the light

volume is. If our softening algorithm is using this center distance then it would

certainly explain over-softening for stencil shadows (where we don't have contact hardening)

but it may also play into Shadow Maps if the map depth capture is predicated on this

false distance (thus you will ALWAYS have low resolution because you are sampling at the

far edge away from the light center).


Please visit TDM's IndieDB site and help promote the mod:

 

http://www.indiedb.com/mods/the-dark-mod

 

(Yeah, shameless promotion... but traffic is traffic folks...)

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Sign in to follow this  

×
×
  • Create New...