Jump to content


Photo

TDM Engine Development Page

idtech4 development engine

  • Please log in to reply
1089 replies to this topic

#1076 Springheel

Springheel

    Creative Director (retired)

  • Admin
  • 37406 posts

Posted 28 October 2018 - 12:39 PM

Most of the lights there are noshadows already. The few remaining lights are relatively small and don't intersect with much of the foliage.

 

 

That was one example picked at random.  Are you confident about all other places foliage textures are used in existing maps?

 

Let's use some specific numbers, because I could be totally wrong about how this works.  The hedge01_square_long.lwo model has a 96 tris shadow mesh, but 5128 tris of an alphamap foliage texture set to noshadows.  Let's imagine a short path with these hedge models lining each side of the path-say 10 in total, hit by two lights.  Currently that's maybe 2000 shadow-casting tris at most, which isn't an issue.  Are you saying that if that foliage texture was suddenly forced to be shadow-casting, the 100,000 extra tris wouldn't have more than a 5% effect on the drawcalls of that scene?


TDM Missions:   A Score to Settle   *   A Reputation to Uphold   *   A New Job   *    A Matter of Hours
 
Video Series:   Springheel's Modules   *   Speedbuild Challenge   *   New Mappers Workshop  *   Building Traps

#1077 duzenko

duzenko

    Advanced Member

  • Active Developer
  • PipPipPip
  • 1707 posts

Posted 28 October 2018 - 01:20 PM

 

That was one example picked at random.  Are you confident about all other places foliage textures are used in existing maps?

 

Let's use some specific numbers, because I could be totally wrong about how this works.  The hedge01_square_long.lwo model has a 96 tris shadow mesh, but 5128 tris of an alphamap foliage texture set to noshadows.  Let's imagine a short path with these hedge models lining each side of the path-say 10 in total, hit by two lights.  Currently that's maybe 2000 shadow-casting tris at most, which isn't an issue.  Are you saying that if that foliage texture was suddenly forced to be shadow-casting, the 100,000 extra tris wouldn't have more than a 5% effect on the drawcalls of that scene?

Any model with separate shadow geometry must stay as it is - i.e. keep the noshadow flag on the main mesh material.

No argue there.

I'm absolutely 'pro' all and every speed trick there is - this one especially.

 

OK, let's just imagine we did just that. We'd still have 100K tri's in depth render, 100K for ambient light, 200K for point lights - plus the added 200K. Even on a system clearly limited by tri numbers it's only 200/600 = 33%. Now the big question is, are we actually having tri-limited scenes in TDM? Because if we do, that's already a big problem of its own. A mapper can target 60fps on his GTX 1070 but the average Joe's Radeon 7670 will barely squeeze 15 fps. We have lots of graphics options in TDM that can allow smooth gameplay on wide range of hardware but there is no option to alleviate the tri-count bottlenecks.

 

That means that 3d scenes can never get even close to limiting FPS by tri count. That's going to be causing massive problems with or without shadows on transparent materials. Putting very-high-poly models in maps is the best way to make the game unplayable for a good half of our users. Forcing alpha shadows is just making the margin to be a bit higher e.g. 50% vs 60%. The right way is to make the low-poly LOD versions for models like that.

 

But again, I don't suggest to lose the shadow meshes, or to force shadows on all perforated materials.



#1078 Judith

Judith

    Advanced Member

  • Member
  • PipPipPip
  • 1644 posts

Posted 28 October 2018 - 02:22 PM

AFAIK, you really have to try hard to make your FPS go down because of excessive geometry. It scales with hardware very well. Last time I tried, I needed like 4 million tris per scene to make the FPS go down. Drawcalls or shadow tris were always first to bring FPS down.


  • Springheel likes this

#1079 Springheel

Springheel

    Creative Director (retired)

  • Admin
  • 37406 posts

Posted 28 October 2018 - 03:15 PM

shadow tris were always first to bring FPS down.

 

 

Yes, adding new shadow tris can't be done casually...that's why I'm confused about how 100,000 new shadow tris to a scene (in the example above) wouldn't be a performance issue.


TDM Missions:   A Score to Settle   *   A Reputation to Uphold   *   A New Job   *    A Matter of Hours
 
Video Series:   Springheel's Modules   *   Speedbuild Challenge   *   New Mappers Workshop  *   Building Traps

#1080 duzenko

duzenko

    Advanced Member

  • Active Developer
  • PipPipPip
  • 1707 posts

Posted 28 October 2018 - 03:27 PM

 

Yes, adding new shadow tris can't be done casually...that's why I'm confused about how 100,000 new shadow tris to a scene (in the example above) wouldn't be a performance issue.

Because it's not stencil shadows


  • Judith likes this

#1081 Judith

Judith

    Advanced Member

  • Member
  • PipPipPip
  • 1644 posts

Posted 28 October 2018 - 03:34 PM

With shadow maps, complex shadow scenes will now rely on GPU memory. Not sure how we'll measure these things, I'll have to make some new test maps to see how it works on my system now.



#1082 HMart

HMart

    Advanced Member

  • Member
  • PipPipPip
  • 742 posts

Posted 28 October 2018 - 03:35 PM

I'm also of the opinion shadow meshes should still be included, they are a performance trick even for shadow maps and yes, not all alpha materials should cast shadows but that is obvious. The ability to control shadow mapping on/of per material and or entity should also be a most.

 

Btw i'm almost afraid to say it, because this is really a radical suggestion... but what would solve many of the problems i'm seeing coming up is...removing stencil shadows all together.

 

Wait! Don't come with the pitchforks right away! Hear me out ;P ...By doing that, you can then remove noshadow from the materials with no fear of breaking what is not there, and imo logically, removing shadows or not from a object, should be in the hands of the mission maker, not TDM forcing everyone that uses the oficial materials to not have shadows (apart from the obvious ones that should never cast shadows), this way there's no need for a extra keyword either. Anyone wanting to control the shadows, for existing materials per mission, just use a material skin override with noshadow on it, like always, missions where shadow anomalies are detected, because someone faked shadows, should be updated by the mission makers or by a third party if the author is nowhere to be found. Yes this is a radical change indeed but would save many headaches and pretty much all engines i know, use one or the other, supporting two radically different shadow systems in the same game is not easy.

 

If this is still not the time to think of that then, i may suggest not removing noshadows, let that keyword control only stencil shadows, make another to control shadow maps, that  way you don't mess with stencil, for shadow maps, editing the missions at least to remove faked shadows wouldn't be necessary (nor do you want that if playing with stencil shadows), why, because you would only need to stop those particular materials from casting shadow maps and this could be done like i already said,  by implementing that new "noshadowmap" keyword on a skin and then just make a script that at map/mission start time would detect the materials you want and swap them. 

 

I can't think of anything better unfortunately.   

 

Because it's not stencil shadows

 

Yes shadow maps are just textures they do not create extra geometry like stencil does.



#1083 Judith

Judith

    Advanced Member

  • Member
  • PipPipPip
  • 1644 posts

Posted 28 October 2018 - 03:40 PM

I think Duzenko already mentioned that stencil shadows are a dead end in terms of development, so you can expect that at some point, r_shadows 2 will be default / mandatory setting.



#1084 duzenko

duzenko

    Advanced Member

  • Active Developer
  • PipPipPip
  • 1707 posts

Posted 28 October 2018 - 04:11 PM

removing stencil shadows all together.

Is not on the table ATM, for a number of reasons

 

I think Duzenko already mentioned that stencil shadows are a dead end in terms of development, so you can expect that at some point, r_shadows 2 will be default / mandatory setting.

It's all about what the current hardware offers. In 2004 stencil made a lot of sense on that time GPU's with relatively limited shaders and powerful ROPs.

Since then, nVidia has been investing a lot of effort in adding more stuff in their hardware that would allow more and more benefits for shadow mapping. I don't know if it was the best course but they chose it and here we are now - with lots of hardware optimization and algorithm RnD leveraging each other.

Maybe in a few years the RTX technology will make shadow maps obsolete again but until then we'd be an odd duck trying to cling on to stencil. There have been fundamental issues in both techniques but for shadow maps they're largely resolved while stencil mostly remains where it was 15 years ago.

Throw the constantly increasing abuse of stencil shadows by mappers on top of that.

It's just too aged now.


  • Judith and HMart like this

#1085 nbohr1more

nbohr1more

    Darkmod PR, Wordsmith

  • Development Role
  • PipPipPipPipPip
  • 9035 posts

Posted 28 October 2018 - 10:52 PM

It seems that PCSS approaches a proper ray-traced solution as long as you have "infinite shadow map resolution"

so the newest shadow solutions like HFTS and RTX ray tracing are sold as "solutions" to this last resolution hurdle.

 

I'd still like to see this horse race:

 

1) Stencil Shadows with PCSS blurring

2) Penumbra Wedges

3) HFTS Shadows

4) RTX raytraced shadows

 

I'll bet the result would be 1 as the clear winner as long as GPU skinning is also done.

HFTS and RTX waste more resources than Stencil Shadows ever dreamed of to achieve that look.

 

Though, ideally, we'd just have better shadow map resolution management that isn't too expensive

for real-time moving lights. I'm not sure how CSM or Parallel Split Shadow Maps, etc perform in dynamic

scenes.

 

 

Also, I like the idea of a new noStencilShadows flag. It's clear that there are cases where enabling

shadows will only be possible in shadow map mode.


Please visit TDM's IndieDB site and help promote the mod:

http://www.indiedb.c...ds/the-dark-mod

(Yeah, shameless promotion... but traffic is traffic folks...)

#1086 nbohr1more

nbohr1more

    Darkmod PR, Wordsmith

  • Development Role
  • PipPipPipPipPip
  • 9035 posts

Posted 29 October 2018 - 10:51 AM

Whoa!

 

I went back to looking at Trapezoid Shadow maps and what recent work has been done in similar

areas of research.

 

Look at this stuff!

 

https://marciocerqueira.github.io/#2


  • HMart and duzenko like this
Please visit TDM's IndieDB site and help promote the mod:

http://www.indiedb.c...ds/the-dark-mod

(Yeah, shameless promotion... but traffic is traffic folks...)

#1087 nbohr1more

nbohr1more

    Darkmod PR, Wordsmith

  • Development Role
  • PipPipPipPipPip
  • 9035 posts

Posted 30 October 2018 - 08:01 AM

Robert Beckebans is looking into getting his GL conventions to use Direct State Access:

 

https://github.com/F...penGL-Functions

 

I think we might've made some of these changes already but this might come in handy for further modernization efforts.


  • Bikerdude and HMart like this
Please visit TDM's IndieDB site and help promote the mod:

http://www.indiedb.c...ds/the-dark-mod

(Yeah, shameless promotion... but traffic is traffic folks...)

#1088 duzenko

duzenko

    Advanced Member

  • Active Developer
  • PipPipPip
  • 1707 posts

Posted 31 October 2018 - 08:58 AM

Whoa!

 

I went back to looking at Trapezoid Shadow maps and what recent work has been done in similar

areas of research.

 

Look at this stuff!

 

https://marciocerqueira.github.io/#2

Much of that is too complicated for my feeble mind.

However the revectorization PDF got me thinking in a particularly exciting direction.

My first attempt at shadow AA, unsurprisingly, ended with a puff, however the second time came out better - see the attached

This is obviously very much WIP, but the next version should add enough quality to get rid of shadow edge corners almost completely

You can try it with current svn trunk - multi light shader

Attached Files


  • Judith, HMart, nbohr1more and 1 other like this

#1089 nbohr1more

nbohr1more

    Darkmod PR, Wordsmith

  • Development Role
  • PipPipPipPipPip
  • 9035 posts

Posted 31 October 2018 - 03:23 PM

I just read through the slides and the papers to get a fuller picture.

Initially, I thought this was some sort of space convolution trick akin to Trapezoid Shadow maps
since it was one of the first things that came up for a search for that.

I now see what this really is...

The algorithm treats all the silhouette pixels like graph plot points then reconstructs
the silhouette by plotting a curve through the points. It kinda reminds me of post process
AA like MLAA or FXAA... or (more fittingly) like a pixel art up-scaling algorithm:

https://en.wikipedia...ling_algorithms

You can see in a slide where the shadows from a thin tree branch produces scattered pixel
dots with breaks and discontinuities that the algorithm turns those scattered pixels into
curved blobs rather than restoring the original branch silhouette. Thus this algorithm
requires a certain minimum amount of shadow map resolution to be effective.
Please visit TDM's IndieDB site and help promote the mod:

http://www.indiedb.c...ds/the-dark-mod

(Yeah, shameless promotion... but traffic is traffic folks...)

#1090 duzenko

duzenko

    Advanced Member

  • Active Developer
  • PipPipPip
  • 1707 posts

Posted 01 November 2018 - 05:01 AM

I just read through the slides and the papers to get a fuller picture.

Initially, I thought this was some sort of space convolution trick akin to Trapezoid Shadow maps
since it was one of the first things that came up for a search for that.

I now see what this really is...

The algorithm treats all the silhouette pixels like graph plot points then reconstructs
the silhouette by plotting a curve through the points. It kinda reminds me of post process
AA like MLAA or FXAA... or (more fittingly) like a pixel art up-scaling algorithm:

https://en.wikipedia...ling_algorithms

You can see in a slide where the shadows from a thin tree branch produces scattered pixel
dots with breaks and discontinuities that the algorithm turns those scattered pixels into
curved blobs rather than restoring the original branch silhouette. Thus this algorithm
requires a certain minimum amount of shadow map resolution to be effective.

The way I understand it we'd need a pre-processing pass. Render the player's view into a temporary texture that stores the shadow occlusion results.

Then we use that to traverse the shadow edge in the shadowmap texture space.

My approach is going to be somewhat different to what they did because I want to leverage textureGather.

The biggest problem is how to store the shadow occlusion for all lights to use in the multi-light shader.

I'm only doing this in the multi-light shader path so as not to pollute the single-light path with 'construction garbage'

 

On the other hand I could combine that and what I have been planning for a while now - shadows for all lights in one pass.

 

The good ol' question is, do we want to wait until 2.07 branch.

I already have a simple nearest-neighbors AA filter in the multi-light shader while the above easily could take weeks of spare time coding.

@stgatilov any ideas on the 2.07 branch date?

 

Then, this is probably going to be mutually exclusive to SS and if so how do we present it to the audience?







Also tagged with one or more of these keywords: idtech4, development, engine

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users