Performance optimization idea: Update LOD (frame skipping) for light / shadow updates

[MirceaKitsune]

I hope I'm not proposing some unfeasible idea that was already imagined before, this stuff is fun to discuss so no loss still. Riding the wave of recent optimizations, I keep thinking what more could be done to reach a round 144 FPS compatible with today's monitors. An intriguing optimization came to mind which I felt I have to share: Could we gain something if we had distance-based LOD for entity updates, encompassing everything visual from models to lights?

How it would work: New settings allow you to set a start distance, end distance, and minimum rate. The further an entity gets the lower its individual update rate, slowly decreasing from updating each frame (start distance and closer) to updating at the minimum rate (end distance and further). This means any visual change is preformed with frame skips on any entity: For models such as characters animations are updated at the lower rate, for lights it means shadows are recalculated less often... even changes in the position and rotation of an entity may follow it for consistency, this would especially benefit lights with a moving origin like fireplaces or torches held by guards which recalculate per-frame.

Reasoning: Light recalculation even animated models or individual particles can be significant contributors to performance drain. We know the further something is from the camera the less detail it requires, this is why we have a level-of-detail system with lower-polygon LOD models for characters and even mapmodels. Thus we can go even further and extend the concept to visual updates; Similar to how you don't care if a far away guard has a low-poly helmet you won't notice, you won't care if that guard is being animated at 30 FPS out of your maximum of 60, nor if the shadow of a small distant light is being updated at 15 FPS when an AI passes in front of it. This is especially useful if you own a 144 Hz monitor and expect 144 FPS: I want to see a character in front of me move at 144 FPS, but may not even notice if a guard far away is animating at 60 FPS... I want the shadows of the light from the nearby torch to animate smoothly, but can care less if a lamp meters away updates its shadows at 30 FPS instead.

The question is if this is easy to implement in a way that offers the full benefit. If we use GPU skinning for instance, the graphics card should be told to animate the model at a lower FPS in order to actually preserve cycles... does OpenGL (and in the future Vulkan) let us do this per individual model? I know the engine has control over light recalculations which would probably yield the biggest benefit. Might add more points later as to not make the post too big, for now what are your thoughts?

[MirceaKitsune]

A few additions and observations: We may get even better results using not just distance but also the entity's size, given the rate should probably depend on how much the entity is covering your view at that moment. As this shouldn't need much accuracy we can just throw in the average bounding-box size as an offset to distance to estimate the entity's total screen space. A small candle can decrease its update rate even closer to the camera, while a larger torch will retain a slightly higher rate for longer.

To prevent noticeable sudden changes, the way LOD models can be seen snapping between states in their case, the effect can be applied gradually without artificial steps given it's just a number and may take any value. It might be best to have a multiplier acting on top of the player's maximum or average FPS: If your top is 60 FPS, the lowest update rate beyond the maximum distance would be 30 FPS for a 0.5 minimum setting... along the way one entity may be 0.9 meaning it ticks at 54 FPS, a further one 0.75 meaning 45 FPS, etc.

Internally there should probably be different settings for model animations and lights: A low FPS may be obvious on AI or moving objects so you probably don't want to go lower than half the max (eg: 30 FPS for 60 Hz)... for lights the effect can be more aggressive on soft shadows without noticeable ugliness (eg: 15 FPS for 60 Hz). In the menu this can probably be tied to the existing LOD option which can control both model and frameskip LOD's.

Edited June 15, 2023 by MirceaKitsune

[stgatilov]

In order to better discuss performance, please look into this article: https://wiki.thedarkmod.com/index.php?title=Tracy:_timeline_profiler#Gameplay
Perhaps inspect some Tracy records too.

The game consists of four main parts, most of them run in parallel:

1. game modelling
2. renderer frontend
3. renderer backend
4. GPU

Let's suppose n-th part consumes Tn milliseconds of time on some interval, then the interval takes min(t1 + t2, t3, t4) of time in total. Usually either p.1 + p.2 or p.4 are bottleneck.

If you try doing less updates, you cannot make renderer backend and GPU faster: it still has to render all the stuff every frames. Except for something like reusing shadow maps between frames, which is not implemented yet.
 

You can make game modelling faster if some entities skip thinking.

The most time-consuming entities are AIs, and this kind of optimization is applied to them since the very beginning of TDM, known as "interleaded thinking optimization" (Doom 3 had similar but stronger "dormancy" concept). If an AI is behind closed doors and far from player, then he thinks rarely, e.g. 3 times per second. You can use cv_ai_opt_forceopt cvar to experiment with it.

It took much effort to make this work reliably, but without it TDM should be very slow on anything but small levels. And this optimization caused some headache later, for instance it caused distant AIs to randomly die with uncapped and low FPS.

All the other entities think every frame, as far as I know. Some entities are known to be dangerous to think rarely: that's mostly physics-related things like ropes and ragdolls.
Another thing is that many entities spend negligible time in their Think methods, and most entities don't think at all (only "active" entities think, see listActiveEntities command).

Speaking of renderer frontend, you can update entities less often. Then you can reduce time on understanding which areas entities belong to, and on generating interactions (that's the main part I think).
However, there are parts of frontend which are view-dependent, so they need to run every frame.

You can experiment with cvar r_skipUpdates.
As far as I understand, it disables all updates of render entities/lights from game code.
The game modelling still runs, guards still move around, see you, speak, chase you, and kill you. But renderer displays the obsolete state of the world from the moment when you set the cvar.

[MirceaKitsune]

Cheers. Initially I was thinking of this for lights... later thought to include animated models too, mesh deformation isn't that expensive so I can see why there's little benefit. Especially as I realized per-pixel lighting would still be recalculated each frame, specularity also depends on camera angle not just model movement... technically we could frameskip that too but I'm getting way ahead of myself for what would likely be a tiny benefit

Could this still work for lights though? Recalculating shadows when something moves in radius of a light is a big cost, even if it's gotten much better with the latest changes. A shadow recalculation LOD may give a nice boost.

We could test the benefit with an even simpler change: A setting to cap all shadow updates to a fixed FPS. This would probably be a few lines of code so if you can give me a pointer I may be able to modify my local engine clone to try it. If it offers a benefit it can be made distance-based later.

Another way would be to make a light's number of samples slowly decrease with distance, the furthest lights dropping to just one sample like sharp / stencil shadows: Shadow samples also have a big impact.

What do you think of this solution as a form of light LOD, maybe mixed with just a shadow update LOD? These actually sound like they make sense; If you think it's worth it I can post those two on the tracker so they're not forgotten.

[stgatilov]

In case of stencil shadows, there are two parts:

1. computing shadow volume (on CPU)
2. rendering shadow volume (on GPU)

Shadow volume computation is cached in interactions table for each light+entity pair. It is recomputed only when at least one of them changes.
Rendering shadow volume cannot be cached because it depends on camera too much.

So with stencil shadows it is already good.
Yeah, it is possible to skip shadow volumes computation for distant lights.

In case of shadow maps, there is also some CPU filtering and actual rendering on GPU.

CPU filtering is also cached and only recomputed on light or entity change, but on GPU shadow maps are rendered from scratch every frame. And this is not good: at the very least, static light which has static set of shadow casters should not have its shadow map recomputed every frame. Perhaps it is also good to have shadow map of static entities saved persistently, then every frame copy it and render only dynamic entities on top.

I have this issue in my mind for some time already.
There are some issues here, since shadow maps which are not recomputed must not change their location in shadow map atlas, and thus they can block other shadow maps from taking proper space (basically, typical memory allocation issues). Also if we try caching only static portion of shadow map, then we have to create another shadow atlas for them, which raises memory requirements dramatically.

Right now it is not possible to skip anything for shadow maps at all.
It might become possible after I implement reusing shadow maps in static cases.

Overall, I can't say I like this idea.
Even if light is distant, it is still ugly when shadow does not move in sync with shadow caster.
But shadow maps should definitely be optimized for static cases.