[Feature discussion] Depth of Field effect

[MirceaKitsune]

10 minutes ago, datiswous said:

It might be good to know if devs are willing to integrate a dof effect, if it works, in the engine. Otherwise you have to fork tdm to implement it. Most current posting devs seem uninterested, just saying.

Now that I understand what I'm working on better, I'm trying create a double use shader: It can be used as a view overlay material to get DOF by piping in the depth texture, as well as a surface material to get blurry glass (eg: condensation) by using a dirt map instead. heatHazeWithMaskAndBlur has a constant hardcoded blur, you can't use it to get dirty glass that blurs selectively, the same shader would thus allow glass windows with that effect.

At least the shader will hopefully be acceptable. If for any reason we can't get DoF in the vanilla engine, I can at least create a simple pk4 mod for it this way, albeit that's a lot more limited so I hope it can be done the right way. This is something I've always wanted in TDM so in my book it's worth it, though only if I can get a beautiful and clean result... this will likely require using multiple samples where the image is offset in 4 directions for each one.

[MirceaKitsune]

I'm likely going to shelf this one, given there isn't much support from anyone else while any method of implementing it properly is beyond my knowledge. I did want to suggest one more idea that came to mind first, which would have multiple uses and also be per-map without inducing changes by default.

Could we get ambiental blur with an ambient light or fog light texture designed to support blurring by distance? It would require a custom fog shader if one is possible for this purpose, but this approach would allow mappers to add and customize it to their own liking and do so per area. It can for example be used underwater to make it more blurry the lower it goes, in places like a cold cave or refrigerator, or to simulate blurry mist which in some cases appears to be a thing you can see with the naked eye early in the morning. A nice photo to go with the later idea... it's not DOF per say but shows how mist with a little background blur can make sense, blur and color intensity could be adjusted separately so anyone can tweak it to their desired purpose.

[MirceaKitsune]

There was a thought still stuck in my mind that I wanted to put up. I noticed DOF as a post-processing feature isn't desired and probably won't happen, I'll likely attempt it as a new fog light shader at some point. But an interesting concept popped to mind which I felt would be interesting at least for the sake of discussion.

People complain of how expensive anti-aliasing is, something I can fully confirm as even 2x or 4x reduce more than 10 / 30 FPS at best. I suggested implementing shader based AA like FXAA for this very reason but that's another story. As its name suggests, MSAA uses multiple samples so each pixel is rendered several times with different interpretations. I believe MSAA is a fixed OpenGL feature in the graphics driver and likely can't be modified directly.

My thought was: If we're trying to get rid of sharp edges and may use multiple samples to do so, a custom AntiAliasing system could do it using positional / angular variation per sample. Think of it as physically based sampling, which as a consequence can also be used to achieve a DOF like effect by naturally emulating the f-stops of a real camera.

This was inspired by how the Eevee rendering engine in Blender seems to function: You give it a fixed number of samples. Each sample is used to randomize various things in the frame, including pixel directions to emulate DOF. Even the soft shadow samples are controlled by it... this might be an interesting way to get soft shadowing, which could be achieved by having each sample interpret the light source as being at a slightly different position.

Unfortunately this would likely be slower than MSAA: It requires rendering the camera multiple times per frame from a slightly different position / rotation to mix up the average result... this must be excruciatingly slow, even at a low value like 4 samples which wouldn't be enough for good looking results. As such it's likely an experiment that won't be considered worth attempting; I'm mainly curious at a concept level how such a thing could work and look like... the technique must have been done before in realtime engines, there's likely even a term for it though I couldn't find it.

Edited December 4, 2022 by MirceaKitsune