Jump to content
The Dark Mod Forums

[Feature discussion] Depth of Field effect


MirceaKitsune

Recommended Posts

10 minutes ago, datiswous said:

It might be good to know if devs are willing to integrate a dof effect, if it works, in the engine. Otherwise you have to fork tdm to implement it. Most current posting devs seem uninterested, just saying.

Now that I understand what I'm working on better, I'm trying create a double use shader: It can be used as a view overlay material to get DOF by piping in the depth texture, as well as a surface material to get blurry glass (eg: condensation) by using a dirt map instead. heatHazeWithMaskAndBlur has a constant hardcoded blur, you can't use it to get dirty glass that blurs selectively, the same shader would thus allow glass windows with that effect.

At least the shader will hopefully be acceptable. If for any reason we can't get DoF in the vanilla engine, I can at least create a simple pk4 mod for it this way, albeit that's a lot more limited so I hope it can be done the right way. This is something I've always wanted in TDM so in my book it's worth it, though only if I can get a beautiful and clean result... this will likely require using multiple samples where the image is offset in 4 directions for each one.

Link to comment
Share on other sites

I'm likely going to shelf this one, given there isn't much support from anyone else while any method of implementing it properly is beyond my knowledge. I did want to suggest one more idea that came to mind first, which would have multiple uses and also be per-map without inducing changes by default.

Could we get ambiental blur with an ambient light or fog light texture designed to support blurring by distance? It would require a custom fog shader if one is possible for this purpose, but this approach would allow mappers to add and customize it to their own liking and do so per area. It can for example be used underwater to make it more blurry the lower it goes, in places like a cold cave or refrigerator, or to simulate blurry mist which in some cases appears to be a thing you can see with the naked eye early in the morning. A nice photo to go with the later idea... it's not DOF per say but shows how mist with a little background blur can make sense, blur and color intensity could be adjusted separately so anyone can tweak it to their desired purpose.

foggy-forest-35401-1920x1200.jpg

  • Like 2
Link to comment
Share on other sites

  • 4 weeks later...

There was a thought still stuck in my mind that I wanted to put up. I noticed DOF as a post-processing feature isn't desired and probably won't happen, I'll likely attempt it as a new fog light shader at some point. But an interesting concept popped to mind which I felt would be interesting at least for the sake of discussion.

People complain of how expensive anti-aliasing is, something I can fully confirm as even 2x or 4x reduce more than 10 / 30 FPS at best. I suggested implementing shader based AA like FXAA for this very reason but that's another story. As its name suggests, MSAA uses multiple samples so each pixel is rendered several times with different interpretations. I believe MSAA is a fixed OpenGL feature in the graphics driver and likely can't be modified directly.

My thought was: If we're trying to get rid of sharp edges and may use multiple samples to do so, a custom AntiAliasing system could do it using positional / angular variation per sample. Think of it as physically based sampling, which as a consequence can also be used to achieve a DOF like effect by naturally emulating the f-stops of a real camera.

This was inspired by how the Eevee rendering engine in Blender seems to function: You give it a fixed number of samples. Each sample is used to randomize various things in the frame, including pixel directions to emulate DOF. Even the soft shadow samples are controlled by it... this might be an interesting way to get soft shadowing, which could be achieved by having each sample interpret the light source as being at a slightly different position.

Unfortunately this would likely be slower than MSAA: It requires rendering the camera multiple times per frame from a slightly different position / rotation to mix up the average result... this must be excruciatingly slow, even at a low value like 4 samples which wouldn't be enough for good looking results. As such it's likely an experiment that won't be considered worth attempting; I'm mainly curious at a concept level how such a thing could work and look like... the technique must have been done before in realtime engines, there's likely even a term for it though I couldn't find it.

Edited by MirceaKitsune
Link to comment
Share on other sites

  • 8 months later...

Glad to hear it's not just me who still wants this. I believe others stated they find DOF useless or not worth the effort; I didn't agree either but seeing everyone was against it I stopped focusing on the matter. I'd still love Depth-of-Field as a feature: I have no doubt it would make every FM look even more beautiful and amazing if done right.

I'm remembering a bit of what I did in my final test. I think I never shared it at the time, I should do so as a template for others to experiment with. I don't think I can get more done on my own, please see if you can make anything of it.

DOF.zip

To test it I believe you'd unpack that in the active FM then set "g_testPostProcess dof" in the console to activate it. Use "reloadDecls" and "reloadGLSLPrograms" to test changes without needing to restart TDM, this way you can just Alt + Tab switch to make a modification then see what it does.

I think in that archive I was able to copy the basic blur filter used underwater and draw the depth pass: It draws copies of the view with slight offsets to create the illusion of blur. What I got stuck on is making the depth pass influence this blur, I couldn't find a way to convert the brightness of the depth image into custom coordinates for the offset passes and distort them accordingly.

  • Like 1
Link to comment
Share on other sites

1 hour ago, wesp5 said:

MirceaKitsune, I believe you should make a mod with all your improvements inside. Because it usually is very hard to get something into the base game ;)!

I should keep this in mind, though note that a custom shader can only be done into the base game: It requires editing the engine and adding this to the post-process phase. The technique I'm using for testing is a custom variable to test custom shaders, it even relies on a fake material mapped to the screen. At the moment I'm stuck on this project, especially as I have little to no experience with shaders... I only shared it in hopes someone else can make use of it.

For proper quality this should be done with a Gaussian Blur or Fast Gaussian filter, which the underwater view should also be switched to at that point. Do we have one compatible with the shaders we're using? Maybe Doom3BFG already did something we can use and we could snatch it from there?

Edited by MirceaKitsune
Link to comment
Share on other sites

There is a whole mess of graphics mods for Doom 3 you could peek at. Sikkmod for example is not based off BFG and has a depth of field effect when reloading weapons which sounds like basically what you are looking for and I doubt it requires engine level changes to work:

At a minimum you could download one of these mods and given doom3 stuff is typically all text formats in .pk4s there might be enough there you can just read to figure out how it works.

As far as it’s potential inclusion in the game I have no real opinion - though given it would likely be optional I don’t see it hurting anything either.

  • Like 2

-=  IRIS  =-    ♦    = SLL =

Link to comment
Share on other sites

On 11/3/2022 at 9:51 AM, Xolvix said:

I generally dislike DOF in games except in three circumstances: (1) in-game cinematics, (2) characters engaging in conversations with someone, or (3) some sort of in-game menu system. Basically so long as the depth of field is being used in non-gameplay parts of the game, that's completely fine and often enhances the result. But TDM is virtually all gameplay and doesn't have such systems, so all DOF is going to do is make the game harder by making things in the distance less clearly visible.

All that said, I ain't got a problem with it being integrated into the engine. There's a handful of missions that do have cinematics, albeit small ones, so it'd be kinda cool to see it being used in such situations.

For a looking glass too.

Task is not so much to see what no one has yet seen but to think what nobody has yet thought about that which everybody see. - E.S.

Link to comment
Share on other sites

1 hour ago, lowenz said:

For a looking glass too.

My goal was to make / find something that works both as a screen post-process filter for DOF, but can also be placed on transparent surfaces for blurry glass like condensation. We have distorted glass but still no glass that blurs, especially not by using a grayscale map to determine the level of blur so you can have realistic stains.

Link to comment
Share on other sites

20 hours ago, Wellingtoncrab said:

There is a whole mess of graphics mods for Doom 3 you could peek at. Sikkmod for example is not based off BFG and has a depth of field effect when reloading weapons which sounds like basically what you are looking for and I doubt it requires engine level changes to work:

At a minimum you could download one of these mods and given doom3 stuff is typically all text formats in .pk4s there might be enough there you can just read to figure out how it works.

As far as it’s potential inclusion in the game I have no real opinion - though given it would likely be optional I don’t see it hurting anything either.

Sikkmod has some cool shaders indeed but wasn't the old ARB2 render totally removed in TDM and now only uses the modern GLSL one? At lest that is what I seem to have read some time ago.

So if the ARB2 render was removed, sikkmod shaders will not work in modern TDM as is they would need to be converted to GLSL, don't know how hard that is.

  • Like 1
Link to comment
Share on other sites

6 hours ago, lowenz said:

Yes, they're ARB-based

Wasn't TDM moving away from ARB to GLSL? Actually I think it was but the transition was incomplete so we're still using the former. Wonder what it would be best to do this under today, seems like the later is ideal if it's supported by both post-process as well as material shaders.

Link to comment
Share on other sites

5 hours ago, MirceaKitsune said:

Wasn't TDM moving away from ARB to GLSL? Actually I think it was but the transition was incomplete so we're still using the former. Wonder what it would be best to do this under today, seems like the later is ideal if it's supported by both post-process as well as material shaders.

The *DOOM3* shaders are ARB2 ('cause of old GeForce support)

carmack plan + arb2 - OpenGL / OpenGL: Advanced Coding - Khronos Forums

Edited by lowenz

Task is not so much to see what no one has yet seen but to think what nobody has yet thought about that which everybody see. - E.S.

Link to comment
Share on other sites

So it should be possible to find a generic GLSL shader for Gaussian Blur and / or DOF and integrate it. Just not something I ever worked with and understand well, especially as it requires engine changes. This discussion also came up on Discord though and several other players mentioned they want this, so there's hope and a reason for it happening after all... just hope there's someone with knowledge in shaders who can and wishes to help out.

Link to comment
Share on other sites

On 12/4/2022 at 4:05 PM, MirceaKitsune said:

This was inspired by how the Eevee rendering engine in Blender seems to function: You give it a fixed number of samples. Each sample is used to randomize various things in the frame, including pixel directions to emulate DOF. Even the soft shadow samples are controlled by it... this might be an interesting way to get soft shadowing, which could be achieved by having each sample interpret the light source as being at a slightly different position.

Unfortunately this would likely be slower than MSAA: It requires rendering the camera multiple times per frame from a slightly different position / rotation to mix up the average result... this must be excruciatingly slow, even at a low value like 4 samples which wouldn't be enough for good looking results. As such it's likely an experiment that won't be considered worth attempting; I'm mainly curious at a concept level how such a thing could work and look like... the technique must have been done before in realtime engines, there's likely even a term for it though I couldn't find it.

TDM has such an implementation, called something like camera jittering.

It was used to create high-quality screenshots. It jitters both camera and light positions, which results in perfect antialiasing and soft shadows. But it is very slow, and the resulting images are basically a lie.

As far as I remember, it stopped working at some moment because the engine is constantly being modified.
Anyway, this idea is useless to combat complaint about MSAA being slow.


And simply blurring the whole image would result in trash of course.
 

Link to comment
Share on other sites

3 hours ago, stgatilov said:

TDM has such an implementation, called something like camera jittering.

It was used to create high-quality screenshots. It jitters both camera and light positions, which results in perfect antialiasing and soft shadows. But it is very slow, and the resulting images are basically a lie.

As far as I remember, it stopped working at some moment because the engine is constantly being modified.
Anyway, this idea is useless to combat complaint about MSAA being slow.


And simply blurring the whole image would result in trash of course.

Interesting to know! Such jittering could have technically addressed both AA and DOF the right way. But since it means multiple renders per frame that would be extremely slow and not worth it as a solution.

The simple and realistic fix seems like a simple Bokeh filter who's intensity is controlled by a range in the depth pass. I remember getting my custom shader to read and render the depth image as a test, it just needs an actual blur shader running with it and to be done in the engine. Then it can also be used underwater and optionally when taking damage or low on health.

Link to comment
Share on other sites

3 hours ago, MirceaKitsune said:

The simple and realistic fix seems like a simple Bokeh filter who's intensity is controlled by a range in the depth pass. I remember getting my custom shader to read and render the depth image as a test, it just needs an actual blur shader running with it and to be done in the engine. Then it can also be used underwater and optionally when taking damage or low on health.

You are talking about DOF effect exclusively, I don't see how antialiasing can be achieved this way.

Also, blurring does not actually antialias, except if it is strong blur.

Link to comment
Share on other sites

20 minutes ago, stgatilov said:

You are talking about DOF effect exclusively, I don't see how antialiasing can be achieved this way.

Also, blurring does not actually antialias, except if it is strong blur.

I did at some point wonder if the same shader could be used to achieve both effects. My idea was we could have a single blur post-process: Its minimum range would be something like 0.5 pixels to anti-alias sharp edges (both geometry and alpha textures) on top of which the depth map may increase it to also achieve distance blur. Would be a neat trick to solve two problems at once, but it surely won't work the way I imagine it and either not be effective or blur too much. Thus DOF should probably be its own thing, with antialiasing done as a separate shader if and when that happens... at best they may share and inherit from a single component if that's optimal.

Edited by MirceaKitsune
Link to comment
Share on other sites

2 hours ago, AluminumHaste said:

That jittering still works as far as I know, though I think with some effects, you can get weird artifacts.

Typing "screenshot 2560 1440 32" still works.

 

SOb68wY.png

Oh gosh: That looks incredible when seeing the full image! The FPS it must cost though... so sad it's not practical 😭

Technically speaking though, it's an easy way to achieve DOF as well as antialiasing in one go: Just jitter the camera angle which will cause greater differences between pixels (thus blur) the further something is. Meanwhile interpreting light sources at slightly offset positions per sample offers an alternative way to get distance based soft shadows, some performance would be recovered by no longer needing multiple samples per light. There's quite a few things multiple renders per frame can achieve!

Is it possible to at least reuse some render data between samples for some kind of performance optimization? Actually I wonder: What if we use past frames so that we don't have to render extra ones in between? This might be what temporal antialiasing does, except we could use it for even more things! I'm actually intrigued in an experiment to test this :)

For example: Let's say we set it to 16 frames. This means the past 16 frames that have been rendered are blended together and you see a mix of their total on the screen. At the same time each frame interprets the camera and lights at an ever so slightly different position / angle using a seed, causing the end result to be a soft average.

One problem I see is with fast camera movements: If you look around or run too fast it will likely cause some form of motion blur... then again this might be a feature not a bug, we don't have any motion-blur anyway and it's not like it would be wrong to even if ghosting is the bad sort. Another problem is you could see flickering as lights are interpreted at randomized positions... the way I'm imagining this may cause everything to appear as if it's vibrating which is no good.

Edited by MirceaKitsune
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

  • Recent Status Updates

    • Ansome

      Finally got my PC back from the shop after my SSD got corrupted a week ago and damaged my motherboard. Scary stuff, but thank goodness it happened right after two months of FM development instead of wiping all my work before I could release it. New SSD, repaired Motherboard and BIOS, and we're ready to start working on my second FM with some added version control in the cloud just to be safe!
      · 1 reply
    • Petike the Taffer  »  DeTeEff

      I've updated the articles for your FMs and your author category at the wiki. Your newer nickname (DeTeEff) now comes first, and the one in parentheses is your older nickname (Fieldmedic). Just to avoid confusing people who played your FMs years ago and remember your older nickname. I've added a wiki article for your latest FM, Who Watches the Watcher?, as part of my current updating efforts. Unless I overlooked something, you have five different FMs so far.
      · 0 replies
    • Petike the Taffer

      I've finally managed to log in to The Dark Mod Wiki. I'm back in the saddle and before the holidays start in full, I'll be adding a few new FM articles and doing other updates. Written in Stone is already done.
      · 4 replies
    • nbohr1more

      TDM 15th Anniversary Contest is now active! Please declare your participation: https://forums.thedarkmod.com/index.php?/topic/22413-the-dark-mod-15th-anniversary-contest-entry-thread/
       
      · 0 replies
    • JackFarmer

      @TheUnbeholden
      You cannot receive PMs. Could you please be so kind and check your mailbox if it is full (or maybe you switched off the function)?
      · 1 reply
×
×
  • Create New...