Jump to content

[Feature discussion] Depth of Field effect


MirceaKitsune
 Share

Recommended Posts

This is something I wanted to try implementing myself more than an year ago, but ultimately figured it's overly advanced engine stuff for me: If no one else wants to I may still attempt it someday, in the meantime I wanted to write and ask about it at least. I don't know if this was previously discussed and I hope I'm not repeating something old: I've been seeing all the amazing changes done recently like volumetric lights... watching how far TDM has come makes me excited to see it get even further and become even more amazing.

One of the last remaining renderer features I feel we're missing is the Depth Of Field effect. I do believe having this would make every environment in each FM even more incredible and realistic, while bringing TDM even closer to having top notch visuals on par with most modern engines (if we exclude PBR materials). Existing shaders and tutorials for the GLSL implementation are out there possibly even an existing idTech4 port; Initially I hoped it might be as easy as grabbing a freely licensed shader and integrating it in the engine, but usually things are always harder in reality. For starters I'm curious if others are with me in wanting this, if there are any plans for adding the feature, and if there remain any major roadblocks in the engine preventing it... engine wise I know the shader just needs access to the Z-buffer and can typically work its magic from the depth map.

The best implementation I'm hoping we can get is the dynamic one, which smoothly adjusts focus based on where the player is looking (distance of the center pixel in the middle of the screen). By default the effect definitely shouldn't be excessive, only blurring distant or near things very slightly unless you're looking at a surface right in front of you. I don't believe it needs to be enabled by default as long as it's in the menu, like the sharpness filter it shouldn't bother anyone who doesn't want it and make things look beautiful for those that do.

  • Like 1
Link to comment
Share on other sites

Please not... I absolutely detest that. It's not realistic either. You focus your eyes on things, not the environment, or some kind of optic lense in front of your eye, which does it automatically.

In every game which has that, it simply doesn't work for me. The focus is wrong, and the effect is usually completely exaggerated as well.

Edited by chakkman
  • Like 2
Link to comment
Share on other sites

See also:

I think it could be useful for ingame cinamatics, or for intro's based on screenshots.

 

8 minutes ago, chakkman said:

It's not realistic either.

Indeed, I think it's implemented for it's cool looks, not for it's realism. Loads of stuff in TDM is unrealistic.

It's used in photograpy and film, so it gives you that look.

Edited by datiswous
Link to comment
Share on other sites

Personally I don't like depth of field, in real gameplay at lest, for screenshots and cinematics is fine but if you want to try including it yourself, sikkmod for Doom 3 has a ARB shader for it, so if you learn basic Doom 3 ARB and TDM GLSL shader code, you could translate it to TDM, both engines follow the same basic principles.

 

  • Like 2
Link to comment
Share on other sites

There are too many other higher priority render issues to address in comparison with what a feature like this would offer.

If a professional screen-shot artist such as https://deadendthrills.com/  came to us and said that they would promote TDM if we had DOF effects we might consider an implementation but otherwise this is far out there in back-burner territory.

Even then, if a screen-shot artist wants DOF in TDM they can use Reshade.

 

  • Like 1

Please visit TDM's IndieDB site and help promote the mod:

 

http://www.indiedb.com/mods/the-dark-mod

 

(Yeah, shameless promotion... but traffic is traffic folks...)

Link to comment
Share on other sites

11 minutes ago, HMart said:

Personally I don't like depth of field, in real gameplay at lest, for screenshots and cinematics is fine but if you want to try including it yourself, sikkmod for Doom 3 has a ARB shader for it, so if you learn basic Doom 3 ARB and TDM GLSL shader code, you could translate it to TDM, both engines follow the same basic principles.

Personally I think it's unrealistic if it's overdone: It needs to represent not what a camera lens sees but what a person's eyes will, albeit that's different for individuals like people who wear glasses. Anyone can however notice that if you put a finger in front of your eyes then focus either on it or the background the other will become blurry: I don't think that's a bad effect to simulate, which would be both realistic and good looking... for our environments especially I think it would look incredible.

It's very much a thing of taste though. Like I said I don't think it should be enabled by default, heck I'd be fine with it even as a hidden cvar I can put in my autoexec.cfg (seems frob highlight ended up like that). If so few others like this effect enough to want to work on it though, I may have to try it myself at some point; Wonder if someone could at least help with the code part so I could try doing the dynamic shader myself based on examples, plus confirm it would be accepted so I wouldn't work for nothing.

Also do we still use ARB shaders? I thought we switched everything to GLSL years ago. Also curious if our shaders currently have access to the information needed to produce this effect, namely the Z buffer (grayscale depth pass).

2 hours ago, datiswous said:

I gues making the render scale lower on distance is maybe possible? Making it based on LOD?

I wonder if using an independent resolution per area is possible though I strongly doubt it: I think it's an universal camera property, doubt it could be easily made to work per-pixel. It would be interesting since then it could also improve / recover some performance by rendering near / distant stuff at a smaller resolution?

2 minutes ago, New Horizon said:

Not really interested in it.  I don't care for the effect.  It was dabbled with in TDM quite some years ago.

Is there anything left of that on the forum or elsewhere? Maybe it could help with a new implementation.

Link to comment
Share on other sites

2 hours ago, MirceaKitsune said:

Personally I think it's unrealistic if it's overdone: It needs to represent not what a camera lens sees but what a person's eyes will, albeit that's different for individuals like people who wear glasses.

The ultimate goal of on-screen computer graphics is photorealistic rendering, in which case modelling the camera lens is the only option that makes sense.
If you want to aim for eye-realistic rendering, it belongs to VR mod: there you can use eye tracking to detect focus distance for the eyes.

Overall, I personally don't like the idea of Depth of Field effect in TDM, and I suppose most of people won't.

But yeah, you can probably use g_testPostProcess cvar to add a full-screen effect, using _currentRender and _currentDepth as input texture, and writing custom shader.

  • Like 2
  • Thanks 1
Link to comment
Share on other sites

I don't see how DoF can possibly work in a game which isn't tracking your eyes. How does the game know what you are looking at? Your eyes could be focussed anywhere on screen, which might show objects at a variety of distances. If the depth is taken from the central aim point, that means that instead of just moving your eyes to look at something on screen, you'd also need to move the mouse to remove the artificial blur.

DoF looks nice in photos, and as stgatilov says it might work in a VR simulation with eye tracking, but for a 2D image on a standard screen I can't see it adding any value.

  • Like 3
Link to comment
Share on other sites

1 hour ago, stgatilov said:

The ultimate goal of on-screen computer graphics is photorealistic rendering, in which case modelling the camera lens is the only option that makes sense.
If you want to aim for eye-realistic rendering, it belongs to VR mod: there you can use eye tracking to detect focus distance for the eyes.

Overall, I personally don't like the idea of Depth of Field effect in TDM, and I suppose most of people won't.

But yeah, you can probably use g_testPostProcess cvar to add a full-screen effect, using _currentRender and _currentDepth as input texture, and writing custom shader.

Thanks, that's very useful! Does this mean it can be doable without requiring engine changes at least as an initial proof of concept? If so I could create this as a mod and pk4 addon at very least. Also is there a command to reload such effects in realtime, so I can alt-tab switch to make changes instead of needing to restart TDM for each test?

I was thinking we already have a blur effect when the player is underwater: It's a constant blur but maybe not too hard to have the intensity adjusted by a range in the depth buffer? I'll have to find its file see how that's implemented and how to scale it per area. I wonder if it will require a script too, to read custom cvar settings or smoothly adjust focus based on the value of the center pixel... will have to see how much existing examples help when I'll look into it.

  

8 minutes ago, OrbWeaver said:

I don't see how DoF can possibly work in a game which isn't tracking your eyes. How does the game know what you are looking at? Your eyes could be focussed anywhere on screen, which might show objects at a variety of distances. If the depth is taken from the central aim point, that means that instead of just moving your eyes to look at something on screen, you'd also need to move the mouse to remove the artificial blur.

DoF looks nice in photos, and as stgatilov says it might work in a VR simulation with eye tracking, but for a 2D image on a standard screen I can't see it adding any value.

Most engines use DOF for years, pretty much every AAA engine these days has it... I enable it in Godot where it's also a builtin camera effect. The best solution for non-VR is using the center pixel to measure distance and adjust, I've seen that approach before and personally like the feel; Ideally I could use the average value of a circle around the crosshair, the average grayscale value determining the focus point.

Edited by MirceaKitsune
Link to comment
Share on other sites

23 minutes ago, MirceaKitsune said:

Thanks, that's very useful! Does this mean it can be doable without requiring engine changes at least as an initial proof of concept? If so I could create this as a mod and pk4 addon at very least. Also is there a command to reload such effects in realtime, so I can alt-tab switch to make changes instead of needing to restart TDM for each test?

You can try to use "g_testPostProcess rotoedge", then implement a similar material and shaders.
The last time I checked, rotoedge worked properly.
If it works, then you can get away without rebuild.

You can also call reloadGLSLPrograms to reload modified shader without restarting TDM.
And you can use reloadDecls to reload modified material.

Link to comment
Share on other sites

46 minutes ago, stgatilov said:

You can try to use "g_testPostProcess rotoedge", then implement a similar material and shaders.
The last time I checked, rotoedge worked properly.
If it works, then you can get away without rebuild.

You can also call reloadGLSLPrograms to reload modified shader without restarting TDM.
And you can use reloadDecls to reload modified material.

Thanks again. It says "material not found" so it's probably named differently, but I tried it with other materials as a test and they show over the screen so technically this should work. Digging through the files now but once I find what I need I may just have the requirement to get me started. Once more never underestimate how awesomely idTech 4 was designed :) This is what I get with "g_testPostProcess xray_glasses_overlay_mixed_warped" for instance:

624193006_inplainsight(2022-11-0220-13-25)(-762.05627.63-299.75).thumb.jpg.130f6994fc98477d8769e64569b56eff.jpg

I vaguely remember that long ago there was discussion about making water and / or stained glass blur with distance rather than constantly. Was this ever achieved to use as an example? It's what tdm_water.mtr - textures/water_source/water_blurred seems to be doing, it uses heatHazeWithMaskAndBlur.vfp which can probably be simplified as I only need the blur component. I can't find where vfp files are stored though, glprogs only contains fs / vs / glsl files.

Edited by MirceaKitsune
Link to comment
Share on other sites

dof
{
	description	"Depth of Field effect."

	discrete
	nonsolid
	noshadows
	translucent

	{
		program heatHazeWithMaskAndBlur.vfp
		vertexParm 0 0
		vertexParm 1 0
		fragmentMap 0 _currentRender
		fragmentMap 1 textures/sfx/vp1
		fragmentMap 2 textures/water_source/vp_water
		fragmentMap 3 _currentDepth
	}
}

With "g_testPostProcess dof" the following effect can be achieved:

1833316313_inplainsight(2022-11-0221-19-52)(-763.49624.25-299.75).thumb.jpg.e06b7a339f6ee1606ca3382460732a70.jpg

But the blur is constant, I can't find a way to tweak even the global intensity let alone per area: vertexParm 0 and 1 are the distortion time and strength, 2 and beyond don't appear to do anything. Each fragmentMap doesn't seem to be modifiable so it would be a constant effect regardless. Clearly going to need my own fragment / vertex program based on that, may work without needing a custom GLSL shader though.

Link to comment
Share on other sites

2 hours ago, OrbWeaver said:

I don't see how DoF can possibly work in a game which isn't tracking your eyes. How does the game know what you are looking at? Your eyes could be focussed anywhere on screen, which might show objects at a variety of distances. If the depth is taken from the central aim point, that means that instead of just moving your eyes to look at something on screen, you'd also need to move the mouse to remove the artificial blur.

DoF looks nice in photos, and as stgatilov says it might work in a VR simulation with eye tracking, but for a 2D image on a standard screen I can't see it adding any value.

In every game I've played that has DoF (Sykrim, Elden Ring etc) it just blurs as if you're focusing on what's in front of you, but the effect is increased the closer to the viewport the object is. So if you walk right up to a post and put it in the middle of your screen, the background will have large amounts of blur.

Conversely, if you're not close to the object that's over the center of the screen, then the effect is barely there.

I usually end up turning it off as I'm often looking at things that are off to the side of my screen with my actual eyes and the game has blurred it all to oblivion.

  • Like 1

I always assumed I'd taste like boot leather.

 

Link to comment
Share on other sites

17 hours ago, MirceaKitsune said:

The best implementation I'm hoping we can get is the dynamic one, which smoothly adjusts focus based on where the player is looking (distance of the center pixel in the middle of the screen). By default the effect definitely shouldn't be excessive, only blurring distant or near things very slightly unless you're looking at a surface right in front of you.

I would expect a completely static version only blurring farther away objects to be least intrusive. You can't know where the player actually looks at and mouse-"scanning" the screen gets unimmersive pretty fast.

But like very cheap fog, a deapth of field effect could be used to enable more aggressive level of detail reductions for far-away objects. So i would see the effect more as a performance optimization for outdoor areas. It might enable mappers to create more realistic city streets which don't have to do an artifical sharp bend every few meters to allow for geometry culling.

P.S.: Don't forget to turn the effect off when using the monocular.

Link to comment
Share on other sites

Making it static would be a dozen times easier too so I might do that after all. I'd like to only add some slight distance blur, possibly near blur too but only for very close objects like if you're standing as close as the engine lets you.

For now I'm stuck understanding how shaders work. When do I even want glsl or fs or vs, how are they each loaded for which purpose? The proper way seems implementing it as a stage, using the quality gaussian blur shader already included... however this requires engine changes as each stage has its own C++ counterpart, also the only way to have it use cvar settings. I'd have to rely on the material trick for my implementation... then if what I get working would be good enough I'd need a dev to add it to the engine, which would require more changes to the shader so I dunno how to best go about it.

Link to comment
Share on other sites

I generally dislike DOF in games except in three circumstances: (1) in-game cinematics, (2) characters engaging in conversations with someone, or (3) some sort of in-game menu system. Basically so long as the depth of field is being used in non-gameplay parts of the game, that's completely fine and often enhances the result. But TDM is virtually all gameplay and doesn't have such systems, so all DOF is going to do is make the game harder by making things in the distance less clearly visible.

All that said, I ain't got a problem with it being integrated into the engine. There's a handful of missions that do have cinematics, albeit small ones, so it'd be kinda cool to see it being used in such situations.

A word of warning, Agent Denton. This was a simulated experience; real LAMs will not be so forgiving.

Link to comment
Share on other sites

Just for info (I don't know if it helps):

18 hours ago, MirceaKitsune said:

I can't find where vfp files are stored though, glprogs only contains fs / vs / glsl files.

vfp is for ARB shaders (I think): https://wiki.thedarkmod.com/index.php?title=Replacing_ARB_shaders . Files heatHazeWithMaskAndBlur.fs and  heatHazeWithMaskAndBlur.vs are present.

Link to comment
Share on other sites

2 hours ago, datiswous said:

Just for info (I don't know if it helps):

vfp is for ARB shaders (I think): https://wiki.thedarkmod.com/index.php?title=Replacing_ARB_shaders . Files heatHazeWithMaskAndBlur.fs and  heatHazeWithMaskAndBlur.vs are present.

I remember now: There was a goal to convert every ARB shader to GLSL... the right way would be doing it all in GLSL from the start then. I'm not seeing any examples of this being done via material without requiring engine edits: Every vertexProgram / fragmentProgram entry in the default materials points to a vfp file. Does it support using glsl there?

The issue with heatHazeWithMaskAndBlur is nothing seems to adjust the blur intensity. Am I missing a parameter in my test or is it simply designed this way and will need to be forked into something else?

5 hours ago, Xolvix said:

All that said, I ain't got a problem with it being integrated into the engine. There's a handful of missions that do have cinematics, albeit small ones, so it'd be kinda cool to see it being used in such situations.

I definitely like it in-world too, but yeah I seem to be in a minority of TDM players that do. Hopefully no one would object to it being integrated into the engine, but I'll surely have to do the work for it which I don't mind as long as I can figure it out.

And yeah: As a separate feature I was thinking if we should blur the background like we do underwater when the player is accessing a readable, which doesn't need to be DOF per say and is also easy to do with existing shaders. If others agree it would look better I can try that as a separate change sometime.

Edited by MirceaKitsune
Link to comment
Share on other sites

On 11/2/2022 at 11:12 AM, MirceaKitsune said:

Is there anything left of that on the forum or elsewhere? Maybe it could help with a new implementation.

I honestly couldn't say.  It was so long ago.  There certainly wouldn't be anything left of it ingame after all the work that has been done.

Link to comment
Share on other sites

6 minutes ago, New Horizon said:

I honestly couldn't say.  It was so long ago.  There certainly wouldn't be anything left of it ingame after all the work that has been done.

Fair enough. I started working on the custom shader... sticking to ARB with material overlay as I have existing examples to practically work with, it can be ported someday when the other stuff would be. This approach is especially handy as it allows using the shader on surfaces too, if you want to have a foggy window that selectively blurs what's behind it... won't include deform as that would make everything too complicated, that can be used on its own layer.

The part I'm not fully clear on is how to scale the effect per pixel as it's still constant. Existing examples for deforms work by having a texture push the vertices around, in my case I want a constant offset to generate each blur pass but the distance of this offset needs to be based on the value on the Z buffer. Maybe I can just use the depth map itself as the deform texture, bur that's a grayscale image not a normal map. How would I read the intensity at a certain pixel of the second fragment map?

Link to comment
Share on other sites

On 11/2/2022 at 6:13 AM, MirceaKitsune said:

This is something I wanted to try implementing myself more than an year ago, but ultimately figured it's overly advanced engine stuff for me: If no one else wants to I may still attempt it someday, in the meantime I wanted to write and ask about it at least. I don't know if this was previously discussed and I hope I'm not repeating something old: I've been seeing all the amazing changes done recently like volumetric lights... watching how far TDM has come makes me excited to see it get even further and become even more amazing.

One of the last remaining renderer features I feel we're missing is the Depth Of Field effect. I do believe having this would make every environment in each FM even more incredible and realistic, while bringing TDM even closer to having top notch visuals on par with most modern engines (if we exclude PBR materials). Existing shaders and tutorials for the GLSL implementation are out there possibly even an existing idTech4 port; Initially I hoped it might be as easy as grabbing a freely licensed shader and integrating it in the engine, but usually things are always harder in reality. For starters I'm curious if others are with me in wanting this, if there are any plans for adding the feature, and if there remain any major roadblocks in the engine preventing it... engine wise I know the shader just needs access to the Z-buffer and can typically work its magic from the depth map.

The best implementation I'm hoping we can get is the dynamic one, which smoothly adjusts focus based on where the player is looking (distance of the center pixel in the middle of the screen). By default the effect definitely shouldn't be excessive, only blurring distant or near things very slightly unless you're looking at a surface right in front of you. I don't believe it needs to be enabled by default as long as it's in the menu, like the sharpness filter it shouldn't bother anyone who doesn't want it and make things look beautiful for those that do.

good idea. do it.

Link to comment
Share on other sites

It might be good to know if devs are willing to integrate a dof effect, if it works, in the engine. Otherwise you have to fork tdm to implement it. Most current posting devs seem uninterested, just saying.

Link to comment
Share on other sites

23 hours ago, OrbWeaver said:

I don't see how DoF can possibly work in a game which isn't tracking your eyes. How does the game know what you are looking at? Your eyes could be focussed anywhere on screen, which might show objects at a variety of distances. If the depth is taken from the central aim point, that means that instead of just moving your eyes to look at something on screen, you'd also need to move the mouse to remove the artificial blur.

DoF looks nice in photos, and as stgatilov says it might work in a VR simulation with eye tracking, but for a 2D image on a standard screen I can't see it adding any value.

Most modern games only use it in cutscene-like scenes, where the players eye is clearly guided towards relevant image content. Doom Eternal uses a very shallow DOF when the Doom Slayer picks up a new weapon and shows it off a little or when performing a glory kill. Elder Scrolls Online uses DOF when talking to NPCs. While the DOF-implementation of ESO is not the best, both examples work really well.

In TDM, there are not really many suitable situations where this rendering technique could be deployed, so I don't think it'd be worth the effort.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

 Share

  • Recent Status Updates

    • peter_spy

      It's funny how perception changes over the years. I've been replaying Bioshock in its Remastered version, and I'm surprised how bad the level design actually is. It's just a series of abstract corridor mazes that you can't really map out in your head. There's nothing that would ground them in any kind of reality, fictional or otherwise, no sense of place people could live in. It's almost like Wolfenstein 3d with cool art deco assets.
      · 1 reply
    • STiFU

      Anyone here played Inscryption? At first, I was like "no, I am not going to play a stupid card game", but this game is so much more than that. It is so meta, full of 4th wall breaks, and feels so damn spooky and weird because of it. Absolutely recommended, if you can stand the visuals. I recommend not reading up too much on it, as that could easily spoil half the incredible surprises. Just know that what you see in the beginning is by far not everything.
      · 4 replies
    • freyk

      Tried to make a tdm advertisement commentpost at one of civvie11 youtube videos about T2. Post got marked as spam. His problem (to not discover TDM for himself),..not my problem.
      But some help of some fellow TDM yt-videocomment posters would be nice. To ask him and others, to play TDM. To get more players/creators. 
      · 5 replies
    • datiswous

      Currently Profile Information has 3 fields, these are shown in forum posts under your avatar:
      1. Gender
      2. Location
      3. Interests
      I think that it could be useful to have an extra field called "Operating system" (under location). It can be useful for tech support and to see what people use.
      Alternatively it could be a more general term, like PC system, so that you can for example state that you use an AMD gpu.
      · 2 replies
    • OrbWeaver

      Greetings fellow kids.
      · 11 replies
×
×
  • Create New...