hdr Tone mapping

[peter_spy]

Why? When I make assets I always use default values, and I think it's pretty much the only way to ensure everything looks the same on all players' computers. Obviously they can set it up as they like, but they have an easy reference point, in case they want to return to intended look.

Besides, in order to get consistent results, you have to at least try to assume a model, something that will return somewhat logical results. Since this is non-PBR, it's hard to do; you don't even have any consistency in how models, materials or other declarations were made, nor how mappers use lights in maps.

[Arcturus]

It looks bad with or without the bloom, as you can see in my last video. With bloom set to half it looks only slightly less bad than with bloom set to max — but still bad.

Yes, there's no consistency to anything in darkmod because it was made by a bunch of amateurs. What's your point exactly? That nothing should be changed until the game is completely refactored to meet some nebulous standards of physical and mathematical correctness? Because I was told that is never going to happen (and I agree). So at the very least we can make the game look slightly less bad for now.

[Arcturus]

On 11/24/2024 at 12:11 PM, stgatilov said:

Here is some simplistic approach.

It multiplies the whole image by alpha = 0.7, making is darker.
But if the original color is brighter than 1 in any component, then the max color component is transformed as:

• x -> 1 - A / (B+x)

The proportion between RGB components is saved.

You can try to tweak alpha in shader, but it is a compromise between darkening normal images (e.g. color < 1 ones) and making overly high colors more distinguishable (color > 1 case).

Note that you can run reloadGLSLPrograms in console after editing shader, no need to restart game.

tonemap.frag.glsl 6.5 kB · 1 download

Out of the box it makes the image darker, so I increased brightness in settings:

Spoiler

32 bit, no bloom:

64 bit, no bloom:

64 bit, medium bloom:

64 bit, high bloom:

32 bit, no bloom:

64 bit, no bloom:

 

Original .glsl for comparison:

Spoiler

original 32 bit, no bloom:

original, 64 bit, no bloom:

original, 64 bit, medium bloom:

original, 64 bit, high bloom:

original 32 bit, no bloom:

original, 64 bit, no bloom:

AgX:

Spoiler

AgX:

32 bit, no bloom:

64 bit, no bloom:

64 bit, medium bloom:

64 bit, high bloom:

32 bit, no bloom:

64 bit, no bloom:

 

[MirceaKitsune]

I think it looks really good especially with bloom. Only thing that may affect both realism and gameplay is indeed the decreased brightness. You can turn up the brightness to compensate, but this has the unfortunate side effect of also making readables / main menu / every GUI exponentially brighter; I have r_postprocess_brightness 1.5 which is still a bit too dark in-world but makes every menu appear hideously bright... gamma doesn't do this but makes everything more gray and ugly instead so I don't increase it.

[stgatilov]

On 11/23/2024 at 2:16 PM, Ralle321 said:

If you are interested in the AgX display view transform, i have cobbled up the full round trip from various sources a while ago, the main one being https://github.com/bWFuanVzYWth/AgX (uses MIT license).

There is one thing about this shader that does not make sense.

It takes the framebuffer from the engine and passes it directly into agx_tonemapping function.
The function expects input in linear space (according to the comment), but our framebuffer is more like gamma/srgb space, since we usually send it into monitor almost "as is".

Then takes the output of tonemapping and does srgb-to-linear conversion to it.
The comment says the output is in linear space, but I think it is wrong. After looking here I think it is actually in srgb/gamma space. Yes, the referenced implementation uses different math for the curve, but I checked in Wolfram alpha that it produces the same curve. Also the referenced implementation performs srgb-to-linear conversion because the author configures output framebuffer as sRGB and thus back conversion is done automatically after the shader.

UPDATE: And one specific issue I see with the shader is that it clamps dark spots to full black.

I tried to I fix the gamma/linear confusion (i.e. convert srgb-to-linear before, not after). Then the clamping happens because of this:

    const float min_ev = -12.47393;
    const float max_ev = 4.026069;

    // Log2 space encoding
    val = clamp((log2(val) - min_ev) / (max_ev - min_ev), vec3(0.0), vec3(1.0));

This AgX shader only supports range from 1/5600 to 16 without clamping.
But since input should be in linear space, this becomes only 1/48 .. 3.5 range in gamma space. Darkest colors in 8-bit gamma space are 1/255, i.e. much darker.

[MirceaKitsune]

Yes. I'm not familiar with how the TDM engine handles color internally, but when it comes to color spaces like Filmic or AgX I remember one important component to how they work: Color is meant to be in a range above the conventional 0 - 255, meaning bright spots surpass the 1.0 or 256 mark.

To my knowledge TDM never had the concept of over-brightening: To our engine the brightest spot is pure white in visible range, treated either as 1 in a 0 - 1 range or 255 in a 0 - 255 range. It doesn't understand a pixel having a brightness of 5 when 1 is pure white on the screen, 1234 out of 255, etc.

That's likely why some colors appear washed out, it makes sense now that I think about it. I believe bloom is also supposed to use pixel brightness outside of the visible range for more accurate results.

I imagine this is possible to do by tweaking the color range internally, but requires changing the way all light sources brighten their surroundings. It should also necessitate a larger color buffer since we're then storing more than 256 colors per channel... which may further impact performance, I still disable the 64-bit color setting in the main menu and live with color dithering as using it reduces FPS considerably (even if not as much as MSAA).

[I.C.H.I.]

AgX looks so much nicer and softer to my eyes than the original. The original lighting always gives me eyestrain, especially with bloom enabled.

But I'm comparing only these screenshots, this needs even more comparisons, because judging by the carpet some textures lose contrast unfortunately.

Edited November 30, 2024 by I.C.H.I.

[Ralle321]

20 hours ago, stgatilov said:

The function expects input in linear space (according to the comment), but our framebuffer is more like gamma/srgb space, since we usually send it into monitor almost "as is".

Ah, in that case i wrongly assumed linear space as the input.

I updated the tonemap.frag.glsl accordingly, now it converts srgb space to linear before applying AgX and i fixed the gamma conversion afterwards.

It's probably not ideal going from srgb space to linear, but it seems to work well with 64bit enabled. 32bit though shows a significant loss in detail.

Yeah, looks more correct that way. Not as dull and the menu doesn't appear overly soft anymore.

Screenshots:

Spoiler

AgX:

https://s20.directupload.net/images/241201/wm46whvq.jpg

https://s20.directupload.net/images/241201/mial7rig.jpg

AgX Blender:

https://s20.directupload.net/images/241201/brzbu9e2.jpg

https://s20.directupload.net/images/241201/2ufb6ols.jpg

My look (+10% contrast & saturation):

https://s20.directupload.net/images/241201/de85szmp.jpg

https://s20.directupload.net/images/241201/5v5dotbj.jpg

Standard Darkmod 64bit:

https://s20.directupload.net/images/241201/hnx46rxy.jpg

https://s20.directupload.net/images/241201/bx8y4q98.jpg

 

tonemap.frag.glsl

[stgatilov]

37 minutes ago, Ralle321 said:

I updated the tonemap.frag.glsl accordingly, now it converts srgb space to linear before applying AgX and i fixed the gamma conversion afterwards.

And what should we do with clamping on the dark side?
Arguably, dark colors are more important for TDM than bright ones.

Ultimately, we only have 8-bit output space.
If you want to make more distinction in bright colors, you have to sacrifice something.
It seems this specific implementation of AgX sacrifices the dark side.

[Ralle321]

A loss of information is obviously bad and i have no solution for that.

I guess this implementation is a dead end then, but it was worth trying out. 

 

[MirceaKitsune]

3 hours ago, Ralle321 said:

A loss of information is obviously bad and i have no solution for that.

I guess this implementation is a dead end then, but it was worth trying out. 

I hope we don't have to give up especially when the shader seems so close to finished. I don't know how the color space works internally, but it will likely require expanding the size of the color buffer and how the brightness range is calculated by the engine. Like a lot of new things it will likely take more work but I don't see it as impossible... other things took and will require more effort, such as realtime reflections when we get those too eventually.

[stgatilov]

2 hours ago, MirceaKitsune said:

I hope we don't have to give up especially when the shader seems so close to finished. I don't know how the color space works internally, but it will likely require expanding the size of the color buffer and how the brightness range is calculated by the engine.

The rendering color buffer is already expanded.
It is our monitors which only have 8-bit output precision normally. Unless you have some HDR monitor, which can display more with proper program support.

In the reality I think there are several approaches for tonemapping:

1. Based on modelling something
2. Just arbitrary math or 0-1 monotonous curves which look good to you

AgX belongs to p.1, it expects proper physical quantities as input in order to work properly.
TDM does not belong to the world of p.1 / PBR, so we can just as well tweak some curves without any regard for physics.

[MirceaKitsune]

That's good. The monitor is fine: I was referring to storing color information outside the visible range. Conventionally renderers would output directly to a 0 - 255 range per channel, for things like Filmic or AgX it needs to work with the concept of interpreting higher color values which are then translated into that visible range.

[I.C.H.I.]

I always loved indoor lighting in Metro: Last Light, absolutely nothing better comes to my mind of all I games I've ever played. Can we reverse-engineer it somehow?

[MirceaKitsune]

8 hours ago, I.C.H.I. said:

I always loved indoor lighting in Metro: Last Light, absolutely nothing better comes to my mind of all I games I've ever played. Can we reverse-engineer it somehow?

That's indeed something a better color space will help with. In my opinion at least, the contrast being off by default is one of the things making TDM less realistic: Everything feels hazy right now, maximum darkness doesn't feel black enough while the way light universally falls off (light textures aside) doesn't always represent what the eye expects.

Our lack of automated bounce lighting is also an issue. I once tried to solve this by giving all normal lights an additional ambient light with noshadows (lamps, torches, candles, etc). Sadly apart from being a performance nightmare and even producing crashes, it caused lights to shine through walls so I dropped that approach, even if it looked amazing and increased realism quite a bit.

[stgatilov]

I have attached another attempt at custom tonemap.

The tonemapping/compression curve consists of initial and tail parts, each with its own function.

We don't want the compression to significantly change the current look of the images with normal brightness, so we try to achieve y(x) ≈ x for x < 1. That is why the initial part is linear * exponent. The exponential multiplier is added in order to bend the curve down at x ≈ 1, so that the tail part has more space (i.e. smaller starting value and derivative).

Also we want y(x) → 1 for x → ∞, so I selected Reinhard tonemap x / (x + 1) with some coefficients and tunable power for the tail function.

You can see the math, code and the curve in the Jupyter notebook.
You can enable/disable the compression curve, or tweak the curve in the attached shader, it should be compatible with the latest dev build.
 

When testing this, make sure the brightness is exactly 1.0, otherwise it will either clamp bright colors or leave the upper part of the output range unused. Instead, I have added "exposure" parameter, which is basically a multiplier before tonemap/range compression. If our engine was physically correct, it would coincide with exposure of the camera

Also we have existing instruments that you can use to adjust the final look similar to the agxLook function:

• r_postprocess_gamma = 1 / power
• r_postprocess_desaturation = 1 - sat

Since the color range compression (when applied independently to the channels) makes the image look a bit desaturated, you might want to set e.g. r_postprocess_desaturation = -0.2 to compensate. Or not

tonemap.frag.glsl

[Filizitas]

Everything looks so clean and amazing with the tonemapper lol Almost realistic!!!!

[stgatilov]

17 minutes ago, Filizitas said:

Everything looks so clean and amazing with the tonemapper lol Almost realistic!!!!

Since I know that you are a good programmer and you worked with modern Unreal Engine (which most likely has proper PBR implementation), I suppose this is trolling I'd definitely join you in this endeavor: almost realistic!

[Filizitas]

I am not trolling  i generally like this alot! Pleace put this into the game as feature!

And i am not just a good programmer, i am one of the best.

[Arcturus]

Works nicely. It doesn't desaturate primary colors (like AgX does) but one could argue that's an advanced feature.

14 hours ago, stgatilov said:

modern Unreal Engine (which most likely has proper PBR implementation)

Interestingly, Filmic tonemapper used in Unreal as I understand is based on ACES - standard established by American Film Academy:

https://dev.epicgames.com/documentation/en-us/unreal-engine/color-grading-and-the-filmic-tonemapper-in-unreal-engine

Troy Sobotka (a film industry veteran himself) was very critical of ACES. And I don't think it's just a crazy person's talk cause from what I've seen in discussions around web many agree with him. Just because something is le industry standard, doesn't necessarily mean it's absolutely best.

[stgatilov]

12 hours ago, Arcturus said:

Works nicely. It doesn't desaturate primary colors (like AgX does) but one could argue that's an advanced feature.

This most likely happens because of the matrix transformation.

Could you please share the testmap where you have very bright lights of several colors (especially RGB)?
I will try to add this feature.

 

[Arcturus]

@stgatilov I uploaded test/tonemap.map and test_tonemap.mtr

[stgatilov]

The new dev build has the new tonemap stuff.
All of it is optional yet, i.e. disabled by default.

The list of cvars:

r_postprocess_compress 1: Enable the custom-made curve which compresses overbright colors.

r_postprocess_exposure != 1: This is intensity multiplier for the rendered frame before tonemap.
I think the "brightness" slider in the main menu should be redirected to this if we enable compression by default.

r_postprocess_overbright_desaturation > 0: strength of desaturation of overbright colors.

r_tonemapOnlyGame3d 1: Don't use tonemapping shader outside game: main menu, briefing, etc.

So I suggest the following changes in behavior:

• r_postprocess_compress := 1
• r_postprocess_overbright_desaturation := 0.1
• r_tonemapOnlyGame3d := 1 --- will no longer mess with menus and exhibit color banding on cinematics
• r_postprocess_exposure: tie to brightness slider in the graphics menu
• r_postprocess_desaturation -0.2 (optional)

The last point about oversaturation is questionable.
Unfortunately, color compression inevitably reduces contrast and desaturates the image, at least it seems so to me. So it seems that making the colors a bit stronger in the end is a good idea, at least the suggested AgX integration does this with about the same strength.

[MirceaKitsune]

Sad that r_tonemapOnlyGame3d doesn't prevent tone mapping from over-brightening in-world readables, only the main menu is fixed. Otherwise everything looks absolutely wonderful with r_postprocess_compress, no need to tweak anything else after turning that on: It's subtle but everything looks perfect to me, especially when it comes to handling bright areas and avoiding unrealistic pure white.

Edited December 10, 2024 by MirceaKitsune

[stgatilov]

18 minutes ago, MirceaKitsune said:

Sad that r_tonemapOnlyGame3d doesn't prevent tone mapping from over-brightening in-world readables, only the main menu is fixed.

It would be great to fix this, but certainly not for 2.13.

That would require to tag GUI overlays which correspond to rendering (e.g. underwater overlay) and making sure they render first.