TDM Devbuild AA discussion

[MirceaKitsune]

Hope I'm not being too off-topic for this thread, but I was playing a lovely new FM and something I saw reminded me of another reason why I hate MSAA apart from bad performance: It doesn't even always do the job right, including (but not limited to) to fixing pixel-sharp patterns on textures.

Zoomed in for a close look and to catch a good example, see at full res. Seems to be the bumpmap in this case. Possibly caused by the texture being so high-res it surpasses the screen resolution so there's no blurring any more?

I set MSAA to 4x and it didn't affect it. Which makes sense since I'm sure it only handles geometry and has no concept of textures. It would further be great if we had an alternative form of AA, which weren't just cheaper than MSAA but also handled sharp pixels on textures including alpha edges alongside geometry edges: Right now I think we have no fix to that grain.

[AluminumHaste]

AA is for edges of geometry, not texture sampling. It's possible someone forgot to include mipmaps for that texture

[HMart]

Like AluminumHaste said MSAA is for edge of geometry/polygons, if you want to affect textures/shaders/transparencies, you need to use a post-process AA like TAA (it will blur the scene a little contrary to old school MSAA thou).

You can use Reshade for TAA. 

Ops: MirceaKitsune sorry you already knew msaa was for geometry only, didn't read your reply well sorry...

Edited June 28, 2023 by HMart

[MirceaKitsune]

32 minutes ago, HMart said:

Ops: MirceaKitsune sorry you already knew msaa was for geometry only, didn't read your reply well sorry...

It's okay. Just yeah made me wonder if any of the alternate AA types could also handle sharpness in textures, so we can taclke geometry and texture aliasing in one go seeing we get both. Given the reasons mentioned to not use FXAA and other mainstream ones, I'm wondering if we may want to create our own AA post-processing shader for TDM's specific needs, given we need both something attuned for performance and various sharpness scenarios.

AFAIK the Linux amdgpu driver offers no way to enable those universally: I believe some users said they can enable FXAA in their driver settings, but that's probably AMD Catalyst or Nvidia on Windows. Never tried reshade but heard about it.

[lowenz]

ReShade has NOT access to frame queue and motion vectors, so no "true" TAA possible (but there are motion vectors estimation techniques: GitHub - JakobPCoder/ReshadeMotionEstimation: Optical Flow for games in realtime )

Only FXAA/SMAA/CMAA (Intel), etc.

10 years ago they were called "morphological AA"

Edited June 28, 2023 by lowenz

[HMart]

On 6/28/2023 at 10:43 PM, lowenz said:

ReShade has NOT access to frame queue and motion vectors, so no "true" TAA possible (but there are motion vectors estimation techniques: GitHub - JakobPCoder/ReshadeMotionEstimation: Optical Flow for games in realtime )

Only FXAA/SMAA/CMAA (Intel), etc.

10 years ago they were called "morphological AA"

oh I didn't knew that, I just seem to recall Reshade having a TAA like shader but I most be wrong then.

[AluminumHaste]

On 6/28/2023 at 11:24 AM, MirceaKitsune said:

Hope I'm not being too off-topic for this thread, but I was playing a lovely new FM and something I saw reminded me of another reason why I hate MSAA apart from bad performance: It doesn't even always do the job right, including (but not limited to) to fixing pixel-sharp patterns on textures.

Zoomed in for a close look and to catch a good example, see at full res. Seems to be the bumpmap in this case. Possibly caused by the texture being so high-res it surpasses the screen resolution so there's no blurring any more?

I set MSAA to 4x and it didn't affect it. Which makes sense since I'm sure it only handles geometry and has no concept of textures. It would further be great if we had an alternative form of AA, which weren't just cheaper than MSAA but also handled sharp pixels on textures including alpha edges alongside geometry edges: Right now I think we have no fix to that grain.

What mission is this? I'll take a look at the texture in question.

[MirceaKitsune]

25 minutes ago, AluminumHaste said:

What mission is this? I'll take a look at the texture in question.

Spoiler

Seeking Lady Leicester. It was in the spirit world, I believe on the little floating island with a stone and some greenery on it.

 

[datiswous]

On 6/28/2023 at 9:05 PM, MirceaKitsune said:

I believe some users said they can enable FXAA in their driver settings, but that's probably AMD Catalyst or Nvidia on Windows.

You can set FXAA in the nvidia linux driver settings. Then you have to disable AA in tdm settings. Since it's an nvidia tech, I understand you cannot set it in the amd driver settings.

[lowenz]

It's not NVidia only, AMD just has its own spin (Morphological AA) since 2011.

FXAA was created for consoles starting from Foliage AA techniques seen in Crysis in 2007 (edgeAA) as a performance oriented form of AA (old consoles can't handle MSAA)

Edited June 30, 2023 by lowenz

[AluminumHaste]

11 hours ago, MirceaKitsune said:

  Reveal hidden contents

Seeking Lady Leicester. It was in the spirit world, I believe on the little floating island with a stone and some greenery on it.

 

Can you go into the game and type 'getviewpos' into the console and paste the output here?

[AluminumHaste]

Where did the rest of the replies go?

EDIT: It's a TGA normalmap, no mipmaps, so you might get shimmering or other texture artifacts when you move away from it, or if it's used on a really small object like in this case.

[HMart]

On 6/30/2023 at 10:28 AM, datiswous said:

You can set FXAA in the nvidia linux driver settings. Then you have to disable AA in tdm settings. Since it's an nvidia tech, I understand you cannot set it in the amd driver settings.

to add to what lowenz said MLAA from AMD is generally considered better than FXAA on AMD hardware, for obvious reasons, so if you have one, is better to use that instead of FXAA. But unfortunately many games only implement FXAA, and forcing it on the GPU driver smooths everything on screen, including the text and GUI elements, that is not generally what you want. That happens the same with FXAA forced on the nvidia drivers...

But today IMO is better to forget FXAA and MLAA and support/use SMAA (Enhanced subpixel morphological antialiasing) that is a updated version of MLAA.

Edited July 2, 2023 by HMart

[lowenz]

MLAA and SMAA have one big problem FXAA doesn't have: they morph straight lines.....into something like ~

Edited July 2, 2023 by lowenz

[MirceaKitsune]

So some clarification: This thread branched off from my post in the general devbuild thread, some of my replies didn't get moved with it... sorry again for not noticing the move in time! Here's my reply with the viewpos for that texture.

I gave my take on why I think we need shader based anti-aliasing in TDM: MSAA is still very expensive despite recent improvements, even at 4x you're going to lose at best 10 to 20 of your precious FPS. Making matters worse, it addresses only sharp edges caused by geometry, MSAA can't do anything about sharpness in alpha textures... in fact even geometry around sharp areas like lamps doesn't get smoothed out properly.

I proposed FXAA, TAA, SSAA, etc. In another post Stgatilov made good points on why those have their own issues. This left me wondering if we may want our own custom AA shader, fit for TDM and idTech 4's specific needs... the devs best know the answer to that question. I can only say that as a player, I definitely feel we need a cheaper and all-encompassing solution to pixel-sharp edges produced under various circumstances, don't know enough to do more myself.

Edited July 3, 2023 by MirceaKitsune

[cabalistic]

I don't think anyone's going to just randomly come up with a better alternative to SMAA. That would take serious knowledge of this particular domain and probably a lot of research and experimentation.

TDM is not currently in a state where we can implement TAA well, because good modern variants (including FSR 2) require motion vectors, and we cannot currently create proper motion vectors. Even then, it is by far the most complicated technique to implement well.

So, realistically, that leaves FXAA or SMAA. Personally I think FXAA is terrible and not much more than a glorified blur filter and would be a complete waste of effort. SMAA is at least ok in some sense, though for me absolutely not a replacement for MSAA.

[Wellingtoncrab]

@MirceaKitsune 

I don't say this to be discouraging, but a developer has already responded that replacing the AA in the game is not arbitrary and has no ETA so not sure what else you are looking for? 

Even if the engine is overhauled to the point TAA is an option, there is still no perfect AA solution. The image quality of the most modern games often reveals shortcomings under scrutiny - there are always trade offs.

So how helpful is your suggestion which paraphrased is basically: "we need a bespoke AA solution that cheaply provides perfect results" when it comes to actually solving this problem?

In a game where the image is typically dark and there isn't much high frequency detail it does not take much AA to provide an acceptable image in 95% of circumstances imo.

It is we mappers who ultimately control what you are seeing in a given mission, and these scenes in SLL are very brightly lit with large volumetric light volumes. This for example reveals a lot of limitations in the resolution of shadow maps for these lights and presents normal map interactions that are not typical for the game, all that were quite obvious in development. Ultimately we liked the overall effect, so we accepted the trade offs.

While there will always be room for improvement and advancements (just look at the VL light quality improvements coming from 2.10 to 2.11) it is that overall presentation, including those compromises, that is what real time game art is all about.

[MirceaKitsune]

I think the new thread started from me posting about a sharp texture, I didn't understand what was causing it to be pixel-grainy, it was later clarified that lack of mipmaps can do that. I suspected it might be a common occurrence and combined that with the same happening to sharp edges. Hence me wondering if a solution to address all such artifacts can exist.

Personally I think FXAA would be an improvement, but if the other devs feel it would look ugly I understand that. I don't agree the issue isn't noticeable due to darkness, I tend to notice it a lot and look for ways around it. I understand an easy solution isn't possible now, like I said the thread branched as I posted about a sharp texture error which I misunderstood as being related to (texture) AA in some form.

[Wellingtoncrab]

As other users have pointed out it's at least as much if not more an art "problem" as it is a technology one. While I could definitely see mipmaps helping, as I have mentioned I am not aware of any normal maps in core assets which are compressed and leverage mipmaps. I think it comes down to the overall intensity and then the frequency of detail in the normal map relative to the scale of the texture and lighting in the scene.

The art of the game was made to suit the technology of the game. This means the intensity of the normal maps in many stock assets are tuned to present detail from small often transient dynamic lights and they do not hold up as well under these kinds of lighting conditions or when used outside of their intended texture scale. This one is probably intended to be a large stone floor which is probably lit by a torch instead of small platforms lit by giant spotlights.

This was within our power to "fix", but we didn't consider it a priority admittedly in the context of all the work to be done to finish and release the mission.

[HMart]

On 7/2/2023 at 9:08 PM, lowenz said:

MLAA and SMAA have one big problem FXAA doesn't have: they morph straight lines.....into something like ~

MLAA has a bad tendency to smooth text indeed but imo SMAA is better than that, specially at x2 or above.

This is what Unity docs say about it.

Quote

is higher quality than FXAA, but slower. It doesn't require motion vectors either, but is not compatible with mobile devices or XR. Depending on the visual style of your project, SMAA can match the quality of TAA.

Is all a matter of opinion but if is good for Unity is good for TDM?  

[MirceaKitsune]

There's at least one thing I felt I need to ask about: I keep seeing this and it's making me think even conventional AA must be broken sometimes. In any case it's clearly not doing its job right at the moment.

I enabled 4x MSAA just to alternate. As can be seen on most objects, it usually works as you'd expect. But then why do I get such sharp edges only on the newspaper? It's not an alpha texture, the model is fully opaque and this is its geometry edge... it should this be subject to AA as anything else, yet it doesn't seem to be affected and still looks jagged.

I see this effect on bright lamps sometimes. I didn't report it as I presumed it's a high-res alpha channel which I know current AA can't address. But this clearly isn't the case so now I'm curious what's happening?

Edit: The forum insists on scaling the image to a lower resolution so here is the full res.

Edited September 19, 2023 by MirceaKitsune

[AluminumHaste]

Turn off bloom?

[MirceaKitsune]

22 minutes ago, AluminumHaste said:

Turn off bloom?

Is bloom causing it? Didn't think to try that: Just started the FM so next time I pass by the area I'll try that as well. Also the sharpness filter even if it's at a low setting, I'll feel silly if that was literally undoing the effects of AA

I was thinking the newspaper being so bright compared to the background might have something to do with it. Though IIRC AntiAliasing scans geometry edges only and shouldn't care about brightness per say.

[nbohr1more]

Could be the sharpness shader. Try turning it off or lowering the value. It uses contrast to detect edges so that bright paper vs dark background may be throwing it off a little.

[MirceaKitsune]

I turned off both bloom and sharpness filter, yet AA continues to generate fully sharp edges around that newspaper. Now it's really peculiar... wonder if anyone else can reproduce this? FM Iris, getviewpos:

4251.78 123.7 -271.75   9.2 -30.5 0.0