Grain effect post-process

[7upMan]

Hi folks, a while ago someone (I think it was one of our Russian friends, but I couldn't find the original post) said that he thinks TDM FMs look too "clean" and tidy. When playing Knighton Manor by Sotha (great FM, by the way) it struck me that maybe this is exactly what the poster meant: Contrary to T1/T2 TDM has cleaner textures and less problems with color gradiation because of dithering. I imagine someone used to that "smudgy" look might miss that in the overly clean TDM FMs.

 

So how could that dirty look be achieved (if the player so wishes)? Well, I remembered that Left4Dead uses a Film Grain post-processing effect that smudges up the picture quite a bit. Here is an example, please do have a look at the pictures bevore and after that.

 

What do you think, would the atmosphere in TDM benefit from such a filter? Would it be very hard to implement it and have it as an option in the Graphics menu? What are your thoughts on this?

 

 

 

 

 

 

 

 

EDIT: Direct link to the picture mentioned above.

Edited December 8, 2010 by 7upMan

[Aprilsister]

So how could that dirty look be achieved (if the player so wishes)?

 

Said old-schooler could also buy some drugstore glasses that will fuck up their vision; grub-up their screen with some sloppy-yet-arty smear of filth; set their resolution to 160x120... many wonderful solutions to such wonderful needs...

Edited December 8, 2010 by Aprilsister

[nbohr1more]

The Sikkpin mod already has a Post Process film grain filter so it would technically be possible... but it would have to be weighed against other factors (primarily performance).

 

I still think it would be cool if a Photoshop style "impressionist" filter could be applied in real-time but I think that is still too heavy for current GPU's ?

[STiFU]

I believe the "too clean" comments are based on the fact that Doom 3 looks pretty synthetic sometimes due to hardedged shadows and missing lightmaps. A Grain filter could possibly dirty it up a little, but it would have to be very, very subtle and not too dynamic, because TDM is a slow paced game and a shivery aggressive noise-overlay wouldn't concur with that.

 

However, Sikkpin also has a way better solution to this problem implemented in his Mod, namely Screen-Space Ambient Occlusion!

[Bikerdude]

@SSA, would that even be possible with the D3 engine and at what performance cost..?

Edited December 9, 2010 by Bikerdude

[nbohr1more]

It's possible but the performance cost is VERY heavy. Sikkpin has also demonstrated that this can be toggled off and on as a stage in his mod.

 

I believe that it is STiFU's wish that SSAO would be incorporated as an "Ultra High" graphic mode but I get the impression that the TDM team doesn't want the bad PR of having yet another reason for dumb end-users to claim that the Mod runs "too slow".

 

From stuff in the Bugtracker and the Wiki I also conjecture that the v1.03 Ambient lighting has been enhanced in ways that may reduce the need for SSAO (unless those changes were rolled-back etc). The answer to that question should arrive shortly methinks?

Edited December 9, 2010 by nbohr1more

[Mortem Desino]

Unless for graininess you'd prefer the use of detail textures?

 

I always thought that film grain felt so cammed and fake. I haven't found a game that really pulled it off well.

[nbohr1more]

I must try this out tomorrow!!!!!!!!!!!!!!!!!!!!!!!!

 

Wow!

[Tels]

I believe the "too clean" comments

 

I think they are more due to the fact that we provide the "clean" version of all textures (like a newly build brickwall) and the mapper needs to add dirt, cracks, etc etc. and this is very timeconsuming, and so often skipped. It would be better if we had a lot of run-down, old textures, so you don't have to dirty up every wall extra.

 

The grain-filter is a nice idea, tho.

 

It could technically work like an underwater-overlay, but as Mortem says, "detail textures" are much better than just a full-screen effect. Unfortunately, we don't have detail textures yet...

[Sotha]

If the grain-texture-thingy is very performance expensive, other solutions would be better.

 

I think they are more due to the fact that we provide the "clean" version of all textures (like a newly build brickwall) and the mapper needs to add dirt, cracks, etc etc. and this is very timeconsuming, and so often skipped. It would be better if we had a lot of run-down, old textures, so you don't have to dirty up every wall extra.

 

First of all, I agree that maps do look too clean. Tels comment is the main thing. I like mapping, but I hate boring stupid tasks like putting grime decals on every...fricking...single.... surface. I once tried to put those, but it is a fact that if you put grime on one surface, you'll have to bother to put grime on every single surface, because otherwise the non-grime surface shines out. Imagine short flights of steps. That is a helluva lot of surfaces to grime.

 

That is why I do not bother to do grime at all. I never will, unless a really easy system is generated.

 

So... The solution might be to dirty up our textures. boards_rough -texture would get is alternate cousin: boards_rough_dirty. I agree that this would be a lot of work and increase mod size a lot. Maybe not feasible? I we had more dirty textures available, I certainly would use them.

 

For the already existing clean textures: I wish DR had some neat grime function, which could automate decal setting a lot. I know there is the create decal patch -function but that requires a lot of manual work per surface: select surface ->create decal patch -> move patch to avoid z-fighting -> apply texture -> align texture.

 

If I could simply select all surfaces in a scene, click the surface decal button and automagically non-z-fighting decals were created (0.5 units above the underlying surface). After the button press, all the new patches would be selected and I could choose a continuous tiling grime texture, and set it to 0.5 natural alignment.

 

Basic idea: if DR would allow a few click grime-up-scene procedure, that would be great.

 

Oh and one question: does a grime decal, which penetrates visportals cause trouble?

[ungoliant]

I think they are more due to the fact that we provide the "clean" version of all textures (like a newly build brickwall) and the mapper needs to add dirt, cracks, etc etc. and this is very timeconsuming, and so often skipped. It would be better if we had a lot of run-down, old textures, so you don't have to dirty up every wall extra.

 

oh my god you are preaching to the choir. i've spent hours randomizing dirt decal distributions on every single exterior surface of my map. And i'm nowhere near done. All to make things look "dirty". The end effect is totally worth it, but it sure would be nice if i didn't have to do all the legwork.

 

Sotha: if you want decals that do not z-fight on creation, edit the decal texture you are using and add keyword "DECAL_MACRO" to it. Be careful with this though. anything that sticks out in front of this surface (a painting, a banner, trimwork, etc) the decal will render on top of when the player moves a certain distance away, and can create very unsightly "pop-up" effects. Its a pain in the ass.

 

edit: the decals will still z-fight in DR, which I hate looking at, but its no worse than dealing with shadow texture surface clones. It will work properly in game though.

Edited December 9, 2010 by ungoliant

[Sotha]

oh my god you are preaching to the choir. i've spent hours randomizing dirt decal distributions on every single exterior surface of my map. And i'm nowhere near done. All to make things look "dirty". The end effect is totally worth it, but it sure would be nice if i didn't have to do all the legwork.

 

-Hm.. So others feel it is awkward too, but are just diligent enough to pull grime-decalling through.

 

Sotha: if you want decals that do not z-fight on creation, edit the decal texture you are using and add keyword "DECAL_MACRO" to it. Be careful with this though. anything that sticks out in front of this surface (a painting, a banner, trimwork, etc) the decal will render on top of when the player moves a certain distance away, and can create very unsightly "pop-up" effects.

 

Mappers should not need to edit the textures. I would then need to distribute custom material file with the map.

 

Its a pain in the ass.

Agreed. Maybe a DR improvement could help and make this easier for the mappers? What if the surface decal button worked as I suggested? Maybe there could be an option, which lets the mapper to choose how high above the surface the decal floats? This could make it easy to make several decaling pass: basic grime at 0.25 units above. Then the crack layer on 0.50 units above the underlying brush. Z-fighting free grime-decalling should be easier.

[Fidcal]

I agree adding grime is a tedious boring process and it's hopeless trying to apply it everywhere in a big map. I'm trying to be selective but it's hard.

 

AFAIK I've never noticed any problem overlapping a visportal nor should there be. My latest effort up a long wall pass through at least one and I think two visportals with no problem.

 

The main answer for Dark Mod will no doubt be a combination of improved textures plus selective use of improved grime decals plus lighting. Also not all FMs have to be eye candy. We need plenty of FMs that are built quickly and have good gameplay too.

[ungoliant]

material files are almost a negligible in size. Perhaps it is possible to have a small array of dirt/grime images that can be blended on top of existing textures to "dirty" them, and list them in their respective folders "dirty1" "dirty2" etc. This way, mappers don't need to customize much, and the end user sees a minimal increase in download size in official releases of both FM's and TDM client.

 

edit: if the dirt decal overlays for said idea could "tile" together with the original texture, that would be insanely awesome. I spend a lot of time with existing dirt flipping, rotating, scaling, etc, to fool the player that 2 or 3 decals thrown together on 300 surfaces look completely random

Edited December 9, 2010 by ungoliant

[Aprilsister]

This decal thing looks like it is something that bothers FM authors in a big way. Did this get mentioned in What Are The Two Most Annoying DarkRadiant Issues ?

 

I guess it doesn't exactly fit the question but it does in a round about way. And, though I was happy to see greebo shift from DR to TDM (and even more so to tdmupdate) (at least I thought this was what occurred recently), it sounds like if this idea of a please-splatter-this-area-with-some-random-grime button could be programmed, it would be a major hurrah.

 

I still and will always maintain that the example above where the whole scene is mucked-up is a staggering step backward, but, sure, there are surfaces, maybe the majority, that might need griming (hey! that word "griming" is a word! I didn't expect that.). It will never matter too much to me as compared to the apparent pains that FM authors are going to, but, if they are convinced that they need it and aren't just doing it for the pleasure of being artists; if it is pure drudgery; well then I hope that this can be accomplished.

 

Otherwise, as a fan of the fan missions (fan made? fan played? me? obviously playing on the "played"), I promise you that the griming (loving that word) for hours is going under-appreciated. I'd so much rather have the mission now.

[Melan]

I think they are more due to the fact that we provide the "clean" version of all textures (like a newly build brickwall) and the mapper needs to add dirt, cracks, etc etc. and this is very timeconsuming, and so often skipped. It would be better if we had a lot of run-down, old textures, so you don't have to dirty up every wall extra.

We are working on that with Serpentine (or rather, Serpentine is working on it - I am working on a few new textures now and then that will be submitted later).

 

In general, textures are very important since textured architecture is maybe 90% of the game world you see. Many of TDM's textures are indeed too clean (although dirt decals help), but more than that, they often tend to be too light. It may have been thought that lighting would make much of the difference, but that's not entirely correct - a scene built with a dominance of darker textures looks different from a scene built with a dominance of brighter ones. Saturation also matters: saturated colours, when used well create stronger, more dramatic contrasts.

 

This, along with a different use of shadows and light, is in fact the major visual difference between Thief 1 (slightly grainier, but darker and more contrasted world) and Thief 2 (sharper but lighter and more "even" world). An example comparison from OMs:

Disregarding technical issues (e.g. that Thief 1 used a 256-colour pallette), the end results are still very divergent - in spite of coming from the same team and using almost the same engine.

Edited December 9, 2010 by Melan

[STiFU]

Phew much to comment on here.

 

It's possible but the performance cost is VERY heavy. Sikkpin has also demonstrated that this can be toggled off and on as a stage in his mod.

 

I believe that it is STiFU's wish that SSAO would be incorporated as an "Ultra High" graphic mode but I get the impression that the TDM team doesn't want the bad PR of having yet another reason for dumb end-users to claim that the Mod runs "too slow".

There has been no internal discussion regarding the possible addition of SSAO until now. Sure, one could argue that one small partition of users would moan about TDM being too slow in "Ultra High" mode, but at least nobody could moan about TDM being too clean anymore, because the only thing to do in order to dirty it up would be to buy a 50$ gfx card. I run one of Sikkpin's first SSAO implementations smoothly on my crappy geforce 8800 GTS, one of the first DX10 cards (hack, I was even able to run it smoothly with his softshadows, DOF and all that other fancy stuff enabled). Moreover, Sikkpin offers 7 SSAO methods by now I believe, all stuffed with optimizations and coolness. The biggest advantage of SSAO is that its performance is completely independent of the complexity of the scene-structure, because it is a post-effect. The performance hit is solely dependent of the resolution and the SSAO settings. Cheaper settings will lead to better performance and you'll still have a big improvement over the synthetic Doom 3 look. Decals on the contrary raise the complexity of a scene (most surfaces are doubled), which can have worse results performance-wise than SSAO in certain scenarios.

 

From stuff in the Bugtracker and the Wiki I also conjecture that the v1.03 Ambient lighting has been enhanced in ways that may reduce the need for SSAO (unless those changes were rolled-back etc). The answer to that question should arrive shortly methinks?

The answer arrives now. JC Denton has created a new ambient lighting method, that's correct, but it has in no way any relation at all to the need of SSAO, since it is still just ambient lighting. Sorry!

 

First of all, I agree that maps do look too clean. Tels comment is the main thing. I like mapping, but I hate boring stupid tasks like putting grime decals on every...fricking...single.... surface.

...

If I could simply select all surfaces in a scene, click the surface decal button and automagically non-z-fighting decals were created (0.5 units above the underlying surface). After the button press, all the new patches would be selected and I could choose a continuous tiling grime texture, and set it to 0.5 natural alignment.

 

Basic idea: if DR would allow a few click grime-up-scene procedure, that would be great.

Exactly my Point. You currently have to put grime everywhere and you do it mostly in order to emulate Ambient Occlusion. I understand the pain the mappers are going through and I thought a "real-time grime computation" aka Screen-Space Ambient Occlusion would be the most elegant solution and it is also the best looking solution. However, I have also been thinking about writing a DR plugin for griming up surfaces. Until now, I couldn't think of a good concept, that completely gets rid of manual mapper-power. Well, there is a concept, but I am not sure how to implement it. Also, I am stuck in my diploma thesis, so writing a big plugin is probably out-of-scope for me in the next half year, but it's good to at least get a concept down, if we don't merge SSAO eventually into TDM.

 

My concept currently looks like this: You select surfaces you want grime on and click on "GrimeIt!". The plugin detects which surfaces are connected and treats them as a single surfaces. In a small window popup you can then decide which sides and edges are supposed to be grimed. A Patch is generated, proper material is applied and the patch if offset on z.

 

Regarding the dirty-look of the classic thief games: The textures themselves weren't that dirty actually. In my opinion the look originates from the dithering, which creates a similar effect to grain, and from the lightmaps, hence my suggestions for SSAO.

[rebb]

The biggest advantage of SSAO is that its performance is completely independent of the complexity of the scene-structure, because it is a post-effect.

 

If the game gives access to a Z-Buffer Texture that's true, but Doom3 doesn't unfortunately.

The scene has to be rendered once more per frame for a Z-Depth Capture Step - so it is ( currently ) somewhat dependent on scene-complexity.

 

But as you said it should still run fine on modern systems.

[STiFU]

Thanks for clarification Rebb. I honestly just thought it was independent of scene-structure because my experiments with it suggested it. I never saw any real fps-variations when walking through a level.

 

But you guys see? The master of ambient occlusion himself says it runs fine on modern systems. (I don't consider my system modern anymore though)

[7upMan]

SSAO: It's possible but the performance cost is VERY heavy. Sikkpin has also demonstrated that this can be toggled off and on as a stage in his mod.

 

Well, while experimenting with Sikkpin's SSAO implementation I noticed that using it creates more problems than it solves. I will post a few screenshots soon to illustrate what I mean. OTOH, it could very well be that a different SSAO setting wouldn't have caused any problems. I think I'll need to experiment a little more. Also, I'm using an ATI Radeon HD5770 with Cat. 10.9 an AI disabled. Maybe the implementation works better on GeForce cards.

 

 

Regarding the Grain filter, could it be done the same way as the Bloom filter? After all, while not officially supported, it's still available as an option. I agree with STiFU though that the effect should be subtle.

[_Atti_]

I think the thing is that, Thief1,2 had simple geometry, lowres textures, the mind of the player puts in the detail, the grime.. there isnt any really..we just think there is.

 

Getting this illusion in TDM is a lot of hard work, because with highlevel geometry and big resolutions come with more detail the mapper must show the player for his imagination to start working and fill it all with more detail/grime, and what starts the player's imagination is immersion.

 

I think the same effect is in works here why high-res texture packs for old games suck, and ruin the whole thing.

[nbohr1more]

@STiFU:

 

Since you are being more forthright about v1.03 features, is JC Denton's new Ambient method at all related to the "Dynamic Ambient Light" reference in the "Location Settings" wiki article:

 

http://wiki.thedarkmod.com/index.php?title=Location_Settings#Ambient_light_settings

 

...And if so, could you not simply setup darker "zones" in the recessed areas and thus fake AO in that way?

 

[STiFU]

The dynamic ambient light has been in the mod a lot longer than the new ambient method. I believe that Tels implemented it and that JC Denton wasn't even on board back then, but I am not sure. Sure, you could have darker areas in a level, but it still won't be fake AO, as the whole levelgeometry will get uniformly darker/colored and not just concave edges like it would be with AO.

[nbohr1more]

Create tiny "zones" around the concave edges, or is that not possible?

[STiFU]

See the ambient light as a post effect sort of, that tints the whole screen in the desired color and brightness when the player is in a zone. What you want to achieve can only be done via pointlights and falloff I think.