Jump to content
The Dark Mod Forums

Tutorial: Better textures for your materials (WIP, part 2a is up)


peter_spy

Recommended Posts

Part 2: Normal maps

Normal map is a vector map stored in RGB format, where each channel represents shading in one of the dimensions (X, Y, Z). That’s why you can’t treat normalmaps like any other images: using tools like brightness or sharpening can destroy information stored in RGB channels. There are several filters available for making normalmap. There is an old normalmap plugin for Gimp, but I wouldn’t recommend it (see below), you can also try out software like nDo, Crazybump, or nJob. Since nJob is free and pretty good for typical applications, I’m using it as my main tool.

To create good normalmaps from 2d images, you need to prepare a heightmap image that will be converted by the filter. Heightmap is a greyscale image, where dark pixels are lows and bright pixels are highs (grey color is kind of middle ground). Heightmap can be based on your diffuse image converted to greyscale, but typically you have to do some more work to get proper results. Applying normalmap filter to color images often will result with wrong look. Classic example is a red brick texture with white grout.

obraz.png


Most normalmap filters will interpret this as in the lower part of the image:

obraz.png


Sometimes a simple value invert will be enough to create correct heightmap. In most cases, your ability to create good normalmaps will depend on how you made your diffuse map. If your diffuse texture is composed from several layers, you will have greater degree of control over the final result.
For example, my tile floor was made with 5 layers:

obraz.png


This way I can decide which parts of the image I want to include in the heightmap, and I can regulate their intensity separately. If I had only the final image, the image processed by normalmap filter would result in this:

obraz.png

 

Normalmap filter took all pixels in the image and converted them into depth information. That includes things like stone pattern, dirt, and tile color. That’s not a good result. Here the depth information is mostly grout and some subtle edges taken from the stone pattern to indicate age.

obraz.png

 

Example workflow for creating normalmaps

  1. When you finish making your diffuse texture, save all the layers as another .xcf file. This will be your base for heightmap.
  2. Desaturate all the layers, and see if they are representing the height in correct way: dark pixels for cavity and bright pixels for convexity. Adjust layers’ modes, opacity, or color values, if you see that they don’t represent bumps or indents in the way they should.
  3. Use Copy visible (Ctrl+Shift+C) in Gimp to copy the result to clipboard.
  4. Paste the image into nJob and use Heightmap to normalmap filter (or press 4 on the keyboard).
  5. Adjust the filter settings. I use Traditional mode most of the time. There is no numeric scale here, but I usually reset everything first, and then I click on the scale, so the slider moves by the same interval. Now adjust Scale and Blur radius to your liking. If you don’t know what to go for, just click the Scale slide once. Copy (Ctrl+C) the result, paste it back to Gimp, and export it as your normalmap.
  6. Launch TDM and run your test material map to see whether the result is correct. If not, switch to nJob, Undo (Ctr+Z) the filter operation, and repeat steps in point 5 until you get desired/correct result.

If you don’t know what to look for, search online for photo, or video reference of a surface you’re making. This will give you a sense of direction and you’ll eliminate guesswork. That said, don’t be afraid to experiment. Using some crazy values might lead you to interesting effects that you’ll be able to use later!

 

 

Notes:

 

Diffuse + normal vs. diffuse + normal + specular

 

If you only have diffuse and normalmap textures in your material, the result will be different when you add a specular map. Your normalmap will look stronger with specular, and your material will have more depth. In general, there’s no reason why you wouldn’t want to create a specular map: all real-world surfaces are shiny and reflective to some extent. There are materials in stock TDM that use only _d and _n textures, but the other way around will be more beneficial to your materials. Specular complements diffuse so closely, that it should be included every time, while normalmap doesn’t always look right. Sometimes the surface is very smooth, and details from the diffuse texture may look ugly or blocky when transferred to a normalmap. If that’s the case, it will be better to make a material without a normalmap, rather than without a specular.

 

As I mentioned earlier, it good to prototype 3 maps as fast as possible, and work on them simultaneously – they all influence each other. If you don’t know what to look for in a specular yet, and you want to prototype your normal first, you can use either _grey system texture in the specular slot.

 

obraz.png

Floor material with and without specular.

 

 

Normalizing normalmap images

 

One of most important steps in making normalmaps is normalizing your image. This is something similar to setting proper color range for your diffuse, so the game engine will interpret the image correctly. nJob does it pretty well automatically. If you use normalmap plugin for Gimp, you have to choose Conversion -> Normalize only, and the use 1 in Scale field (default). Unfortunately there is a slight difference in how both filters use normalization. This is related to how both filters treat flat normal color reference. While the logical value would be 127, 127, 255 (the range is 0-255, not 1-256), but idTech 4 and other game engines use 128, 128, 255. So does nJob and other normalmap applications. Normalmap plugin for Gimp uses 127, 127, 255 though. You may think that one pixel isn’t that big of a deal, but it actually offsets reflected light source quite a bit. This surface uses black diffuse texture, white specular, and a light placed right above it:

 

obraz.png

 

 

Even with converting a solid color to normalmap, Gimp normalmap plugin uses 127, 127, 255. If you plan to use it, you have to keep that in mind.

 

 

Stacking / mixing normalmap layers

 

Long ago, there was this popular method of “getting more details” out of heightmaps by duplicating base layer, setting mode to Overlay, using Gaussian blur, and then repeating this step several times. That method destroys and distorts information stored in the RGB channels, as Overlay mode introduces contrast from the overlaid image to the pixels below it. Try to stick to making the most of your normalmap out of your layers combined in a heightmap and converted to a normalmap first.

 

That said, idtech4 is pretty sensitive to normalmaps, especially when used with speculars. Sometimes a slight change in the opacity will make a difference in your material, while it will be almost invisible on the normalmap texture. Prototyping it the usual way is too time-consuming, and prone to guesswork.

 

In such cases, you can prepare your base normalmap, and then put another one on top, e.g. to have some surface details, like cracks or dents. To do this correctly, you have to preserve depth information in your base normalmap though. The most important thing here is the blue channel: it represents shading when you’re looking at the surface in a perpendicular line. In order to mitigate this, you can use second normalmap in Overlay layer mode, but you have to keep the blue channel of the normalmap underneath intact. Overlay mode takes bright and dark pixels of the current layer, and makes the layer underneath bright or dark. If you create a layer and fill it with solid grey color (RGB 128), and then set its mode to Overlay, you’ll notice that the color has no effect, it becomes invisible. You have to do the same with blue channel of your second normalmap.

 

You can do this in Channels tab in Gimp. Just deselect all the channels but blue, and use the Bucket Fill tool with RGB 128 color selected. Be sure to hold Shift to Fill the whole selection. You should see the normalmap changing color to grey, with green and magenta shapes. Now you can set the layer mode to Overlay and adjust the opacity to your needs.

 

obraz.png

Smooth or rough? The choice is yours, just be mindful of consequences.

 

 

Next part will cover specular maps, how they work in TDM, and how you can use them to achieve interesting effects.

Edited by Judith
  • Like 4
Link to comment
Share on other sites

 


3. You dont want the light to decay into complete darkness, thats why you need an ambient_world light. Place another light in the same room, make the radius long enough so it encompasses the whole room. Make it an ambient light (properties -> Classname -> Lights -> atdm:ambient_world; light properties -> light texture -> ambientlightnfo). Set the color to something like RGB 8, 8, 8.

If you use this entity the setting are already set. You only need to set them if you create the light from scratch. The ambient_world light is identified by its unique name (not classname), and is important for missions as it is rendered differently. In an example setup as described here it is not necessary (although it doesn't hurt either). Neither the main ambient nor any other ambient light needs to be atdm:ambient_world.

FM's: Builder Roads, Old Habits, Old Habits Rebuild

Mapping and Scripting: Apples and Peaches

Sculptris Models and Tutorials: Obsttortes Models

My wiki articles: Obstipedia

Texture Blending in DR: DR ASE Blend Exporter

Link to comment
Share on other sites

Hmm, maybe I remembered it wrong, but if I just create a light and name it ambient_world, the map is dark. If I use both entity class and ambient_world name, it works as it should.

 

If I don't set the ambient_world name, I see a message in the console telling me that ambient_world hasn't been found, but there is a light with ambient_world entity class, so it will be used as one anyway.

Link to comment
Share on other sites

https://www.iddevnet.com/doom3/bumpmaps.php

 

I will refer to that later in the next part of the tutorial. I just found out that the old Gimp normalmap plugin wasn't usin this value, and there are certain consequences. In short, it's better to use nJob and the like.

Edited by Judith
Link to comment
Share on other sites

Hmm, maybe I remembered it wrong, but if I just create a light and name it ambient_world, the map is dark. If I use both entity class and ambient_world name, it works as it should.

 

If I don't set the ambient_world name, I see a message in the console telling me that ambient_world hasn't been found, but there is a light with ambient_world entity class, so it will be used as one anyway.

Well, you have to set both shader and light color when creating a new light.

 

If you don't set the name to ambient_world, it will work as an ambient light, of course, if it is one. But it will not use the shader reserved for the main ambient light (and setting ambient rendering to simple will most probably cause your mission to turn pretty dark). As the code comment says that it attempts to use a different one, there is a little chance that it picks the one you want it to, but that's a bit random.

 

I am refering to what I remember and would have to take a look at the code, though to see how it works exactly.

FM's: Builder Roads, Old Habits, Old Habits Rebuild

Mapping and Scripting: Apples and Peaches

Sculptris Models and Tutorials: Obsttortes Models

My wiki articles: Obstipedia

Texture Blending in DR: DR ASE Blend Exporter

Link to comment
Share on other sites

Very good work and very informative. I quite like that the information is extensive but it is always centered around how it will look in game, and talks about ways of doing that systematically.

 

"There are materials in stock TDM that use only _d and _n textures, but the other way around will be more beneficial to your materials. Specular complements diffuse so closely, that it should be included every time, while normalmap doesn’t always look right. Sometimes the surface is very smooth, and details from the diffuse texture may look ugly or blocky when transferred to a normalmap. If that’s the case, it will be better to make a material without a normalmap, rather than without a specular."

 

I really subscribe to that.

 

Looking foward to the next section.

  • Like 1
Link to comment
Share on other sites

I really subscribe to that.

 

 

I'm curious what your reasoning is. The prevailing opinion until now has been that specularmaps in idTech4 tend to make materials look like plastic. It's why we stopped using them for non-shiny materials early on.

Link to comment
Share on other sites

Oh, its just that Ive noticed that really fine surface details, specially on smaller texture sizes, usually look better with a good specular and a detailed diffuse map, rather than a normal bump, which (again, when the resolution is not that high) tends to make them look very rough, very coarse. IMO, a lot of situations where you need subtle details can be solved without a normal map.

  • Like 1
Link to comment
Share on other sites

I quite like that the information is extensive but it is always centered around how it will look in game, and talks about ways of doing that systematically.

 

Thanks, that's exactly what I'm going for: trying to understand the main principle and get predictable results every time.

 

As for the speculars, I only checked several TDM materials, but all of them were using grayscale speculars. The thing is, idTech4 also supports RGB speculars, and you can use that to get interesting results. Also, it's hard to get a decent look using speculars and TDM's stock entity lights, as they are very bright and contrasty, often burning out highlights, even without postprocessing enabled. Having more toned down lights solves this problem, although it has some gameplay and design consequences. That's either thing for a separate article, or at least a longer footnote :)

Edited by Judith
Link to comment
Share on other sites

The specular maps worked alright in Doom 3 because the environment called for lots of shiny metallic surfaces. But in a more real world setting there are more dielectric surfaces than not. And since you can't adjust glossiness to tone down bright highlights you darken your specular to compensate and sacrifice color saturation.

  • Like 1
Link to comment
Share on other sites

Couldn't that be solved by altering the parts of the interaction shader that deals with the specular part?

FM's: Builder Roads, Old Habits, Old Habits Rebuild

Mapping and Scripting: Apples and Peaches

Sculptris Models and Tutorials: Obsttortes Models

My wiki articles: Obstipedia

Texture Blending in DR: DR ASE Blend Exporter

Link to comment
Share on other sites

  • 1 month later...

Unfortunately the last part will take much longer to make. While the basic premise is mostly this: http://www.manufato.com/?p=902, I already saw cases where the greyscale specular actually looked better than the inverted diffuse color. Also, I've only worked with stone, wood, and metal so far. I need to investigate organic stuff, glass, maybe some wet surfaces as well, etc.

  • Like 1
Link to comment
Share on other sites

Couldn't that be solved by altering the parts of the interaction shader that deals with the specular part?

 

Yes it could. The solution to the "plastic problem" is the glossmap, which is a separate greyscale map which indicates the glossiness of each texel, to go alongside the specularmap which gives the colour and brightness of the reflection at that point.

 

The logical place to store the glossmap would be in the alpha channel of the specular map image, which is currently unused (of course there would need to be a suitable default value which matched the existing behaviour, since none of the current specularmap images will have alpha channels). I experimented with this in the very early days when D3 was still closed source, but could not get it to work, either because my shader was wrong or the engine simply wasn't passing through the specularmap's alpha channel to the shader at all. Now that everything is open source, it ought to be fixable.

  • Like 1
Link to comment
Share on other sites

  • 1 month later...

This is a post about (colored) specular maps and I don't have a better intro.
If you're familiar with the issue and have read threads like this, feel free to skip right down to where the pictures end.

The aim is to get a better understanding of systems currently in place in TDM renderer, and of possibility to change them. Some stuff has changed since vanilla idtech4, some stayed the same, it's not very clear and there seem to be some myths around it. I'll put it the way I see it, I expect to be corrected.

Basics:
According to iddevnet, specular map is "a gray scale image that defines how 'shiny' an object is". Immediately that's suspicious, since half of Doom 3 specmaps aren't grayscale at all. So, first we need to expand that: a specular map is an image that defines the color of the highlight on a given material. Like so:
WK0j5xO.jpg

 


Alright, why would you use anything but grayscale? Well for one, metals have different physical properties from everything else. To mimic the way light would interact with them in reality, one would color specular maps the same color as diffuse maps:
bzPCCwe.jpg

 


Most everything other than metals should have highlights matching the light that causes them. White light = white highlight, with a specular map tinting the highlight's color. If that were the case, why are all the organic specmaps colored as well? And why does this authoritative Quake 4 tutorial paint the specular of blood into bright blue?
q70B0yo.jpg

 


Well, there is no simple answer to that, it dives into tech talk right away. The engine should color the highlight white, but it doesn't. Instead, because of "black box" stuff with how the image textures go in, how they're processed, and how they end up on the screen, the contribution of a specular map gets skewed, and you end up with warped colors. Take cacodemon, for instance:
U9x6rNR.jpg

One has the original textures, the other has its specular map desaturated. Even though the diffusemap isn't that colorful you can see the effect (if maybe not from afar / in motion). A grayscale specmap doesn't give us a white highlight, instead it gets reddish.

Working around:
We know the theory (somewhat), we have the examples straight from the horse's mouth, seems like problem solved. Not quite. If we just invert diffuse and colorize our grayscale with it, it's pretty unlikely we'll get a perfect result:
xWBHJcO.jpg
(Actually, it's entirely possible it looks just fine to you if your monitor isn't calibrated, but completely different to someone else. That's a whole other can of worms - adjacent to the topic at hand, but too involved to include here)


Alright, there's some advice online on how you can blend your specular in "linear dodge" mode, emulating the way the engine would do it, let's try that out:
kPVcHFu.jpg
At this point it's harder to see it with a naked eye, but there's still a bluish tint. Well, let's try other modes, like "screen". Wouldn't make sense math-wise, but we're just looking for some baseline here.
Y8PDR1N.jpg
Hey, that looks alright. Getting reddish even.

 

7zI69QL.jpg
...Except from any other angle.

So, here we come to TDM specifics. On D3/Q4 this may have worked, but changes to the renderer since then might have made it impossible to get the "perfect" result. In particular, TDM has a universal Fresnel effect - surfaces get shinier the shallower the angle you observe them from. But it doesn't replace specmap, it intensifies it - which would mean a "perfect" specular color for one angle will be incorrect for every other angle.
There may have been some other changes as well. D3 had a bit of a stigma for looking 'plastic-y' (partially because it even had specmaps), so who knows what else is different. I tried deciphering the current interaction shader to get at some "safe" value for color blending, but it seems a bit over my head for now.

Fixing it:
Maybe I got it all wrong, but to me it seems that no specular map can be technically correct in TDM (not talking "physically correct", just the stated purpose of such a map). Now, it's not that terrible an issue in the first place, and even less so in TDM - it being mostly torch- and candle-lit. But even so, while there doesn't seem to be a 100% solution to this on the art side, the possible engine-side solution seems well-defined.
There's that NVidia article. There's RBDoom's gamma-correct implementation. And in general, while I can't into graphics code, it seems like converting colors from sRGB to linear and back at a couple of points is something largely engine-agnostic and not impossibly hard to implement.
But I expect nothing is easy, and there may be pitfalls around every corner. For example, the soft gamma might play into this both at specular contribution point and at output-to-monitor point.

One thing that shouldn't be much of a problem is existing assets: most TDM specmaps are grayscale already, meaning they "assume" the engine to be gamma-correct. So a change to renderer would only make them more correct, not break them. Even a couple of colored maps in e.g. Volta 2 don't rely on this "neutralization" hack.

Worth mentioning that I've seen claims of gloss maps and HDR being in TDM, and I don't think either is true. On the other hand, I've also seen Fresnel effect as a wishlist item, even though we have that already. So, while I'd sure be glad to see the issue of specmaps fixed, even without that at least knowing what's there and roughly how it works would help quite a bit. Hopefully, this post has been of some help as well.

  • Like 1
Link to comment
Share on other sites

IMO there is no 100% safe solution here, the approach depends on the surface type, and I don't mean the wood, metal etc. The only certain thing I know is that specular works like Add function, so it will add anything you put in it on top of the diffuse. These two have to work together to get a coherent effect. And while it's interesting, IMO most users don't need technical babble, they just need a set of tips and tricks for every surface type they might need (and that's one of the reasons I didn't write this yet).

 

Speculars for dark wood with thick, greasy varnish can look better with grayscale specular than with blueish ones. It's important to think what the last, "outer" layer is. Although greyscale speculars for non-metals typically look like blend between metal and plastic, so I rarely use it. Blueish specs for wood look a bit weird in perfect white "6500K" light, but no light in the game is like that. With torches and candles it actually looks okay, just watch the color saturation and output levels.

 

And the simple colorized diffuse method is not enough IMO. First of all, it's better to have your diffuse texture divided into a few components or groups: surface colors (normal mode), surface pattern (multiply), and surface dirt (multiply). This way, when you make speculars, you can invert the base color, and copy over the surface pattern and dirt. Then you can use Levels on surface color to get "basic specular behavior" of your surface. This is the most tricky part IMO. Remember that specular is relative to your diffuse and lights you use. It seems to me, that inverted (and mostly desaturated) colors influence the look of the hotspot: it's bigger and less contrasty than in metallic surfaces (or with greyscale specs). The intensity (output levels) adjust the strength of the effect. When you get that, you can adjust surface pattern and the dirt, in most cases it's about adjusting the opacity in multiply mode.

 

So that's my basic workflow, the rest is experiments. The easiest surface so far is glass, all you need is pure white specular and some dirt on top :)

 

Edit: btw. one of the mistakes in pictures above is putting the most dirt in specular texture. That works only for tiny fraction of surface damage, like dust or stains from water on a wooden surface for example. You won't see that dirt unless the surface is lit by a light, which looks weird. You have to put the dirt in both diffuse and specular texture, and then find balance between how intensive it has to be in both, to look correct in your lighting.

Edited by Judith
Link to comment
Share on other sites

The only certain thing I know is that specular works like Add function, so it will add anything you put in it on top of the diffuse.

One of the main reasons for writing the post is that this isn't exactly true. Even vanilla D3 shader looks like this:

# modulate by the specular map * 2
TEX    R2, fragment.texcoord[5], texture[5], 2D;
ADD    R2, R2, R2;
MAD    color, R1, R2, color;

So it adds specular twice. Except not really, since the whole gamma thing skews it as well. Sure, you can tweak each material individually, but it would be much nicer to have a baseline (e.g. "colorize at 20% blend") that would apply across the board. For now, using "screen" instead of "add" to check it seems like a compromise. And the point is that it doesn't have to be this way: if the problem gets solve code-wise, grayscale specular maps can look perfect for every material.

Edit: btw. one of the mistakes in pictures above is putting the most dirt in specular texture. That works only for tiny fraction of surface damage, like dust or stains from water on a wooden surface for example. You won't see that dirt unless the surface is lit by a light, which looks weird. You have to put the dirt in both diffuse and specular texture, and then find balance between how intensive it has to be in both, to look correct in your lighting.

I like it when the surface 'plays' when hit by the light, and using the same details across diffuse/normal/specular always seems like a waste. It isn't necessarily dirt either: could be damaged varnish, dust on top, all sorts of hand- and finger-prints, etc. I've kind of thrown this together in an evening as well, these are 2k, usually I downsize them, and slightly harsher specmap helps retain the details.

Good advice on components/groups for working with a single layered image. With multiple outputs in S.Designer it's a bit easier, but the general idea's the same: you first make a common-ground grayscale base (usually doubles as height to get your normal map from), then add details for color map and microsurface, some of these details overlap, some not.

Those shots look really good, what application are you using..?

Substance Designer / Painter. But I'm sure other software, like Quixel and whatever else is out there, can get you similar results. You could do it manually too, but doing things like edge detect from the normal map by hand each time is way too much work.

Edited by chedap
Link to comment
Share on other sites

Fantastic information boys, I dont think we had this amount of detail and insight into the technical aspects of texture-making before. Great contributions. Will definitely be coming back for more. One aspect of TDM that always made me a bit sad was that vast surface areas tend to be in "total darkness" at all times and are not subjected to lights at all, apart from the ambient one (unless you bring a light source with you). This way, a player that keeps to the dark is going to be spending a lot of time looking at textures in their worst state - bascily a washed out, darkened and contrast free albedo with weird normals and zero speculars. Pretty charmless. Thats why even the shadow areas should have some sort of gradient, but since we dont have bounced lights, its not really possible. The only way to simulate this that comes to mind is to have a subtle dark fog covering the whole map, this would make areas that are distant from light sources to become increasingly darker. But this has a cost on performance, thats why I didnt use it in my own map (I might revisit that now that I mention it).

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

  • Recent Status Updates

    • Petike the Taffer

      I've finally managed to log in to The Dark Mod Wiki. I'm back in the saddle and before the holidays start in full, I'll be adding a few new FM articles and doing other updates. Written in Stone is already done.
      · 4 replies
    • nbohr1more

      TDM 15th Anniversary Contest is now active! Please declare your participation: https://forums.thedarkmod.com/index.php?/topic/22413-the-dark-mod-15th-anniversary-contest-entry-thread/
       
      · 0 replies
    • JackFarmer

      @TheUnbeholden
      You cannot receive PMs. Could you please be so kind and check your mailbox if it is full (or maybe you switched off the function)?
      · 1 reply
    • OrbWeaver

      I like the new frob highlight but it would nice if it was less "flickery" while moving over objects (especially barred metal doors).
      · 4 replies
    • nbohr1more

      Please vote in the 15th Anniversary Contest Theme Poll
       
      · 0 replies
×
×
  • Create New...