Jump to content


Photo

Tutorial: Better textures for your materials (WIP, part 2a is up)

tutorial textures materials

27 replies to this topic

#1 Judith

Judith

    Advanced Member

  • Member
  • PipPipPip
  • 1606 posts

Posted 04 June 2018 - 05:46 AM

*
POPULAR

This tutorial will cover the basics of preparing good textures for your custom materials. Youll get to know how individual textures work in TDM, and how to make them react to light in consistent, controllable way. Ill be using simple opaque materials for most examples. Once you develop decent basic workflow, it's easier to work with transparencies, glow, cubemaps, and other more complex materials.
 
Part 0: Basic premise
 
TDM engine (idTech 4) uses non-PBR workflow. This means that there isn't one correct way of making textures for your materials, they won't be physically correct. In pre-PBR games you have to make materials in relation to your lighting model, and TDM/idTech 4 is no exception here. That said, I think you can have relative flexibility here. As long as you don't use extreme values for your lights, you should be able to use the same materials in daytime and nighttime scenarios.
 
Since the whole thing is a bit relative, it might be a bit overwhelming to figure out a starting point for your workflow. I found its useful to keep in mind a few ground rules (and this will be more important during in-engine tests):
 
1. You need an "average" light value for to establish a frame of reference.
2. Materials have to look correct with default game brightness and gamma settings. Brightness is 1, Gamma is 1.2.
3. Materials have to look correct without any post processing enabled, and they have to look good with default post processing on too. Burned highlights are acceptable for post processing, if the material looks as intended.
 
Keep last two points in mind as you create textures and test them in the engine. First point requires some experiments to see how TDM surfaces, light gem, and AI react to lights. What I found out, is that you can easily use grey (RGB 128, 128, 128) light as a kind of "photo studio light" to ensure that your textures behave correctly in the engine and have proper colors.
One thing you might want to keep in mind as well is falloff textures. My favorite multi-purpose light texture is falloff_exp2, as it seems close to inverse square method of calculating light falloff in other engines. I use it for most light sources, and it seems very good for tests.
 
General workflow suggestion
 
I prefer using Gimp and having each texture saved as .xcf file. I don't have to merge any layers, as Gimp uses "copy visible" option while exporting the result to .tga. Gimp will also remember the path and the filename after exporting, so, if I change something and want to export the updated texture, all I need is to use the Export shortcut (Ctrl+E), and update the textures in the engine with reloadimages command.
 
You'll want to have these files open and available for edition and export simultaneously. That's because all images influence each other, and contribute to the overall look of a surface:
 
Diffuse affects brightness, contrast, saturation, but also specularity and bloom highlights in post processing.
Specular map affects contrast and can emphasize darker areas of the image, even if you don't see a light reflected. It also affects the look and strength of a normalmap.
Normal map affects how specularity works on a surface, it also influences darker areas of the diffuse.
 
Test environment map
 
1. In DarkRadiant, create an empty cubic room that youll texture with your material. Dimensions depend mostly on assumed texture scale. Typically, I use 2048 textures that will be scaled down 8 times (x 0.125), so one tile will occupy in-game space of 256 by 256 units. Since I want to check whether my material tiles properly, I often use a 512 x 512 x 512 cube.
 
2. Place a light in the middle of the room. If your room is a 512 cube, make the radius 512 as well. This way your light will hit the wall in the middle of its falloff. If you want to use more realistic falloff, use light texture falloff_exp2. Set color to RGB 128, 128, 128.
 
3. You dont want the light to decay into complete darkness, thats why you need an ambient_world light. Place another light in the same room, make the radius long enough so it encompasses the whole room. Make it an ambient light (properties -> Classname -> Lights -> atdm:ambient_world; light properties -> light texture -> ambientlightnfo). Set the color to something like RGB 8, 8, 8.
 
4. Apply your material on room walls, place PlayerStart inside, and save your map. Run TDM, compile and run your map.
 
 
Now you have set up environment for testing materials.
 
obraz.png
obraz.png
A test map with example wall material applied.
 
Next part will cover diffuse textures, why its important to manage their color range, and how to do it in consistent manner.


Edited by Judith, 10 September 2018 - 05:08 PM.

  • AluminumHaste, SiyahParsomen, Dragofer and 4 others like this

#2 Judith

Judith

    Advanced Member

  • Member
  • PipPipPip
  • 1606 posts

Posted 04 June 2018 - 05:46 AM

*
POPULAR

Part 1: Diffuse workflow

This is a difficult texture to make, since it stores so much information. It’s not only about color values: even without specular, diffuse brightness and contrast influence surface specularity and highlights.

Most important thing to keep in mind while making diffuse textures is to maintain consistency in how they react to light. If you don’t control brightness and contrast of your diffuse textures, you’ll encounter problems with the same surface looking strange in different lighting conditions. What is probably even more important, the whole set of your materials will look jarring, if was no consistent rule used to make them.

The most effective way to tackle this problem is to establish a color range that you’ll use for all your diffuse textures. Along with consistency, this will also tackle a problem of displaying your materials on different monitors, and with different user settings (consistent with rule 2 and 3 above). This is called image normalization, and it’s similar to methods used by other media, like movies or television.

You’ve probably noticed that no movie or TV show uses completely dark or fully bright images. This method decreases image contrast a bit, but it gives a lot of leeway to both TV manufacturers (as there are many color and gamma profiles available) and users, who can adjust image settings to their liking. In all those instances the image will look correct.
In fact, my rule for normalizing diffuse textures is a simplified movie and TV standard called rec. 709. What it means in practical terms, is that you want to limit your output color range to 16 for blacks, and 240 for whites. But before that, you need to make sure your diffuse texture uses the whole color range, without clipping blacks or whites, so you don’t cut off any details in both bright parts and shadows.

The most common way to achieve all this is to use Levels tool.

obraz.png

This is my tiled floor texture along with Levels tool. Black shape is a histogram, it represents the amount of pixels in the image along with their brightness value. You can see that there is a lot of space between darkest pixels on the left, and the value of 0, which represents pure black. The same goes for bright pixels and the maximum value of 255. That means my texture doesn’t use full color range, and you can see it on the image itself as the lack of contrast.

To fix this, I have to adjust Input levels.

obraz.png


The image has better contrast, as it uses the whole range now. Now I have to clamp this range between 16 and 240 with Output levels.

obraz.png

Now the image lost a bit of its contrast, but it has room to be modified further in the material (specular and normal), by the engine postprocess settings, and by brightness and gamma settings.

Note that the image used in the example is mostly greyscale with a bit of blue, but if your texture has more vivid colors, you’ll notice that the saturation has increased. That’s what happens when you increase contrast, you have to offset this with by using slight desaturation. In general, unless you’re aiming for a particular style, your diffuse textures should be slightly desaturated, as they will be affected not only by brightness, but also by the color of your lights. Your test environment uses neutral “6500K-like” light that doesn’t occur in the game world. For actual map lighting, you always want to have some color in your lights, whether you’re aiming for incandescent, fluorescent, or anything in-between.
(Note to self: I probably should change example images to something more colorful later)

Next part will cover normalmaps, how they work in TDM and how you can tweak them without breaking their depth information.
  • Bikerdude, AluminumHaste, SiyahParsomen and 3 others like this

#3 Judith

Judith

    Advanced Member

  • Member
  • PipPipPip
  • 1606 posts

Posted 04 June 2018 - 05:47 AM

*
POPULAR

Part 2: Normal maps
 
Normal map is a vector map stored in RGB format, where each channel represents shading in one of the dimensions (X, Y, Z). That’s why you can’t treat normalmaps like any other images: using tools like brightness or sharpening can destroy information stored in RGB channels. There are several filters available for making normalmap. There is an old normalmap plugin for Gimp, but I wouldn’t recommend it (see below), you can also try out software like nDo, Crazybump, or nJob. Since nJob is free and pretty good for typical applications, I’m using it as my main tool.
 
To create good normalmaps from 2d images, you need to prepare a heightmap image that will be converted by the filter. Heightmap is a greyscale image, where dark pixels are lows and bright pixels are highs (grey color is kind of middle ground). Heightmap can be based on your diffuse image converted to greyscale, but typically you have to do some more work to get proper results. Applying normalmap filter to color images often will result with wrong look. Classic example is a red brick texture with white grout.
 
obraz.png
 

Most normalmap filters will interpret this as in the lower part of the image:

obraz.png
 

Sometimes a simple value invert will be enough to create correct heightmap. In most cases, your ability to create good normalmaps will depend on how you made your diffuse map. If your diffuse texture is composed from several layers, you will have greater degree of control over the final result.
For example, my tile floor was made with 5 layers:
 
obraz.png


This way I can decide which parts of the image I want to include in the heightmap, and I can regulate their intensity separately. If I had only the final image, the image processed by normalmap filter would result in this:

obraz.png

 

Normalmap filter took all pixels in the image and converted them into depth information. That includes things like stone pattern, dirt, and tile color. That’s not a good result. Here the depth information is mostly grout and some subtle edges taken from the stone pattern to indicate age.

obraz.png

 

 

Example workflow for creating normalmaps

  1. When you finish making your diffuse texture, save all the layers as another .xcf file. This will be your base for heightmap.
  2. Desaturate all the layers, and see if they are representing the height in correct way: dark pixels for cavity and bright pixels for convexity. Adjust layers’ modes, opacity, or color values, if you see that they don’t represent bumps or indents in the way they should.
  3. Use Copy visible (Ctrl+Shift+C) in Gimp to copy the result to clipboard.
  4. Paste the image into nJob and use Heightmap to normalmap filter (or press 4 on the keyboard).
  5. Adjust the filter settings. I use Traditional mode most of the time. There is no numeric scale here, but I usually reset everything first, and then I click on the scale, so the slider moves by the same interval. Now adjust Scale and Blur radius to your liking. If you don’t know what to go for, just click the Scale slide once. Copy (Ctrl+C) the result, paste it back to Gimp, and export it as your normalmap.
  6. Launch TDM and run your test material map to see whether the result is correct. If not, switch to nJob, Undo (Ctr+Z) the filter operation, and repeat steps in point 5 until you get desired/correct result.

If you don’t know what to look for, search online for photo, or video reference of a surface you’re making. This will give you a sense of direction and you’ll eliminate guesswork. That said, don’t be afraid to experiment. Using some crazy values might lead you to interesting effects that you’ll be able to use later!

 

 

Notes:

 

Diffuse + normal vs. diffuse + normal + specular

 

If you only have diffuse and normalmap textures in your material, the result will be different when you add a specular map. Your normalmap will look stronger with specular, and your material will have more depth. In general, there’s no reason why you wouldn’t want to create a specular map: all real-world surfaces are shiny and reflective to some extent. There are materials in stock TDM that use only _d and _n textures, but the other way around will be more beneficial to your materials. Specular complements diffuse so closely, that it should be included every time, while normalmap doesn’t always look right. Sometimes the surface is very smooth, and details from the diffuse texture may look ugly or blocky when transferred to a normalmap. If that’s the case, it will be better to make a material without a normalmap, rather than without a specular.

 

As I mentioned earlier, it good to prototype 3 maps as fast as possible, and work on them simultaneously – they all influence each other. If you don’t know what to look for in a specular yet, and you want to prototype your normal first, you can use either _grey system texture in the specular slot.

 

obraz.png

Floor material with and without specular.

 

 

Normalizing normalmap images

 

One of most important steps in making normalmaps is normalizing your image. This is something similar to setting proper color range for your diffuse, so the game engine will interpret the image correctly. nJob does it pretty well automatically. If you use normalmap plugin for Gimp, you have to choose Conversion -> Normalize only, and the use 1 in Scale field (default). Unfortunately there is a slight difference in how both filters use normalization. This is related to how both filters treat flat normal color reference. While the logical value would be 127, 127, 255 (the range is 0-255, not 1-256), but idTech 4 and other game engines use 128, 128, 255. So does nJob and other normalmap applications. Normalmap plugin for Gimp uses 127, 127, 255 though. You may think that one pixel isn’t that big of a deal, but it actually offsets reflected light source quite a bit. This surface uses black diffuse texture, white specular, and a light placed right above it:

 

obraz.png

 

 

Even with converting a solid color to normalmap, Gimp normalmap plugin uses 127, 127, 255. If you plan to use it, you have to keep that in mind.

 

 

Stacking / mixing normalmap layers

 

Long ago, there was this popular method of “getting more details” out of heightmaps by duplicating base layer, setting mode to Overlay, using Gaussian blur, and then repeating this step several times. That method destroys and distorts information stored in the RGB channels, as Overlay mode introduces contrast from the overlaid image to the pixels below it. Try to stick to making the most of your normalmap out of your layers combined in a heightmap and converted to a normalmap first.

 

That said, idtech4 is pretty sensitive to normalmaps, especially when used with speculars. Sometimes a slight change in the opacity will make a difference in your material, while it will be almost invisible on the normalmap texture. Prototyping it the usual way is too time-consuming, and prone to guesswork.

 

In such cases, you can prepare your base normalmap, and then put another one on top, e.g. to have some surface details, like cracks or dents. To do this correctly, you have to preserve depth information in your base normalmap though. The most important thing here is the blue channel: it represents shading when you’re looking at the surface in a perpendicular line. In order to mitigate this, you can use second normalmap in Overlay layer mode, but you have to keep the blue channel of the normalmap underneath intact. Overlay mode takes bright and dark pixels of the current layer, and makes the layer underneath bright or dark. If you create a layer and fill it with solid grey color (RGB 128), and then set its mode to Overlay, you’ll notice that the color has no effect, it becomes invisible. You have to do the same with blue channel of your second normalmap.

 

You can do this in Channels tab in Gimp. Just deselect all the channels but blue, and use the Bucket Fill tool with RGB 128 color selected. Be sure to hold Shift to Fill the whole selection. You should see the normalmap changing color to grey, with green and magenta shapes. Now you can set the layer mode to Overlay and adjust the opacity to your needs.

 

obraz.png

Smooth or rough? The choice is yours, just be mindful of consequences.

 

 

Next part will cover specular maps, how they work in TDM, and how you can use them to achieve interesting effects.


Edited by Judith, 06 June 2018 - 07:11 AM.

  • Bikerdude, AluminumHaste, SiyahParsomen and 2 others like this

#4 Judith

Judith

    Advanced Member

  • Member
  • PipPipPip
  • 1606 posts

Posted 04 June 2018 - 05:47 AM

(Reserved for part 3 specular maps)


Edited by Judith, 04 June 2018 - 05:48 AM.


#5 Obsttorte

Obsttorte

    Scripting guru, Mapper

  • Active Developer
  • PipPipPipPipPip
  • 5683 posts

Posted 04 June 2018 - 01:55 PM


3. You dont want the light to decay into complete darkness, thats why you need an ambient_world light. Place another light in the same room, make the radius long enough so it encompasses the whole room. Make it an ambient light (properties -> Classname -> Lights -> atdm:ambient_world; light properties -> light texture -> ambientlightnfo). Set the color to something like RGB 8, 8, 8.

If you use this entity the setting are already set. You only need to set them if you create the light from scratch. The ambient_world light is identified by its unique name (not classname), and is important for missions as it is rendered differently. In an example setup as described here it is not necessary (although it doesn't hurt either). Neither the main ambient nor any other ambient light needs to be atdm:ambient_world.


FM's: Builder Roads, Old Habits, Old Habits Rebuild
WIP's: Several. Although after playing Thief 4 I really wanna make a city mission.
Mapping and Scripting: Apples and Peaches
Sculptris Models and Tutorials: Obsttortes Models
My wiki articles: Obstipedia
Let's Map TDM YouTube playlist: ObstlerTube
Texture Blending in DR: DR ASE Blend Exporter

End of shameless self promotion.

#6 Judith

Judith

    Advanced Member

  • Member
  • PipPipPip
  • 1606 posts

Posted 04 June 2018 - 02:32 PM

Hmm, maybe I remembered it wrong, but if I just create a light and name it ambient_world, the map is dark. If I use both entity class and ambient_world name, it works as it should.

 

If I don't set the ambient_world name, I see a message in the console telling me that ambient_world hasn't been found, but there is a light with ambient_world entity class, so it will be used as one anyway.



#7 Judith

Judith

    Advanced Member

  • Member
  • PipPipPip
  • 1606 posts

Posted 05 June 2018 - 07:19 AM

Before I go further, a question about normals: what is proper flat normal color for idTech4? I can't find any info on that.

 

Edit: nvm, found it.


Edited by Judith, 05 June 2018 - 07:44 AM.


#8 Bikerdude

Bikerdude

    Mod hero

  • Member
  • PipPipPipPipPip
  • 20189 posts

Posted 05 June 2018 - 10:35 AM

Edit: nvm, found it.

Wonna post the link, so people that come here canl see what your refering too etc.


Edited by Bikerdude, 05 June 2018 - 12:23 PM.


#9 Judith

Judith

    Advanced Member

  • Member
  • PipPipPip
  • 1606 posts

Posted 05 June 2018 - 10:39 AM

https://www.iddevnet...m3/bumpmaps.php

 

I will refer to that later in the next part of the tutorial. I just found out that the old Gimp normalmap plugin wasn't usin this value, and there are certain consequences. In short, it's better to use nJob and the like.


Edited by Judith, 05 June 2018 - 10:43 AM.


#10 Judith

Judith

    Advanced Member

  • Member
  • PipPipPip
  • 1606 posts

Posted 06 June 2018 - 07:28 AM

The second part is up, I will need a few more days for the last part. Probably won't happen until next week.



#11 Obsttorte

Obsttorte

    Scripting guru, Mapper

  • Active Developer
  • PipPipPipPipPip
  • 5683 posts

Posted 07 June 2018 - 03:21 AM

Hmm, maybe I remembered it wrong, but if I just create a light and name it ambient_world, the map is dark. If I use both entity class and ambient_world name, it works as it should.

 

If I don't set the ambient_world name, I see a message in the console telling me that ambient_world hasn't been found, but there is a light with ambient_world entity class, so it will be used as one anyway.

Well, you have to set both shader and light color when creating a new light.

 

If you don't set the name to ambient_world, it will work as an ambient light, of course, if it is one. But it will not use the shader reserved for the main ambient light (and setting ambient rendering to simple will most probably cause your mission to turn pretty dark). As the code comment says that it attempts to use a different one, there is a little chance that it picks the one you want it to, but that's a bit random.

 

I am refering to what I remember and would have to take a look at the code, though to see how it works exactly.


FM's: Builder Roads, Old Habits, Old Habits Rebuild
WIP's: Several. Although after playing Thief 4 I really wanna make a city mission.
Mapping and Scripting: Apples and Peaches
Sculptris Models and Tutorials: Obsttortes Models
My wiki articles: Obstipedia
Let's Map TDM YouTube playlist: ObstlerTube
Texture Blending in DR: DR ASE Blend Exporter

End of shameless self promotion.

#12 RPGista

RPGista

    Advanced Member

  • Member
  • PipPipPip
  • 1590 posts

Posted 07 June 2018 - 05:49 PM

Very good work and very informative. I quite like that the information is extensive but it is always centered around how it will look in game, and talks about ways of doing that systematically.

 

"There are materials in stock TDM that use only _d and _n textures, but the other way around will be more beneficial to your materials. Specular complements diffuse so closely, that it should be included every time, while normalmap doesn’t always look right. Sometimes the surface is very smooth, and details from the diffuse texture may look ugly or blocky when transferred to a normalmap. If that’s the case, it will be better to make a material without a normalmap, rather than without a specular."

 

I really subscribe to that.

 

Looking foward to the next section.


  • Judith likes this

#13 Springheel

Springheel

    Creative Director (retired)

  • Admin
  • 37385 posts

Posted 07 June 2018 - 06:44 PM

I really subscribe to that.

 

 

I'm curious what your reasoning is.  The prevailing opinion until now has been that specularmaps in idTech4 tend to make materials look like plastic.  It's why we stopped using them for non-shiny materials early on.


TDM Missions:   A Score to Settle   *   A Reputation to Uphold   *   A New Job   *    A Matter of Hours
 
Video Series:   Springheel's Modules   *   Speedbuild Challenge   *   New Mappers Workshop  *   Building Traps

#14 RPGista

RPGista

    Advanced Member

  • Member
  • PipPipPip
  • 1590 posts

Posted 07 June 2018 - 11:26 PM

Oh, its just that Ive noticed that really fine surface details, specially on smaller texture sizes, usually look better with a good specular and a detailed diffuse map, rather than a normal bump, which (again, when the resolution is not that high) tends to make them look very rough, very coarse. IMO, a lot of situations where you need subtle details can be solved without a normal map.


  • Judith likes this

#15 Judith

Judith

    Advanced Member

  • Member
  • PipPipPip
  • 1606 posts

Posted 08 June 2018 - 01:43 AM

I quite like that the information is extensive but it is always centered around how it will look in game, and talks about ways of doing that systematically.

 

Thanks, that's exactly what I'm going for: trying to understand the main principle and get predictable results every time.

 

As for the speculars, I only checked several TDM materials, but all of them were using grayscale speculars. The thing is, idTech4 also supports RGB speculars, and you can use that to get interesting results. Also, it's hard to get a decent look using speculars and TDM's stock entity lights, as they are very bright and contrasty, often burning out highlights, even without postprocessing enabled. Having more toned down lights solves this problem, although it has some gameplay and design consequences. That's either thing for a separate article, or at least a longer footnote :)


Edited by Judith, 08 June 2018 - 02:53 AM.


#16 rich_is_bored

rich_is_bored

    Advanced Member

  • Member
  • PipPipPip
  • 883 posts

Posted 08 June 2018 - 07:54 PM

The specular maps worked alright in Doom 3 because the environment called for lots of shiny metallic surfaces. But in a more real world setting there are more dielectric surfaces than not. And since you can't adjust glossiness to tone down bright highlights you darken your specular to compensate and sacrifice color saturation.


  • RPGista likes this

#17 Obsttorte

Obsttorte

    Scripting guru, Mapper

  • Active Developer
  • PipPipPipPipPip
  • 5683 posts

Posted 09 June 2018 - 03:12 AM

Couldn't that be solved by altering the parts of the interaction shader that deals with the specular part?


FM's: Builder Roads, Old Habits, Old Habits Rebuild
WIP's: Several. Although after playing Thief 4 I really wanna make a city mission.
Mapping and Scripting: Apples and Peaches
Sculptris Models and Tutorials: Obsttortes Models
My wiki articles: Obstipedia
Let's Map TDM YouTube playlist: ObstlerTube
Texture Blending in DR: DR ASE Blend Exporter

End of shameless self promotion.

#18 Judith

Judith

    Advanced Member

  • Member
  • PipPipPip
  • 1606 posts

Posted 09 June 2018 - 03:47 AM

You can use speculars based on inverted diffuse maps to simulate dielectric surfaces, the only problem is that you don't have a separate parameter to adjust specular power.



#19 Judith

Judith

    Advanced Member

  • Member
  • PipPipPip
  • 1606 posts

Posted 20 July 2018 - 02:19 AM

Unfortunately the last part will take much longer to make. While the basic premise is mostly this: http://www.manufato.com/?p=902, I already saw cases where the greyscale specular actually looked better than the inverted diffuse color. Also, I've only worked with stone, wood, and metal so far. I need to investigate organic stuff, glass, maybe some wet surfaces as well, etc.


  • Obsttorte likes this

#20 OrbWeaver

OrbWeaver

    Mod hero

  • Active Developer
  • PipPipPipPipPip
  • 7507 posts

Posted 20 July 2018 - 11:15 AM

Couldn't that be solved by altering the parts of the interaction shader that deals with the specular part?

 

Yes it could. The solution to the "plastic problem" is the glossmap, which is a separate greyscale map which indicates the glossiness of each texel, to go alongside the specularmap which gives the colour and brightness of the reflection at that point.

 

The logical place to store the glossmap would be in the alpha channel of the specular map image, which is currently unused (of course there would need to be a suitable default value which matched the existing behaviour, since none of the current specularmap images will have alpha channels). I experimented with this in the very early days when D3 was still closed source, but could not get it to work, either because my shader was wrong or the engine simply wasn't passing through the specularmap's alpha channel to the shader at all. Now that everything is open source, it ought to be fixable.


  • RPGista likes this

#21 chedap

chedap

    Member

  • Member
  • PipPip
  • 102 posts

Posted 04 September 2018 - 09:37 PM

This is a post about (colored) specular maps and I don't have a better intro.
If you're familiar with the issue and have read threads like this, feel free to skip right down to where the pictures end.
 
The aim is to get a better understanding of systems currently in place in TDM renderer, and of possibility to change them. Some stuff has changed since vanilla idtech4, some stayed the same, it's not very clear and there seem to be some myths around it. I'll put it the way I see it, I expect to be corrected.
 
Basics:
According to iddevnet, specular map is "a gray scale image that defines how 'shiny' an object is". Immediately that's suspicious, since half of Doom 3 specmaps aren't grayscale at all. So, first we need to expand that: a specular map is an image that defines the color of the highlight on a given material. Like so:
WK0j5xO.jpg

 

 
Alright, why would you use anything but grayscale? Well for one, metals have different physical properties from everything else. To mimic the way light would interact with them in reality, one would color specular maps the same color as diffuse maps:
bzPCCwe.jpg

 

 
Most everything other than metals should have highlights matching the light that causes them. White light = white highlight, with a specular map tinting the highlight's color. If that were the case, why are all the organic specmaps colored as well? And why does this authoritative Quake 4 tutorial paint the specular of blood into bright blue?
q70B0yo.jpg

 

 
Well, there is no simple answer to that, it dives into tech talk right away. The engine should color the highlight white, but it doesn't. Instead, because of "black box" stuff with how the image textures go in, how they're processed, and how they end up on the screen, the contribution of a specular map gets skewed, and you end up with warped colors. Take cacodemon, for instance:
U9x6rNR.jpg

One has the original textures, the other has its specular map desaturated. Even though the diffusemap isn't that colorful you can see the effect (if maybe not from afar / in motion). A grayscale specmap doesn't give us a white highlight, instead it gets reddish.
 
Working around:
We know the theory (somewhat), we have the examples straight from the horse's mouth, seems like problem solved. Not quite. If we just invert diffuse and colorize our grayscale with it, it's pretty unlikely we'll get a perfect result:
xWBHJcO.jpg
(Actually, it's entirely possible it looks just fine to you if your monitor isn't calibrated, but completely different to someone else. That's a whole other can of worms - adjacent to the topic at hand, but too involved to include here)

 
Alright, there's some advice online on how you can blend your specular in "linear dodge" mode, emulating the way the engine would do it, let's try that out:
kPVcHFu.jpg
At this point it's harder to see it with a naked eye, but there's still a bluish tint. Well, let's try other modes, like "screen". Wouldn't make sense math-wise, but we're just looking for some baseline here.
Y8PDR1N.jpg
Hey, that looks alright. Getting reddish even.

 

7zI69QL.jpg
...Except from any other angle.
 
So, here we come to TDM specifics. On D3/Q4 this may have worked, but changes to the renderer since then might have made it impossible to get the "perfect" result. In particular, TDM has a universal Fresnel effect - surfaces get shinier the shallower the angle you observe them from. But it doesn't replace specmap, it intensifies it - which would mean a "perfect" specular color for one angle will be incorrect for every other angle.
There may have been some other changes as well. D3 had a bit of a stigma for looking 'plastic-y' (partially because it even had specmaps), so who knows what else is different. I tried deciphering the current interaction shader to get at some "safe" value for color blending, but it seems a bit over my head for now.

Fixing it:
Maybe I got it all wrong, but to me it seems that no specular map can be technically correct in TDM (not talking "physically correct", just the stated purpose of such a map). Now, it's not that terrible an issue in the first place, and even less so in TDM - it being mostly torch- and candle-lit. But even so, while there doesn't seem to be a 100% solution to this on the art side, the possible engine-side solution seems well-defined.
There's that NVidia article. There's RBDoom's gamma-correct implementation. And in general, while I can't into graphics code, it seems like converting colors from sRGB to linear and back at a couple of points is something largely engine-agnostic and not impossibly hard to implement.
But I expect nothing is easy, and there may be pitfalls around every corner. For example, the soft gamma might play into this both at specular contribution point and at output-to-monitor point.

One thing that shouldn't be much of a problem is existing assets: most TDM specmaps are grayscale already, meaning they "assume" the engine to be gamma-correct. So a change to renderer would only make them more correct, not break them. Even a couple of colored maps in e.g. Volta 2 don't rely on this "neutralization" hack.

Worth mentioning that I've seen claims of gloss maps and HDR being in TDM, and I don't think either is true. On the other hand, I've also seen Fresnel effect as a wishlist item, even though we have that already. So, while I'd sure be glad to see the issue of specmaps fixed, even without that at least knowing what's there and roughly how it works would help quite a bit. Hopefully, this post has been of some help as well.


  • Bikerdude and RPGista like this

#22 Judith

Judith

    Advanced Member

  • Member
  • PipPipPip
  • 1606 posts

Posted 05 September 2018 - 01:20 AM

IMO there is no 100% safe solution here, the approach depends on the surface type, and I don't mean the wood, metal etc. The only certain thing I know is that specular works like Add function, so it will add anything you put in it on top of the diffuse. These two have to work together to get a coherent effect. And while it's interesting, IMO most users don't need technical babble, they just need a set of tips and tricks for every surface type they might need (and that's one of the reasons I didn't write this yet).

 

Speculars for dark wood with thick, greasy varnish can look better with grayscale specular than with blueish ones. It's important to think what the last, "outer" layer is. Although greyscale speculars for non-metals typically look like blend between metal and plastic, so I rarely use it. Blueish specs for wood look a bit weird in perfect white "6500K" light, but no light in the game is like that. With torches and candles it actually looks okay, just watch the color saturation and output levels.

 

And the simple colorized diffuse method is not enough IMO. First of all, it's better to have your diffuse texture divided into a few components or groups: surface colors (normal mode), surface pattern (multiply), and surface dirt (multiply). This way, when you make speculars, you can invert the base color, and copy over the surface pattern and dirt. Then you can use Levels on surface color to get "basic specular behavior" of your surface. This is the most tricky part IMO. Remember that specular is relative to your diffuse and lights you use. It seems to me, that inverted (and mostly desaturated) colors influence the look of the hotspot: it's bigger and less contrasty than in metallic surfaces (or with greyscale specs). The intensity (output levels) adjust the strength of the effect. When you get that, you can adjust surface pattern and the dirt, in most cases it's about adjusting the opacity in multiply mode.

 

So that's my basic workflow, the rest is experiments. The easiest surface so far is glass, all you need is pure white specular and some dirt on top :)

 

Edit: btw. one of the mistakes in pictures above is putting the most dirt in specular texture. That works only for tiny fraction of surface damage, like dust or stains from water on a wooden surface for example. You won't see that dirt unless the surface is lit by a light, which looks weird. You have to put the dirt in both diffuse and specular texture, and then find balance between how intensive it has to be in both, to look correct in your lighting.


Edited by Judith, 05 September 2018 - 01:32 AM.


#23 Bikerdude

Bikerdude

    Mod hero

  • Member
  • PipPipPipPipPip
  • 20189 posts

Posted 05 September 2018 - 01:51 AM

Those shots look really good, what application are you using..?



#24 chedap

chedap

    Member

  • Member
  • PipPip
  • 102 posts

Posted 05 September 2018 - 09:50 AM

The only certain thing I know is that specular works like Add function, so it will add anything you put in it on top of the diffuse.

One of the main reasons for writing the post is that this isn't exactly true. Even vanilla D3 shader looks like this:

# modulate by the specular map * 2
TEX    R2, fragment.texcoord[5], texture[5], 2D;
ADD    R2, R2, R2;
MAD    color, R1, R2, color;

So it adds specular twice. Except not really, since the whole gamma thing skews it as well. Sure, you can tweak each material individually, but it would be much nicer to have a baseline (e.g. "colorize at 20% blend") that would apply across the board. For now, using "screen" instead of "add" to check it seems like a compromise. And the point is that it doesn't have to be this way: if the problem gets solve code-wise, grayscale specular maps can look perfect for every material.

Edit: btw. one of the mistakes in pictures above is putting the most dirt in specular texture. That works only for tiny fraction of surface damage, like dust or stains from water on a wooden surface for example. You won't see that dirt unless the surface is lit by a light, which looks weird. You have to put the dirt in both diffuse and specular texture, and then find balance between how intensive it has to be in both, to look correct in your lighting.

I like it when the surface 'plays' when hit by the light, and using the same details across diffuse/normal/specular always seems like a waste. It isn't necessarily dirt either: could be damaged varnish, dust on top, all sorts of hand- and finger-prints, etc. I've kind of thrown this together in an evening as well, these are 2k, usually I downsize them, and slightly harsher specmap helps retain the details.

Good advice on components/groups for working with a single layered image. With multiple outputs in S.Designer it's a bit easier, but the general idea's the same: you first make a common-ground grayscale base (usually doubles as height to get your normal map from), then add details for color map and microsurface, some of these details overlap, some not.

Those shots look really good, what application are you using..?

Substance Designer / Painter. But I'm sure other software, like Quixel and whatever else is out there, can get you similar results. You could do it manually too, but doing things like edge detect from the normal map by hand each time is way too much work.


Edited by chedap, 05 September 2018 - 09:57 AM.


#25 RPGista

RPGista

    Advanced Member

  • Member
  • PipPipPip
  • 1590 posts

Posted 05 September 2018 - 10:00 AM

Fantastic information boys, I dont think we had this amount of detail and insight into the technical aspects of texture-making before. Great contributions. Will definitely be coming back for more. One aspect of TDM that always made me a bit sad was that vast surface areas tend to be in "total darkness" at all times and are not subjected to lights at all, apart from the ambient one (unless you bring a light source with you). This way, a player that keeps to the dark is going to be spending a lot of time looking at textures in their worst state - bascily a washed out, darkened and contrast free albedo with weird normals and zero speculars. Pretty charmless. Thats why even the shadow areas should have some sort of gradient, but since we dont have bounced lights, its not really possible. The only way to simulate this that comes to mind is to have a subtle dark fog covering the whole map, this would make areas that are distant from light sources to become increasingly darker. But this has a cost on performance, thats why I didnt use it in my own map (I might revisit that now that I mention it).



Reply to this topic



  



Also tagged with one or more of these keywords: tutorial, textures, materials

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users