Vertex Colors break shading

[OrbWeaver]

2 minutes ago, Ralle321 said:

Btw, i forgot to mention DR doesn't show vertex colors, neither in unlit mode, lighting mode or in the model viewer.

DR will not show vertex colors in fullbright preview mode, but it should should vertex colors in lighting mode if the material has an appropriate vertex color stage. This is certainly working for me when I am testing vertex color export from the ASE and LWO exporters.

If this isn't working for you, could you please confirm which version of DR you are using and if possible provide an example model along with its MTR file which demonstrates the problem?

[peter_spy]

Btw. I updated TDM to the latest beta, and I can reproduce this; sometimes the green model triangles are slightly faceted, sometimes they are not.

[Ralle321]

@OrbWeaver: DR version is 2.8.0 x64 and you can check out my updated testmap: https://mega.nz/file/hBgl2byQ#MdVn2JVIhGRPuuPtXjSDOLZdY9Ix2ohQAA_0zHYIOh0

[peter_spy]

Btw. what color did you use for the vertex color in the model?

Edit: oh I see it, you can use r_showVertexColor to check it. I only know examples of greyscale colors used to blend between textures, although according to original id docs it should just act like a multiply function with any color you like.

Edited June 17, 2020 by peter_spy

[Ralle321]

For the cylinder its red: 0.545, green: 1.0 and blue: 0.22

[OrbWeaver]

diffusemap       textures/darkmod/vertexcolors_break_shading/placeholder2

{
	blend	specularmap
	map 	textures/darkmod/vertexcolors_break_shading/placeholder2
	vertexColor
}

For some reason your material is only blending vertex colors with the specular map, which is why you don't see it in the renderer. If you want to see the vertex colors you need to blend them with the diffuse map, e.g.

{
	blend diffusemap
	map       textures/darkmod/vertexcolors_break_shading/placeholder2
	vertexColor
}

{
	blend	specularmap
	map 	textures/darkmod/vertexcolors_break_shading/placeholder2
	vertexColor
}

If I change this I can then see the vertex colors, although for some reason DarkRadiant does not want to load the "placeholder" DDS file so the wall and floor textures are missing.

[Ralle321]

Thanks, it works now in the lighting preview mode.

I wrongly assumed blending with the diffusemap was unnecessary, because it doesn't show vertex colors in TDM.

The "placeholder" dds file is uncompressed, that's probably the reason why it doesn't load in DR.

[OrbWeaver]

12 hours ago, Ralle321 said:

I wrongly assumed blending with the diffusemap was unnecessary, because it doesn't show vertex colors in TDM.

Really? That sounds like a bug. It certainly should be possible to modulate the diffusemap with vertexColor, and the result should look similar in game to how it does in the DR renderer. Modulating the specular map is certainly possible, but this should only affect the color of the specular highlight, not the overall color of the model which is determined by the diffusemap stage.

Quote

The "placeholder" dds file is uncompressed, that's probably the reason why it doesn't load in DR.

Actually that is useful information, thanks. The error is from OpenGL when we try to call glCompressedTexImage2D, which returns an invalid format. I thought maybe this was because there was a problem with my graphics drivers not supporting the required DXT formats, but if the data is actually uncompressed and we are trying to call the function to upload a compressed image, this might well be the cause of the problem.

[peter_spy]

The engine always supplies the "system textures" in absence of diffuse/normal/specular textures, so should get a black texture, which multiplied by model's vertex color should give black, I guess.

[STiFU]

18 hours ago, Ralle321 said:

Thanks, it works now in the lighting preview mode.

I wrongly assumed blending with the diffusemap was unnecessary, because it doesn't show vertex colors in TDM.

The "placeholder" dds file is uncompressed, that's probably the reason why it doesn't load in DR.

So does this also resolve your shading issue ingame?

[Ralle321]

Quote

Really? That sounds like a bug. It certainly should be possible to modulate the diffusemap with vertexColor, and the result should look similar in game to how it does in the DR renderer. Modulating the specular map is certainly possible, but this should only affect the color of the specular highlight, not the overall color of the model which is determined by the diffusemap stage.

Yeah, i found it odd too, because usually vertex colors are blended with the diffusemap, yet here in tdm it works only with the specularmap.

@STiFU: No, it's still faulty ingame.