Unwanted Shadow Issues

[Springheel]

I thought that I had tracked down the cause of the unwanted shadows we were getting on the edges of tunics and sleeves--I thought the problem was oversmoothing, but apparently not. I've finished tweaking the citywatch, but I still can't get rid of those annoying shadows.

 

What has me perplexed is that they only appear on the md5mesh, not the .lwo. They aren't caused by the shadowmesh peeking through either, because these shots below have the shadowmesh turned off. It's also not caused by a strange normalmap, as the sleeves don't currently have any normalmap applied to them. And yet there are still strange shadows cast where there shouldn't be any (actually, the sleeve material shader is also set to noshadows and noselfshadow).

 

Compare the the .lwo (front) with the md5mesh (back). Not only are there odd shadows but a strange highlighted poly as well.

 

 

 

The problems with the back of the tunic have been around for some time as well, but again, only on the md5mesh version (right). The back of the .lwo is clean.

 

 

 

Anyone care to guess what might cause these shadows, if it isn't the shadowmesh or normal-map? I know D3 is more forgiving with static meshes than it is with md5meshes, but I'm at a loss to guess at what might cause these.

[Dram]

Hmm, it seems that the smoothing angle is set too high, but you say that's not it so it might be that when you export it it automatically welds vertices together (thus causing smoothing issues)?

 

Not sure

[Springheel]

Exporting seems to affect the smoothing values for some reason. If I unweld the verts in those places, the black goes away, but then it looks sharp and crappy. I've managed to find kind of a middle ground for now, by reducing the smoothing value quite a bit.

[Dram]

Exporting seems to affect the smoothing values for some reason. If I unweld the verts in those places, the black goes away, but then it looks sharp and crappy. I've managed to find kind of a middle ground for now, by reducing the smoothing value quite a bit.

 

Well at least it works I guess.

 

Speaking of which, what program are you using/what program do you export from?

 

I've exported from 3ds max multiple times without smoothing issues.

[Springheel]

I've exported from 3ds max multiple times without smoothing issues.

 

I'm using Lightwave, though the same problem existed with the Maya exporter as well. I didn't realize there was a Max to md5mesh exporter.

[Dram]

Very interesting. I'll try exporting a city watch then and see if those issues do come up.

 

Yeah there is der_ton's Max exporter. Very good and I do all the anims in max too.

[Springheel]

It won't help much even if they don't come up, since I don't have Max and don't really want to learn how to use it.

[Dram]

It won't help much even if they don't come up, since I don't have Max and don't really want to learn how to use it.

 

lol, fair enough. Perhaps look for an updated maya or lightwave exporter then?

[oDDity]

To fix that without reducing smoothing or having it appear faceted, you'd just need to select the polys on the inner side of the sleeve and cut and paste them.

You'd have to be careful when weighting it to the skeleton to make sure the two pairs of verts that now exist at each point around the edge of the sleeve and getting exactly the same weight values.

[Springheel]

To fix that without reducing smoothing or having it appear faceted, you'd just need to select the polys on the inner side of the sleeve and cut and paste them.

 

Cut and paste them where? Into a different layer?

[Dram]

Cut and paste them where? Into a different layer?

 

I think he means cut them out, then re-paste them so that they use their own vertices, that way the edges will not get smoothed out. Clever, I'll remember that trick.

[Springheel]

Ah. I thought that happened automatically when you unwelded the verts? I'll give it a try. There's certainly no end to things to learn.

[oDDity]

This smoothing issue only occurs in areas with very sharp angles such as the huge bend from the outer sleeve to the inner sleeve.

When you unweld the verts around the rim you are not only disconnecting the outer sleeve polys from the inner ones, but all of the polys attached to those verts are being disconnected from each other.

You want all the inner polys connected and all the outer ones connected, so you basically have two solid tubes of polys, one on the inside and one on the outside, but not connected to each other.

It sounds like a long explanation, but as I said it 's only a matter of cut and paste on the inner ones.

 

WHat you could do now that you've unwelded all the rim verts is to select all the inner polys and hit 'm' to merge the verts back again and do the same for the outer polys.

Another way is to actually have a transition of polys from the outer to inner sleeve rather than just a sharp bend, so have extra loops of polys creating a rim, but that's really a waste of polys.

[Springheel]

I tried the cutting and pasting method, but no luck. The shadows were still there, which doesn't seem to make any sense. I suppose I could try just deleting the inside polys entirely and see what happens, but I'm quickly losing my motivation to tinker with this any further right now.

 

I'll work on some other stuff and come back to it later. Maybe I can learn something by studying the models where this isn't a problem.

[Springheel]

Some interesting info on D3W, for anyone trying to figure this issue out:

http://www.doom3world.org/phpbb2/viewtopic...=198514#p198514

 

 

If you're gonna use unsmoothedtangents, you really have to render the normal map using that command as well.

 

Exactly. As I mentioned, the normalmap is a result of the highpoly and the lowpoly with its normals, and "unsmoothedtangents" modifies the tangents and normals.

 

The lowpoly model ingame should have the same normals (and tangents) as the lowpoly model that was used to render the normalmap.

 

Thanks alot for that picture, 6th venom, it's a great visual explanation.

 

One has to be aware of the limitations of MD5 files (not limitations of any specific md5 exporter): no vertex normals stored, but calculated by the engine, and no multiple UVs for a vertex, so vertices along a UV seam are SPLIT INTO TWO VERTICES THAT DO NOT SHARE UVs, NORMALS AND TANGENTS. So these limitations should also be artificially enforced on the lowpoly model that you use to render the normalmap, so that the resulting normalmap gives the same result later on when used on the md5 model. The best way to make the normalmap is to use the game engine's "renderbump" command for that, because it does exactly that, it takes care that the normalmap is compatible with the lowpoly model after md5ification.

 

The reason why these limitations (which are actually simplifications and optimizations in regards to performance) on md5 files are possible (they would have been ugly limits in previous-generation engines), is because all normals and tangentspace are simply a reference for the ingame tangentspace normalmapping process to determine the surface normal on each texel. Whatever that reference is, the normalmap is created relative to that reference and then ingame it has to use the same reference and the lighting will look like the lighting on the highpoly model.

 

So to come back to the problem at hand, if remaking the normalmap is really not an option, you would have to make the seams on the lowpoly model in the way they were on the lowpoly model that was used to bake the normalmap. But maybe you can get the highpoly back if it was made by another artist?

 

edit: some general thought, probably interesting for artists: as explained, normalmaps give accurate results if their realtime reference is the same as their bake-time reference (reference here means the lowpoly object and its normals/tangentspace). Realtime-reference is engine-specific, different games may have different ways of determining tangentspace, there is not one single way to handle this but many, especially in cases like discontinuous UV space or mirrored UVs which result in discontinuous tangentspace even when UV space is continuous. When writing my modelviewer I came across those different ways of handling things and sometimes had to take a closer look at Doom3's normalmaps and ingame rendering results to resolve them. That is why normalmapping functionality that is not specific for a game (normalmap bakers in 3d applications or dedicated standalone bakers like ORB) should have some switches and options to be flexible and cater for those ambiguities, and artists should take a look at these options and experiment with them.

I thought I'd check if a community site like iddevnet or the splashdamage forums have any word on this, and voila:

http://community.enemyterritory.com/for ... renderbump

MoP says:

 

Quote:

It looks like you've rendered this normal map from something like 3DS Max or Maya. This probably isn't the best way, since the engine has its own "renderBump" command for generating normalmaps from high-poly models, which creates normals that are perfectly accurate for in-game use.

Normal-maps rendered from an external application will not necessarily look correct when viewed in the game. You might notice some strange shading if you use this in game.

 

Additionally, it's generally not a good idea to have a black background on a normal-map, since at lower mipmap levels, the black can "bleed" over the UV map seams and result in harsh edges and strange lighting when your model is viewed at a distance.

 

 

For anyone using Blender and sometimes having difficulty with the normals or duplicated vertices (like I have ), here is an article with lots of tips:

http://feeblemind.tuxfamily.org/dotclea ... g-problems

 

Also dealing with the subject: http://www.katsbits.com/htm/tutorials/d ... groups.htm

[Springheel]

For future reference: it appears that the md5export process automatically merges the verts in a single mesh that occupy the same space. When there is a sharp edge, like the edge of a tunic or sleeve, this creates those black areas. Cutting and pasting the inside polys *into a separate layer* solves the problem, as the export process will not merge verts on different layers. As Oddity said above, you have to be sure the verts occupying the same space have the same weight value, or else you'll see tearing during animations.

[Baddcog]

yet another difference between Maya and Max exporting

I've never used layers in Max but seperate edges in the same way to avoid smoothing also.

 

I think we could fill a whole Wiki on just the differences in exporting from different programs.

 

Another option would have been to acutally model with a ring of polys around the edge of the sleeve. Of course it would have been at the cost of more polys and probably not worth it due to the location and how small they would've been.

[Springheel]

Another option would have been to acutally model with a ring of polys around the edge of the sleeve.

 

Yeah, that was my next step if the separate layers thing didn't work.