Lighting issues when using a lot of entities

[motorsep]

There aren't really any idTech 4+ resources left where knowledgeable people hangout, so I am asking in TDM Support forum, sorry (and maybe in the future TDM would migrate to Storm Engine 2 anyway )

 

 

I have two rooms, ~56k x 42k units, with ~3200 func_statics (each has 4 LOD meshes, the lod1 is ~2.5k tris, lod2 is ~1.2k tris, lod3 is ~800 tris and lod4 is ~80 tris) both rooms. I also split the room with visportals so it's faster to compile. 2 lights - ambient and parallel (in each room). Rooms are connected with a zigzag tunnel with visportals inside.

 

If I have it as-is, I get what you see on the screenshot - the sharp boundaries of the light is where visportal planes are. The bright green surfaces are fine, the dark green are corrupted ones.

 

If I have only one room (same setup, just twice less entities), I get no issues at all.

 

If I have two room, but replace func_statics with regular non-LOD models, I get no corrupted surfaces on meshes and more area on the map lit. As if there is some limit on total polycount and/or on number of surfaces per map.

 

Does anyone (most likely SteveL, greebo, OrbWeaver, Tels and revelator; sorry if I missed someone - just have't seen anyone else deeply discussing / digging the rendering engine) know why is this happening and how to fix that ?

 

EDIT: video

Edited November 28, 2014 by motorsep

[SteveL]

Hmm there's a lot of information to absorb there.

 

The lit and unlit areas of ground... you say the boundaries are where the visportals cut through? Does the light volume exactly match the VP boundaries, or does it cover the whole plane?

 

It looks like the corrupted enities are self-shadowing, or maybe not picking up the light interaction in the first place. Impossible to say which exactly. Even if it's a shadowing problem, that doesn't necessarily mean your shadow technique is the problem. We see shadow problems with large parallel light covering multiple visleafs too, although in our case the problems can be made to go away by making sure the light is no more than 1 visleaf away from any visleaf where you want to see correct shadow, which cuts out the optimizations that cause the problem. Can you rep that pattern? If you put you light in a corner, does the problem move to the far corner?

 

The list of light interactions to be drawn is a linked list I think, so there's no theoretical limit to the number, but you might be pushing new bounadaries with the size of the vbo cache. Speculating here as it's not a problem we've run into afaik. There are a couple of cvars for getting some diagnostics there: r_showVertexCache and the command listVertexCache. If you move toward the corner where the corrupted entities are so that you reduce the number in view, is there a point where they suddenly correct themselves? I suspect not as you were able to get close to a messed up entity without many in view and it still looked bad.

[OrbWeaver]

I also split the room with visportals so it's faster to compile

 

We see shadow problems with large parallel light covering multiple visleafs too, although in our case the problems can be made to go away by making sure the light is no more than 1 visleaf away from any visleaf where you want to see correct shadow, which cuts out the optimizations that cause the problem.

 

This would be my Random Guess as to the source of the problem. Doom 3 visportals are designed for culling parts of the map that cannot be viewed from the current location; they are not a map compile-time optimisation, nor are they designed for dividing up a large open area which is spanned by large light volumes.

 

I suspect the map compilation and render code was simply never designed to handle the situation you are describing here, and the use of such large always-open visportals may well give a performance hit because of the constant screen-visibility testing that is happening for each visportal (to decide if it should close or not).

[SteveL]

Do you recreate your shadow maps when you switch lod levels? That would mean redoing them every frame in a spot like this, but if not that would probably cause problems...The only entities correctly shadowed would be those at the same lod stage as when the shadow map got made.

 

On the other hand if the problems do all go away when you remove the second room, then it's not a shadow problem or a vp problem. It's probably not even a vbo problem. Can you try the same test, with the two rooms, but with one of VPs in the tunnel blocked by a door or a func portal? That will help narrow it down. The renderer front end treats VPs that are forced shut differently from ones that are merely closed because you can't see them. Areas that are beyond VPs that are forced shut are not added to the "connected areas" array and any shadowcasting lights in them won't be looked at at all. Areas behind visportals that are merely unseen can still contribute light interactions.

 

Also please post an overhead orthoview of the map showing light light volumes and visportals. Will save a lot of questions!

[SteveL]

Ps if your game doesn't use func portals or doors to force VPs shut, then just brick up one of the VPs for the test.

[motorsep]

Hmm there's a lot of information to absorb there. The lit and unlit areas of ground... you say the boundaries are where the visportals cut through?

 

Yes, the sharp boundaries on the floor are exactly where visportal planes are.

 

My lighting setup is simple - one large ambient light, and slightly offset from it one parallel light. 2 lights per "room", so to speak. Moving parallel light doesn't change the issue at all, unfortunately.

 

Despite how it seems to be in theory, visportals cutting up map was advised by Brian, nontheless. Empirically, having no visportals on the open map vs having them reduces map compiling time significantly. It also doesn't cause any artifacts or issues or performance effect with regular polycounts. The only reason I am having this issue is due to massive amount of tris and surfaces in the scene. Note that the first level shown on the video is exactly the same setup as the second level. The only difference is that instead of 1 room, there are 2 rooms (I copied original room with the same setup, including visportals, and connected the rooms). So to summarize, there is no performance hit at all whether I have visportals or not.

 

Shadows have no effect on artifacts - I can load map with shadow volumes, or without shadows at all - it will still be corrupted / half-lit. Shadow maps, just like shadow volumes, are created for LOD1 iteration and don't change when LOD iterations are switched. Shadow mapping has its own LOD system. I also tried having noSelfShadow flag, and that didn't do anything either. As a matter of fact, if I replace LOD entities with standard func_statics, I will have exactly the same issues. The only way to remove artifacts is to reduce total polycount of the scene. Even if I have small rooms cut off from each other with vis portals (with doors or S-tunnels), there issue is still the same - total polycount / surface number matters.

 

func_portals? Doom 3 doesn't have that as I recall. However, in S-shaped tunnel you don't need func_portals (whatever they do). I can see with r_showPortals 1 that the corridor between 2 rooms is shut closed as portals are drawn red.

 

The video I linked in the first post shows layouts for both maps, one- and two-room setup. I can upload map too, just it won't have any textures / models.

[SteveL]

I agree some of the suggestions above are ruled out by your 1 room vs 2 room test. But do do the vp test... As I said above, not all "closed" visportals are the same. There are two ways a vp can close, and the "closed" VPs in your tunnel do not stop the parallel light in each room from generating interactions in the other room. Just try swapping one tunnel vp for a solid wall, which will have the same effect as closing it with a door: it'll prevent the light in room 2 creating interactions in room 1 and vice versa.

I did see the orthoview in your video. The reason I wanted another screenshot is connected to the above: to see whether the lights in each room overlap the other room. I can only see that if you turn on the "show all light volumes" toggle in DR. Or you could just tell me of course :-)

[motorsep]

Here is my top-down layout:

 

 

I'll try blocking tunnel with solid brushes, and also will try moving lights so they won't overlap.

 

Fun fact - I tried my 1 room test with RBDoom 3 BFG build without shadow maps and with shadow maps and it has exactly the same issue (although a bit worse than our engine).

[motorsep]

Found interesting detail - if I keep only one parallel light, I don't have the issue with brushes not being lit. The problem is that I can't stretch one sunlight to light up both areas. Might have to use a hack - script that moves sun light to a new outdoor area :/

 

Blocking tunnel with brush doesn't help at all - one room becomes pitch black.

[SteveL]

Hmm you are getting conflicting results, which always seems to happen where parallel lights are concerned. I did take a look for limits but couldn't find any. Did you check the vbo cache cvars by the way?

 

It feels like this is pointing to parallel lights though, with interactions being culled inappropriately. Whatever the core problem, that's almost certainly happening and it'll make for confusing results.

 

I'm going to throw out a load of points for checking/discussion. I'm casting around a bit in the dark (no pun intended) because I've not got a test case but none of these are random or idle , they're all based on previous bugfixes I've worked on.

1. Does the problem go away if you temporarily switch to shadowcasting omni lights? (and does your shadowmapping technique support omnis ok by the way?)
   
2. What happens if you position the parallel lights so that the light center is only just inside the map (that's required else it'll leak), but the light origin is outside the map? Given the direction of your lights, that'd mean moving them to the upper right corner of the map in orthoview, then enlarging them so they cover the area again.
   
3. It's possible to flood a huge map with a single parallel light, no matter how complex the exterior, but some extra plumbing is required to stop interactions getting culled. Not that I'm recommending that complex solution, but it could make a good test.
   
4. Your original screenshot is definitely showing light interaction loss at visportal boundaries. It's a mystery why changing the number of entities would make the problem go away, but I can see from the shadow direction that we're looking at the lower left visleaf in one of the rooms, and the shadow cuts across at the 7th row, exactly where the VP is.
   

I do (2) in all my maps with big parallel lights by the way. I regularly see missing shadows or light interactions behind the origin, which are cured by making sure there's nothing behind the origin except empty void.

 

This next point probably won't fix the problem but it's good general advice and it'll be interesting to see how it changes the shadow pattern: If you're going to cut up rooms for dmap performance, you should do it with parallel VPs not criss-crossing VPs. In other words, slice the room like a loaf of bread ||||| instead of like this |‡|

 

It'll generate a few longer tris from worldspawn, but so few that it's irrelevant. The reason to do it that way is because the renderer front end (specifically, the hierarchical view frustum culling in r_FlowViewThroughPortals) doesn't much like being able to see into a distant visleaf through multiple different VP combinations at the same time. It ends up checking the entire visleaf for interactions once for every possible path. Standing in your map, there are 3 ways to see the opposite corner. And in the worst spots, where the player is standing right on the long vp plane, it'll think there are many more ways to see each of the furthest visleafs because of epsilon allowances. Slicing the rooms using parallel VPs will cut all of that out. And if the errors move, that'll be another clue.

[motorsep]

Well, there are two issues happening - interactions with brushes (culling of lights at the visportal boundaries), and interactions with meshes (partially lit triangles, non-lit triangles). So, there aren't conflicting results per se

I also tried the same cases using RBDoom 3 BFG before shadow mapping was added, and after. Same issues. So that ruled out us potentially messing up renderer with our implementations.

 

 

1. I tried it and had same issues. I might have forgotten to reset light center, so I will try again. Shadow mapping supports point lights well and I still have shadow volumes working, so I usually test both ways, and the results are identical.

 

2. It's actually other way around - origin needs to be inside, otherwise map leaks. Light center can be outside, and here is the result:

I think that zfighting is the result of my ceiling skybrush casting shadow, since the light direction goes from light center to the origin.

 

3. That looks interesting and as I dig into it I will definitely have a lot of questions

 

I am testing that new visportal setup you recommended ( |||| a la slices of bread). Will report as soon as I have it compiled.

 

EDIT: It seems that there can only be one parallel light per map. Although I might be wrong.

 

@SteveL: I see you are using portal sky in the Moonlight map. What if I don't use portalsky at all? Will it make any difference ? (not to mention portalsky in idTech 4 is massive hack that chops down performance)

Edited December 1, 2014 by motorsep

[motorsep]

A small breakthrough - apparently number of interaction surfaces / tris is limited per area. I split each of the "rooms" with 3 visportals ( ||| - in such manner) and I no longer have corrupted model surfaces on my 3k+ of entities . However, the issue with light not passing through those portals persists (and solution with tubes isn't acceptable as areas get connected as if there are no portals, and I am back to square one with corrupted mesh surfaces).

 

I wonder if there is a way to keep portals closed when the maps is being loaded, and then "open" them so that the light floods the map (this way interactions are created without corrupting meshes).

[SteveL]

I'm not sure how the number of interactions is coming into play but like I said I'll take a look after the weekend to see whether I can shed any more light on it (pun intended this time). You can cure the tubes method by adding a single vp at the end of each tube like I did in that moonlight map. If your light is in the main map instead of a separate room, leave the vp off that one only. The idea is you configure it so there's no more than 1 vp between the light and anywhere you need to see light and shadow from that light. Not an elegant method, I know.

 

Do closed doors keep VPs closed in your game too? You could try your idea by placing closed doors on the VPs then spawning your entities then deleting the doors after. Hmm not sure they take effect during spawn time now I come to think of it, even if you make them spawn first. And if the player view isn't yet active, I'm not sure there's any difference between a closed and open vp.

[motorsep]

Bam! Solved!

 

 

All I had to do was making light dynamic.

 

So what I came up with, empirically, is this - apparently there is a limit on how many interactions/interactive surfaces can be per leaf. If there are too many surfaces (tris) per leaf, only certain number of surfaces (or even tris) get lit. Maybe that's why dmap merges inlined meshes into one surface (based on material). A leaf can be huge and has only few surfaces and it will work well. It can be small and choke full of tris - corrupted surfaces will show anyway.

 

So I am basically chopping tree into more leafs, distributing surfaces between them. Each leaf gets only so many interaction surfaces and as a result it works well.

Edited December 4, 2014 by motorsep

[SteveL]

Ah that's interesting. Let me get this straight: your sun isn't constantly moving, you just needed to move it away and back again after game start to fix the bugs?

 

If so it's likely the problem was in dmapping or the proc file. Dmap generates shadow volumes for each light and writes them as surfaces to the proc file. It also tries to optimize and merge them. It'll keep those precalculated shadows unless you move the light origin in which case it dumps them after the light move.

I don't know whether you changed that setup in any way for your alternative shadow methods, but the results would make sense if it's pre-calc'd shadows at fault.

 

I'll have to test whether this trick fixes any or all of our traditional problems with parallel lights. They are the only kind of shadowcasting light that you'd normally want to stretch across several visleafs.

[Lux]

What's also interesting about this is its a vast outdoor area with no walls and it runs well.

 

@motorsep, did you just put visportal cubes around groups of objects to break up the area invisibly? Would be nice if you showed the same video and enabled show triangles.

[motorsep]

Ah that's interesting. Let me get this straight: your sun isn't constantly moving, you just needed to move it away and back again after game start to fix the bugs?

 

Right. When I press the button, it jumps instantaneously into another "room", just like the one spawn in. The idea is that I can put a trigger into the tunnel connecting the "rooms" and as I pass through, sun jumps and it would feel like I have one massive sunlight that covers both "rooms". One of the reason I did that is that I am working on dynamic day/night system, and the whole system will be bound to a root entity that will jump between areas (having parallel light per area will make it too complicated and I don't really want to make pipes :/ )

 

If so it's likely the problem was in dmapping or the proc file. Dmap generates shadow volumes for each light and writes them as surfaces to the proc file. It also tries to optimize and merge them. It'll keep those precalculated shadows unless you move the light origin in which case it dumps them after the light move.

I don't know whether you changed that setup in any way for your alternative shadow methods, but the results would make sense if it's pre-calc'd shadows at fault.

 

No, dmap has nothing to do with it. If light is static, it's flow is restricted by portals, to my understanding. As soon as light moves, portals stop blocking the flow. So I am thinking that due to visportals, that are on the open, static lighting breaks. I can manually delete shadow model from the .proc file and the same issue will still be present on the map, meaning with dynamic shadows light flow will be limited.

We have both shadow volumes and shadow mapping, but they are mutually exclusive. So when I run with shadow maps, all shadows are fully dynamic. Prelit shadows aren't used/rendered. I forgot to test it with shadow volumes, but I have a feeling no one will run the game with them

[motorsep]

What's also interesting about this is its a vast outdoor area with no walls and it runs well.

 

@motorsep, did you just put visportal cubes around groups of objects to break up the area invisibly? Would be nice if you showed the same video and enabled show triangles.

 

It runs well because it's modified Doom 3 BFG engine, not idTech 4, and also because LOD is working, and because shadows are shadow maps, and not volumes, so there is no extra geometry to calculate/render. I have parallel visportals to split the "room" into 4 areas (set up diagonally / / / ) , but no cubes around entities.

 

I can make that video with r_showTris, if I don't forget

[SteveL]

Ok, thanks for the test results. I have the parallel light+ visportal issue on my tracker list, so I'll be coming back to this thread when I pick it up.

[Lux]

I see. I got excited and forgot you were using BFG. Thanks for the reply.

[motorsep]

Tested it with shadow volumes - works the same way as with shadow maps.

 

I see. I got excited and forgot you were using BFG. Thanks for the reply.

 

Well, it's a good reason to be exited anyway I still don't get why there is a notion that BFG vs idTech4 is like CryEngine vs idTech 4. When it comes to game code, it's almost identical to idTech 4. So I am certain that TMD can be ported, little by little to BFG game code. By BFG I mean Storm Engine 2, since it's a way ahead of BFG in terms of bug fixes, modding capacity and such.