Jump to content
The Dark Mod Forums
Sign in to follow this  
joebarnin

Light count issue

Recommended Posts

I have a performance issue with an FM that working on. The FPS is dropping pretty low in some areas. I narrowed it down to the fact that my light counts are high (r_showlightcount 1). I'm confused why certain lights are contributing to the light count. To help understand this issue, I just created a simple test case, which consists of:

  1. Main room with a light.
  2. Hallway connected to main room
  3. Stairway down connected to the hallway
  4. Basement room (with a light) connected to the stairway. The basement room is directly below the main room.

There is a visportal and door between the hallway and the stairway. When this door is closed the light count in the main room is 1. When it is open, the light count is two.

I hope this video makes it clear: 

This puzzles me - how can the basement light affect the light count in the main room? The light shouldn't be going through the floor/ceiling between the two rooms, right? Why does opening the door make a difference? The door is beyond the range of the light anyway.

Additional data point: It doesn't matter what type of light I use in the basement, the behavior is the same. Any light based on atdm:light_base shows this issue. But, I replace the basement light with an object of class "light" (the thing you get if you do "Create light..." in DR), then the light count stays at one even if the door is open. In fact, I was able to create a light that looks and acts just like the atdm:cagelight. I went through the class hierarchy, starting with atdm:cagelight, then what it inherits (atdm:static_electric_light_unlit_base), then what that one inherits, and so on, and I grabbed all of the spawnargs from each. I ended up with this:

{
"classname" "light"
"name" "light_2"
"AIUse" "AIUSE_LIGHTSOURCE"
"_color" "0.44 0.34 0.17"
"editor_SetKeyValue s_shader" "light_flicker_104"
"editor_displayFolder" "Lights/Model Lights, Static/Switchable/Electric/Outdoor/DoNotUse"
"editor_light" "1"
"editor_setKeyValue light_center" "0 0 -8"
"editor_setKeyValue model" "models/mapobjects/lights/cagelight/cagelight.ase"
"editor_usage" "A lit wall-mounted, grill light. Replacement for Doom3 cagelight. Can be toggled on/off when linked from a switch."
"extinguished" "0"
"lightType" "AIUSE_LIGHTTYPE_ELECTRIC"
"light_center" "0 0 -8"
"light_radius" "260 260 260"
"model" "models/mapobjects/lights/cagelight/cagelight.ase"
"nodiffuse" "0"
"noshadows" "0"
"noshadows_lit" "1"
"nospecular" "0"
"origin" "-848 -136 -736"
"s_looping" "1"
"s_shader" "light_flicker_104"
"s_waitfortrigger" "0"
"scriptobject" "tdm_light_holder"
"shouldBeOn" "0"
"skin" "lights/cagelight"
"skin_lit" "lights/cagelight"
"skin_unlit" "lights/cagelight_unlit"
"sr_class_1" "S"
"sr_radius_1" "500"
"sr_state_1" "1"
"sr_time_interval_1" "977"
"sr_type_1" "STIM_VISUAL"
"texture" "lights/tdm_lanternlight_4fold_small_snd"
}

Which is equivalent to atdm:cagelight in both looks and function. The difference is, this light doesn't add to the light count. I suppose I could use this to work around the issue, but man is that kluge-a-rific.

Anyway, any ideas on what is happening here? The test map is attached.

testl2.zip

Share this post


Link to post
Share on other sites

This is due to the fact how visportals work. As soon as a visportal is open, the stuff behind it gets pre-rendered, so it can be displayed as soon as the player has line of sight to it. For performance assume that everything that is in all areas that are connected by open visportals is rendered, because it may come into view of the player. I would recomend the Visportals Wiki page. There you can find some more info.

You can easily test this, when you create one visportal on the top and one of the bottom of your staircase into the cellar. When you are sufficiently far away in the main room, the lower visportal should be closed and only the upper one (that is in line of sight of the player) should be open. When you get closer and the lower part also come into line of sight, the lower visportal gets opened and you should have the increase in light count that you described.

  • Thanks 1

Share this post


Link to post
Share on other sites

Thanks. But I'm still confused. Here is a screen shot of the scenario you described. I'm in the main room, looking toward the hallway and stairway. r_showportals indicates that the upper visportals are open (green) and the lower one is closed (red):

turn_2020-04-07_09_18_36.thumb.jpg.7b681863f45ac8e1813ab4d2cacff014.jpg

And yet, r_showlightcount indicates two lights in the main room:

turn_2020-04-07_09_18_45.thumb.jpg.a9386a32734bc036dc8b10da9a6672c8.jpg

If the lower visportal is closed, shouldn't the light count be one? Or am I misunderstanding?

Share this post


Link to post
Share on other sites

Where is the radius of the light?

You cannot treat the light source\center as the enclosed component.

For a portal to enclose a light, it's entire radius must be entirely enclosed 2 portals out of view.

Try decreasing the radius and increasing the brightness.

Add a few more portals so that the count adds to 2 quicker.

Use a "hide distance" on the light to force the entity to disappear when x units away.

  • Thanks 1

Please visit TDM's IndieDB site and help promote the mod:

 

http://www.indiedb.com/mods/the-dark-mod

 

(Yeah, shameless promotion... but traffic is traffic folks...)

Share this post


Link to post
Share on other sites

I reduced the basement light radius so that it is entirely contained within the basement (at least, in the X and Y directions; in the Z direction it protrudes into the room above). If I go back upstairs, the floor shows two light sources. So the basement light is going "through" the floor, but only if the door is open? I don't get it.

From a level design point-of-view, I can shrink the light radius so that it fits the room (XY) but I can't shrink the Z direction so that it doesn't go through the ceiling/floor. 

Also, isn't it strange that I see different behavior in these two cases:

1) visportal is closed (red) because it is out of view (beyond a closed visportal). In other words, I open the door, then go back into the main room and go around the corner so that the door visportal is out of view. In this case the lightcount is still 2.

2) visportal is closed (red) because its door is closed - lightcount is 1

Also, why does this behavior not happen if I use a generic "light" object?

Share this post


Link to post
Share on other sites

If you already know this then dismiss but if not, be mindful that TDM engine (idtech 4) lights are totally real time and "interaction" based, meaning, any object/surface that interacts with a light (light radius touches a polygon of that object/surface) will get fully rendered and if a surface "interacts" with more than one light, it will be re-rendered for how many lights "touch" its surface, for example if you have a single 15000 poly mesh and it is interacting with 3 lights, it will be rendered 3x each frame, 1x per-light, even if only one tiny polygon is really lighted, because of this, is best to use many (within reason) but small radius lights then few but large lights. Having said that, there's nothing wrong with using big lights, if you use one or two the scene is not that complex and you are careful that not many lights cast shadows.   

About casting shadows, only "hero" lights, those important for the overall look of the scene should cast shadows, all other "fill" lights should not cast any shadows.

Like others said, make sure the light radius is totally within a single "vis-leaf", meaning a "room" that is seperated from others by one or more vis-portals.

Some tips for idtech 4 lighting taken from here, not sure if all useful for TDM:

Quote

Post subject: Lighting Tips!

 

I figured I would compile a small and simple list of Editing tips that will help you guys navigate the editor. Please feel free to add new ones you discover as well.

The light color scale is as follows:

Type r_showlightcount 1 into the console, when you are in game. You will see the following colors:

1. Black= No light hitting the surface.

2. Red= 1 light hitting the surface (this is excellent).

3. Green= 2 lights hitting a surface (this is great).

4. Blue = 3 lights hitting a surface (This is getting bad. You need to keep these areas small, but it is ok to have a number of smaller areas in a room that are blue).

5. Cyan, pink, or white= 4+ lights hitting a surface this is bad and you should only have this in very small, isolated areas. If you have a large face with any of these colors your lighting is inefficient and you need to either split the brush face according to the light volumes or reduce the overlap.

NOTE: You can get large brush faces that will show up as white or cyan even when there are no actual light volumes overlapping one another in the area, This is because a large brush face will be in contact with multiple light volumes, which is just as bad as having multiple light volumes overlapping. To fix this you can split the large brush face up between the volumes that are making contact with the face. So instead of having one large face with 4-5 lights contacting it you can have 3 brushes with only one or two light volumes hitting the brush faces. This works well only if you can keep the engine from merging your splits back together. There are a number of ways to do this, but I am only going to cover a simple decal method here. In the example above one brush was split into 3. In order to keep all three brushes from merging again, select the center brush of the three and turn it into a caulk brush. Make a simple patch mesh and fit it to be coplanar and exactly the same size as the visible face of the caulk brush. Texture it to fit the textures around it and you now have 3 brushes that cannot be merged back together by the engine in place of one large one. Your light colors should drop from Cyan/white to green and red.

This is the thermometer you have to aid in your journey to a playable level.

r_showprimitives 1 will give you the r_speeds. You should try and keep most areas in the 45-60k region. 70 k is also fine, but if you are making mp maps I would lean towards a max of 65k for any given view. This is pretty subjective because one can simply make a point that their level needs a little bit more machine bulk in their readme if they want to go a little higher end, but this of course limits your download crowd.

com_showFPS 1 will show the Frames Per Second in the top right hand corner of your screen.

Tech. Bits:

One other thing to keep in mind is that large brush faces that have more than one light volume touching them will show a color that registers all of the light volumes in contact with the face. In other words, If you have a brush that makes up a large wall and there are 5 light volumes touching a single face of that wall in various sections (with absolutely no light volume overlap between the 5 lights) the brush face will still show up a pink/white. To avoid this you need to make sure that brushes are not extremely large and are split up accordingly. (explanation above)

One other engine specific trait is that if you just simply split a brush to reduce the amount of light volumes in contact with the face you may lose your split when the map is compiled. Try splitting brushes along the edge of a texture boundary. If the engine finds manual splits across a texture's width (not along its edge) then the engine will say..."Hey both brushes share the same texture... I am going to merge them.." This is a nice feature that reduces extraneous polys during compile, but makes it more challenging to split faces.

Editing Lights:

1. To edit the properties of a light, select the light and hit the j key. This will bring up the light editing window. You can set the light as projected, or make the light origin available for manipulation, change the color and the saturation (brightness) of a light, change the light volume size, and change the light texture in this menu.

2. To edit the light origin you must select the light and hit the j key to bring up the light menu. In the light menu you click the center box and hit apply at the bottom of the menu. Now you have to unselect and reselect the light to actually drag the light origin around.

3. The brightness slider does not work at all so don't waste your time with it. The saturation slider at the far right of the color window is how you change the brightness of a light. You can also choose a texture that dims the light as well.

Brush Logic:

Just so people start levels with efficiency in mind I highly recommend you study the texture sets well. The days of Q3 detail-a-wall-with-masses-of-brushes to fill space are gone! Texture stacking is far superior in look to the brushwork typical in Quake 3. Why is this
?...Because brushes pulled out away from the wall create shadow volumes that are expensive and typically look less impressive in this engine (they cast really harsh shadows that require more light volumes to soften up). I am not saying you cannot break up your walls for detail, but I am saying create a nice balance between the two.

This also means stay away from the tri-soup method of terrain and lean towards modeling terrain sections out in Lightwave or some other modeling program of your choice. The surface has nicer poly spread and less harsh edges than tri-souped or gensurfed terrain chunks. The lighting will look much nicer on a good model mesh.

Question Session (Brush Logic):
 

Quote:

Am I understanding this correctly if you mean that one should try to keep the number of brushes down but at the same time not use too large brush faces cos the lightning is so demanding?




I should have stated it a little better. You do not need to try and keep the overall brush count of a level down. You should try splitting flat walls and allow texture stacking (different textures that fit together) to give you a nice shell of detail, and try to lower the amount of brushes that are pulled out, away, or sunken into the wall along the way. The problem associated with the Q3 style of building is that there are too many harsh shadow volumes that are both expensive and ugly. Try building structures and then making a second pass to see how much can be mere stacking as apposed to brushwork giving the shape. By no means do you have to avoid detail, but just try and use the light and the textures to do a lot of work for you to cut down on shadow volumes. Curves are also brilliant in this engine and they don't LOD. You can hit the "S" Key with any patch and select the Subdivisions box to reduce the patch subdivisions on the fly. This will make a big difference for you and you will notice that curves that are almost diamonds will look like nice cylinders in game in many cases.
 

Quote:

Is this engine gonna make it difficult, in your opinion, to create larger/outdoors areas because of this




Not if you keep all of this in mind. Outdoor areas are very do-able, but the amount of lights have to be reduced to keep the color grid in the red or green in most areas. The other thing I recommend is the use of a modeling program for terrain, so that the mesh will have a nice controlled arrangement of poly's and will light with less shadow volumes than a jagged and non-uniform wad of polys. It is very possible 

Lloyd Morris's Tips:

Not sure if this has been covered yet but one quick way to create a light of the size you want without guessing/calculating values is to create a brush of the size light you want and then right-click and select light entity to convert it to a light.

Also, I haven't played with the original d3 editor (the one which ships with the game) since finishing up d3mp a few months back, so some of these may have changed, but these are some of the useful commands I used while editing:

r_showportals 1 shows the portals working (green is open and the area behind is drawn) red is closed but the area in front is drawn and the area behind isn't.

r_showtris 1,2 or 3 show triangles drawn by the engine. 1 shows tris visible but culls tris behind other tris. 2 is probably most like q3 showtris. 3 shows everything not cut by the engines pvs.

Also while running the game from the editor I used the editlights command. This shows you where lights are ingame and allows you to edit them realtime from within the game.


Modellers should bind the following commands to keys (eg f1 - f4)

bind f1 toggle r_skipdiffuse 0 1 2
bind f2 toggle r_skipspecular 0 1
bind f3 toggle r_skipbump 0 1
bind f4 toggle r_shadows 0 1

They're pretty self explanatory.

 

 

Edited by HMart
wrote "1x per-light" before was perhaps suggesting that it would render 3x per light, that is wrong.
  • Thanks 1

Share this post


Link to post
Share on other sites

HMart - thanks for all of the info. I'm sure my lighting "hygiene" is not as good as it should be. I'll use the advice I've gotten to clean things up. I still think there is something puzzling happening, but using the techniques you and other have mentioned I should be able to minimize the light count.

Share this post


Link to post
Share on other sites

Yeah this happens to me to. Basically the light bounding box "bleeds" through walls which will increase the light and shadow count in adjacent rooms. 

The light is turning off in your example because that visleaf or portal area is being closed and those objects are kicked out of the renderer. 

Try using  "r_showShadows" "2" which will show shadows being cast from lights. (Keep in mind this doesnt work with shadow maps turned on). You also might need to put an object on the floor to see if the basement light is bleeding through your floor. 

In general the engine likes small light volumes kept within the boundary of the room, but its not a rule 😀

Edited by kingsal
  • Thanks 1

Share this post


Link to post
Share on other sites

Be very careful with the years-old advice from Doom3World/idDevNet. There is some absolute garbage floating around on there, including the myth that there is some kind of "limit" of 3 lights hitting a single surface (there isn't), that having a light count of 4 or more causes a massive performance drop (it doesn't), and that you should manually lower light counts by cutting up your brushes (you can if you really want to but it's a waste of time and will likely make no difference to your performance).

Some of that advice might have been useful once but is no longer relevant on modern hardware. Some of it (as far as I can tell) was just made up out of thin air, like the "3 light limit". Some of it was based on a misunderstanding of how the engine works, such as the advice to cut up brushes which completely ignores the effect of light scissoring (which effectively does this for you at render time).

  • Like 2

Share this post


Link to post
Share on other sites

Where does it says that there's a limit of 3 lights per surface? I don't see it but If that is there is indeed wrong. There's no limit but what your target hardware can support.

But now that you talked about a limit, in your opinion, what is the max limit for how many lights should interact with a surface? Taking into account the minimum GPU level you are willing to support? IMO this should be something the TDM engine team could seriously suggest the mission makers to take into account, of course not has a hard requirement.

 

About the advice I posted, I didn't claimed any of it was accurate or useful to anyone making TDM missions, it was just something I knew existed so decided to post it, imo is better to have autodated info than no info at all. 

About if the advice is relevant today, you are right, today GPU's have evolved compared to when Doom 3 came out, so you can get away with way more geometry, lights, shaders/materials, etc and many TDM missions have and run fine on most modern PC's, this is also not "laws of physics" so you can totally ignore the advice but I would suggest, if you want to make your mission run good to a large amount of people, even those with decade old GPUs, you really should be careful with lighting.  But is not like I want all TDM missions to look from 2004 or something, is a balance that the developer needs to work out.

Btw, I vaguely remember a idSoftware developer (don't know is name now) talking about how levels with too many white surfaces, when using "r_showlightcount" was a no no, so the old Doom 3 modding community being cautious about that, was not that surprising. But again, today the limit is higher, so the svar needs to be updated to show white, with a bigger light count than before, this so you do not limit the mission artists of today when is not needed.  

And last imo If you know, more or less how the idtech 4 lighting system works, at the basic level, even if using outdated advice (performance wise) imo is not bad per-se, is a starting point, you just need to test and see how much more you can get with.

 

 

Share this post


Link to post
Share on other sites
On 4/8/2020 at 10:58 PM, HMart said:

Where does it says that there's a limit of 3 lights per surface? I don't see it but If that is there is indeed wrong. There's no limit but what your target hardware can support.

I was referring to D3W advice in general, not the specific text you posted. I've seen several forum posts on D3W and I think even these forums where people have repeated the "fact" that Doom 3 has a problem with more than 3 lights hitting a surface, and advised mappers to cut up brushes to make sure they never see a cyan surface in the r_showLightCount view, even though this was a waste of time that made no difference to frame rates back when I was doing it on a Radeon 9800 XT in 2006.

I suspect that the "3 light limit" comes from the descriptive text in the explanation of r_showLightCount:

1. Black= No light hitting the surface.
2. Red= 1 light hitting the surface (this is excellent).
3. Green= 2 lights hitting a surface (this is great).
4. Blue = 3 lights hitting a surface (This is getting bad. You need to keep these areas small, but it is ok to have a number of smaller areas in a room that are blue).
5. Cyan, pink, or white= 4+ lights hitting a surface this is bad and you should only have this in very small, isolated areas.

It would be easy to interpret these descriptions as implying that something bad happens between 2 lights ("this is great") and 3 lights ("this is getting bad"), even no there is no such limit and these textual descriptions were (as far as I can tell) just off-the-cuff phrases that never actually reflected any empirical performance testing.

Quote

But now that you talked about a limit, in your opinion, what is the max limit for how many lights should interact with a surface? Taking into account the minimum GPU level you are willing to support? IMO this should be something the TDM engine team could seriously suggest the mission makers to take into account, of course not has a hard requirement.

There is no such limit for maximum lights, because light count is just one of many factors influencing performance, along with poly count, shadow count, shadow mesh complexity, number of shadow-casting lights, number and size of open/closed visportals and many other things. Giving suggested limits for particular scene elements is counter productive because these limits tend to get enshrined in documents and forum posts and propagated through time as a sort of "cargo cult", where people believe that as long as they have fewer than a certain number of lights or polygons then their scene will be fast to render, even though there may be many cases where having more lights/polygons would be fine if other elements of the scene are simpler, and vice versa.

The only meaningful way to address performance is based on empirical testing, with guided optimisations based on this data. For example, if you have a scene that is particularly slow to render, and you find that you have a large surface hit by 20 different lights and they all cast shadows, this might be the thing to optimised, whereas in another scene you might find that you have 10 million polygons in view due to a reflection surface and too many open visportals, and these are the things you need to fix. But in no case does this mean that if you stick to 19 different lights and only 9 million polygons, you will never have a problem with performance.

Quote

About the advice I posted, I didn't claimed any of it was accurate or useful to anyone making TDM missions, it was just something I knew existed so decided to post it, imo is better to have autodated info than no info at all. 

No need to get defensive; I wasn't criticising you, just pointing out that some of that old advice from D3W and other forums isn't particularly accurate and shouldn't be relied upon by today's mappers, particularly when it asserts particular numeric limits that quickly become obsolete due to the evolution of hardware, and might not have been correct to begin with.

Quote

Btw, I vaguely remember a idSoftware developer (don't know is name now) talking about how levels with too many white surfaces, when using "r_showlightcount" was a no no,

Nothing wrong with looking at light counts in conjunction with other items (but you must also consider the area of the over-lit polygons, not just the count, and consider the effect of light scissoring which considerably restricts the actual area that is illuminated based on the size of the light, even if that light hits a large polygon), provided you don't become attached to certain fixed "ideal light counts" which are just too crude a measure to be useful.

Overall my advice with light count and everything else (polygons, shadows etc) is:

  • Minimise everything as far as you possibly can without compromising the artistic intent.
  • Use in-game diagnostics and statistics as a guide, without being hung up on particular numeric thresholds and values.
  • Test on a variety of hardware (especially mediocre hardware) if you can, to avoid producing a map which only runs well on your super high-end rig.
  • Like 2

Share this post


Link to post
Share on other sites
51 minutes ago, OrbWeaver said:

There is no such limit for maximum lights, because light count is just one of many factors influencing performance, along with poly count, shadow count, shadow mesh complexity, number of shadow-casting lights, number and size of open/closed visportals and many other things. Giving suggested limits for particular scene elements is counter productive because these limits tend to get enshrined in documents and forum posts and propagated through time as a sort of "cargo cult", where people believe that as long as they have fewer than a certain number of lights or polygons then their scene will be fast to render, even though there may be many cases where having more lights/polygons would be fine if other elements of the scene are simpler, and vice versa.

The only meaningful way to address performance is based on empirical testing, with guided optimisations based on this data. For example, if you have a scene that is particularly slow to render, and you find that you have a large surface hit by 20 different lights and they all cast shadows, this might be the thing to optimised, whereas in another scene you might find that you have 10 million polygons in view due to a reflection surface and too many open visportals, and these are the things you need to fix. But in no case does this mean that if you stick to 19 different lights and only 9 million polygons, you will never have a problem with performance.

Yes completely agree, you are totally right there I just didn't thought that through. 

And if I came of has defensive that was not my intention at all that particular phrase was more a disclaimer than anything, but reading it again I get why it can feel like i'm being defensive.  Is the problem of text not being that good at passing states of mind.

  • Like 1

Share this post


Link to post
Share on other sites

Also worth mentioning: if you do think that light count might be an issue, there is (or at least was) an argument to dmap which caused the compiler to automatically split brushes at light boundaries, effect paying for reduced light counts by increasing the number of polygons. You might want to try this option and see if there is any noticeable difference in FPS in your problematic scene.

  • Thanks 1

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Sign in to follow this  

×
×
  • Create New...