tdm_lg_split on by default?

[Bikerdude]

So tried Bcd again and this time tdm_lg_interleave 1 had a negative impact, dropping the FPS by half.

 

All of the above options made no difference bar the usePortals, but the game was totally unusable. So I used nvidia DSR to run the game at 3360*2100 and that dropped the FPS down to 17-19 from 40 when standing outside the door in the street at the start.

[nbohr1more]

tdm_lg_interleave 1 is the default. So if you set it to 2, it doubles correct?

[Bikerdude]

Nope, 2 dropped it by 10.

[New Horizon]

I finally had some time to test this on my laptop. I had tested the interleave settings years ago and the results now are the same as then unfortunately. My suggestion would be to not mess with tdm_lg_interleave at all. Interleave creates an 'unbalanced load', which causes a type of jerking issue as the processor is hit with a light gem load every second frame (if set to tdm_lg_interleave 2). People with higher end machines may not notice the jerking, but then again they will likely not see much performance improvement from increasing interleave either.

 

tdm_lg_split 1 is fine, as it maintains a balanced load. The gains were modest on my system, and older i3, at around 2 to 5 fps in some scenes. Better than nothing. Ideally, it would be helpful if we could offload some of the lightgem work to a separate core..or some such thing.

 

So yeah, tdm_lg_split 1 is good, tdm_lg_interleave above 1 is bad.

 

Just my two cents.

[Bikerdude]

People with higher end machines may not notice the jerking,

Nope, saw jerking with almost every setting above 0,

[nbohr1more]

In my testing, lg_interleave is pretty smooth if used in conjunction with lg_split at FPS above 40FPS or so.

I can handle the occasional stutter when the FPS drops below that point but I thought it would be convenient to

create an automatic way to toggle it off and on when the FPS threshold is below my comfort level.

 

I was inspired by the new DOOM title to look into this as a step towards other FPS-threshold based optimizations.

(If FPS < X then lower setting Y).

 

One thing that the new DOOM (id tech 6) has that we don't have is a way to predict engine load (lower FPS) before

it happens. I believe this is pre-calculated as a map attribute. We may be able to implement something similar

using entities eg: an fps_bias keyword that can be polled by location entities to tell dynamic effects that "this location

will have x % lower FPS".

[nbohr1more]

Biker:

 

tdm_lg_split = 0 is the default setting. the whole lightgem is calculated for every frame that it is rendered.

 

tdm_lg_split = 1 means that the load is split in half

 

========

 

tdm_lg_interleave = 0 "NO LIGHTGEM CALCULATION"

 

tdm_lg_interleave = 1 is the default setting. Full Lightgem in every frame

 

tdm_lg_interleave = 2 is "Do lightgem this frame, then skip the next, then do lightgem after that, etc" every other frame.

 

tdm_lg_interleave = 4 is "Do lightgem this frame, then skip the next 3 frames and calculate on the fourth".

 

What settings did you test?

[Bikerdude]

let me go through that above list as you have written on both machines and come back to you.

[New Horizon]

The threshold is likely going to be different for people depending on their hardware. My point with interleave though is that it's creating an unbalanced load. The processor is going to be spiking and dropping if anything above 1 is used...unlike tdm_lg_split, which maintains a balanced load. Increasing the interleave also 'decreases' lightgem accuracy. So again, I wouldn't mess with that.

 

If it were possible to take advantage of modern cpu's and their multiple cores, it would go a long way to improving TDM's performance. My laptop is dual core, but idtech4 only uses 1. It's a shame we can't use the whole processor.

[nbohr1more]

Yes parallelism is something to look into eventually.

 

As for frame load? Those are changing all the time. All it takes is a fast moving AI running past, VP opening, or a script trigger event

to drastically alter the frame load from the previous frame. Yes, Lightgem is more drastic than that (usually) but the lg_split arg

eases the switch from no lg to lg pass. This is true of both lg interleave 1 or lg interleave n.

 

Yes, lg interleave reduces lightgem response accuracy so it shouldn't be enabled by default regardless.

It already exists as a performance optimization so I have made an option to make it more useful.

 

What might be a good idea is to have a sorta wave-front model like:

 

Frame 1: Full LG

Frame 2: Half LG

Frame 3: No LG

Frame 4: Half LG

Frame 5: Full LG

 

That would smooth the load transitions, then for higher LG intervals it would be:

 

Frame 1: Full LG

Frame 2: Half LG

Frame 3: Half LG

Frame 4: Half LG

Frame 5: No LG

Frame 6: No LG

Frame 7: Half LG

Frame 8: Half LG

Frame 9: No LG

Frame 10: No LG

Frame 11: Half LG

Frame 12: Half LG

Frame 13: Half LG

Frame 14: Full LG

 

So that the user experience would largely mirror lg_split but you would gain some more of the benefit of lg_interleave.

[New Horizon]

I'm having trouble understanding the benefits of what you outlined above...probably because I only had 3 hours sleep last night. lol I'm just thinking out loud though, doesn't it just make more sense to only do split at all times, since it's balanced, and instead see if there is a way to make the whole light gem system calculations cheaper to perform? I wish I had a more firm grasp on what the whole system does. The gem calculations in T3 don't seem to be as power hungry as our system.

 

From what one of the T3 developers told me, the T3 light system takes a sample from 12 points on the player model...spread out over 12 frames. Each point has a weight to it, and the final result is an average of the weighted 12 points. Something like that...I can dig up the old email.

 

Ahh, here is the relevant bit -

 

 

As you probably remember, Garrett's visibility in T3 was calculated by grabbing light from 12 bone points on the player model (not per frame, but spread out and averaged over several frames) and then running the results through a few modifiers.

 

So even though they're grabbing 12 bone points, it's still just one point per frame....smaller manageable chunks I guess.

[nbohr1more]

Making the Lightgem calculation cheaper is a "huge project". I am doing simple things to make the existing optimizations work better.

Mostly so I can apply this knowledge elsewhere for other optimizations. (Proof of concept.)

 

The scheme I outlined above is better than the current lg_interleave but I guess you and I will have to differ with regard to lg_interleave's usefulness overall.

I didn't create lg_interleave and nobody has petitioned to remove it, so I think it's worthwhile to make it better for those who do find it useful.

 

As for T3, they are probably doing something akin to lg_weak. A simplified math based approach. We could augment our lg_weak code by converting

the light textures into low resolution lookup tables then using that as biasing for the calculation.

 

1) In Image_load.cpp add an additional step to downsize the light textures and feed them to RAM as a lookup table

2) Expose access to this to game_local

3) Poll this via Lightgem.cpp

 

Lot's of coding tasks that are outside my skillset (though I have a rough idea what's involved).

 

Hmm, the image map function in SEED looks like it might work for step 1....

 

int img = gameLocal.m_ImageMapManager->GetImageMap( mapName );

[nbohr1more]

 

 

As you probably remember, Garrett's visibility in T3 was calculated by grabbing light from 12 bone points on the player model (not per frame, but spread out and averaged over several frames) and then running the results through a few modifiers.

 

 

Yeah, a single pixel would be much cheaper to sample than the current setup. I guess you'd just position the camera right in front of the bone for each shot.

(Not my skillset though...)

[New Horizon]

I suspect not using the camera approach at all would be even cheaper, but I have no idea how difficult it would be to flesh out the weak light gem in order to make it equal to the camera based setup.

 

The weak light gem is just a traditional trace from the light to the player. It was the first attempt, but had to be abandoned because Sparhawk didn't have access to the renderer to allow access to different types of lights...which is what lead to using the camera instead, and that's a lot more costly in terms of performance. The original Thief only takes one sample from the players feet and the rest is done with modifiers, however the NewDark releases have added a second trace to the head and averages both traces for the final result, although I don't think it's on by default. I tried it out once and you could now lean into light...or lean into shadow just like TDM.

 

 

I didn't create lg_interleave and nobody has petitioned to remove it, so I think it's worthwhile to make it better for those who do find it useful.

 

Yes, I know. Sparhawk was the one who coded the interleave back in the day. I was helping him test it...oh so long ago. haha

[nbohr1more]

No matter how you implement it, you somehow need to take all the attributes (Shadows, projection textures, etc) that affect that bone and synchronize them.

Using the existing camera setup isn't too much more taxing that implementing a solution without a screen-space transform.

 

Somehow getting the info without sending the verts to the GPU or writing the shadow volume would be a big boost though.

 

I think, however, that rendering such a small ( 1 pixel) viewport would cull most of the geometry overhead too so

it might be comparable to T3 without hooking into the data before GPU submission. Hard to say.

[New Horizon]

During the early light gem testing we tried reducing the render down to 1x1 but it wasn't very accurate. I can't remember the current render size...32x32 perhaps? It should have all geometry cropped out since the captured image is of the top and bottom of the white prism model, taken from the inside of the prism model. I'll poke around this evening to refresh my memory.

[nbohr1more]

Yes. One pixel on the player model, vs one pixel on the lightgem prism, vs one pixel on each "bone" are very different things.

 

I can't see how the 3rd option would be less accurate than T3 but I haven't played T3 that much so I haven't compared

it's lightgem accuracy to TDM.

[New Horizon]

Yeah, I was only talking about testing with 1px on the current system...but over 12 points on the player model it would likely pretty accurate. By the way, are these tests supposed to be done with the SVN build? I was just testing on the release build as I don't have SVN setup anymore.

[nbohr1more]

Yes. The release build is fine. I can get you a compile with the new tdm_lg_interleave_min cvar if you want to try it.

[New Horizon]

Sure, that would be great!

[nbohr1more]

Here ya go:

 

https://dl.dropboxusercontent.com/u/17706561/lg_min_patch.zip

 

Backup or rename your TheDarkMod.exe, gamex86.dll, and tdm_game01.pk4

 

and place these in your darkmod folder.

 

set tdm_lg_split = 1

 

set tdm_lg_interleave = 2

 

set tdm_lg_interleave_min = the lowest FPS you find interleave 2 tolerable or above the stutter zone. (defaults to 30)

[New Horizon]

Tried this out on my laptop tonight.

 

I had to crank up tdm_lg_interleave_min up to 50 to eliminate the stutter. I couldn't detect any real performance increase when interleave kicked in, but tdm_lg_split easily gives me a 3 to 4 fps increase.

 

I also lowered tdm_lg_interleave_min to 30 to see if I could better identify a performance increase and it might have squeezed one more fps but of course the choppiness was reintroduced at the lower setting.

 

tdm_lg_split for the win!

[nbohr1more]

Hmm...

 

I had a thought on the load balancing aspect.

Perhaps the context switch setup itself is part of the pacing issue.

If, instead of skipping shots, we collect single pixels during the NULL interleave frames it might smooth-out the load.

 

Something to tinker with, anyway.