Lightgem optimization

[duzenko]

This must have been discussed before but here it is

I am playing on a skylake core m5

There is a huge difference in fps between software and rendered lightgem calculation

I am talking 30 and 20 fps

I know about the split frame trick but it is not working well for me

The main issue that I see here is low gpu utilization

Software lightgem - 30 fps - 90+% gpu usage

Rendered lightgem - 20 fps - ~60% gpu usage

I can't help but conclude that during the lightgem rendering phase most of the time nothing is actually being rendered

Can someone revise the code for hardware calculation and see if it can be improved?

Or at least additional configuration options added?

Regret software lightgem is practically unusable or else I would not be returning to this now

Edited September 18, 2016 by duzenko

[New Horizon]

By software lightgem are you referring to tdm_lg_weak? If so, that was the original attempt at implementing a lightgem back in 2005,but there was no SDK access to the game renderer back then. It would be great to have something that was less performance intensive.

[nbohr1more]

Rendering to a memory buffer then reading from the CPU is an expensive task.

 

The solution is to make an abstraction (table) of the light textures that stay resident in RAM and use

that rather than a scene render. We do something similar in the SEED system where we input images

and turn them into tables to bias entity placement. I'll look at this when I get a chance but it's probably

above my skill level to address.

 

We were recently discussing this topic in another thread.

 

Another possible option would be to collect single pixel samples at random player bone

coordinates and average them every 12 frames the way that T3 does.

 

Hard to say if that will be good enough for our player base.

Just aimless speculation until I or someone more skilled than myself takes a stab at it.

 

Of course, anything that reduces the CPU impact of the renderer will directly improve the lightgem performance as well.

We have some ideas on the roadmap to improve things there.

[New Horizon]

As long as the end result is the same and performance improves then folks will be pleased I think.

[duzenko]

What I can gather from tdm sources is that lightgem render does essentially same work done by main renderer

That is, visible geometry and lights are calculated twice each frame

If we could reuse results of main rendering work for lightgem it should help a ton

[New Horizon]

The light gem system 'should' be functioning like this...but I'm not coder, so I may be way off base but this is how it was explained to me when I helped test and tweak the settings years ago.

 

 

We have a diamond shaped 3D model that represents the player. We place a camera inside the model and the camera takes a snapshot of the top and bottom of the diamond each frame, unless tdm_lg_split is set to 1, in that case it takes the top one frame and the bottom the next frame. The snap shot is cropped to only render the inside of the diamond. The render snap shot is small, something like 128 x 128 pixels, or 64 x 64. It is analyzed and averaged between the two shots. I don't recall the exact numbers. Here is a picture to illustrate, the red triangle represents the camera inside the 3d model.

 

If you're saying this could be done with less overhead, I'm all for that.

[nbohr1more]

The lightgem shot is from a different viewpoint, has a different resolution and FOV and has a simplified player model representation.

If we somehow reuse the data, it will be a much larger and complex data-set for the CPU to parse.

 

I suppose that the newest GPU's have multi-viewport acceleration tech but then the optimization would only be applicable for them...

 

The main reason for the duplicate work is that the light projection texel manipulation is all done on the GPU so we have no a priori knowledge

about it's final projection state without the feedback loop.

 

Another option would be to analyze the texel samples on the GPU directly via a "compute shader" and feedback to the Game code.

Similarly to the above, we'd have to cutoff legacy GPU support (though this one is a little more reasonable)

but we'd also need to update our entire framework to be OpenGL 4.x compliant to gain access to compute shaders.

(Plus we'd need someone who knows how to write compute shaders...)

[duzenko]

My research shows that the actual analysis of the lightgem screenshot is fast enough on cpu so no use spending effort on that.

I understand that camera projection is different for lightgem rendering but I suspect that actual geometry being rendered is largely the same, because camera position is very close. And I'm almost sure that lights culling result is the same.

If we can't reuse main renderer data for lightgem then another workaround could be doing lightgem in a thread on a separate rendering context - if only d3 engine supported that.

 

What about the subviews feature? The engine appears to have some support of mirrors and cameras so that can be utilised, right?

Edited September 20, 2016 by duzenko

[New Horizon]

I understand that camera projection is different for lightgem rendering but I suspect that actual geometry being rendered is largely the same, because camera position is very close. And I'm almost sure that lights culling result is the same.

 

Just so I'm sure we're on the same page.

 

If you look in the lower left corner of this screen shot, I enabled the lightgem render debug hud as indicated by the red arrow. The camera that captures the lightgem render is inside the diamond model, so all it sees is the interior of the model. That little square in the bottom left corner should be all the camera for the light gem render sees, it shouldn't be capturing any other geometry.

[nbohr1more]

We might be able to enable multiple threads. Carmack discovered that there is a API bug in Windows OpenGL that prevents

spawning new render threads but he was able to find a few ways around the bug with RAGE and discussed the options. We could

restructure the GL initialize to be spawned after the other code has been spawned and threaded. Definitely on the to-do list.

 

As for subviews? I believe the lightgem is one, no:

 

lighgem.h

 

renderView_t m_Lightgem_rv;

 

Though, portalsky and mirror do seem less expensive, I wonder if it would be worth using something akin to

our skybox code:

 

renderView_t portalView = hackedView;

 

Ultimately, I'm still pretty sure this is the problematic part:

 

renderSystem->CaptureRenderToBuffer(m_LightgemImgBuffer);

[nbohr1more]

 

Just so I'm sure we're on the same page.

 

If you look in the lower left corner of this screen shot, I enabled the lightgem render debug hud as indicated by the red arrow. The camera that captures the lightgem render is inside the diamond model, so all it sees is the interior of the model. That little square in the bottom left corner should be all the camera for the light gem render sees, it shouldn't be capturing any other geometry.

gem_render.jpg

 

Well, the final capture looks like that but scene geometry must be present or shadows wouldn't affect the lightgem

 

I may be mistaken but I do think the renderview excludes noshadows geometry though. (or it should...)

[New Horizon]

Ahhh.

[nbohr1more]

I think I am mistaken. NoShadows surfaces are rendered because the lightgem model itself is NoShadows.

 

If that's the case, perhaps we can give the lightgem model some new surface attribute to check for suppression...

then knock out all those surfaces that have no effect on the lightgem...

[New Horizon]

Right, it wouldn't cast shadows..lol..of course. I used to know all this stuff. haha Yeah, eliminating stuff that has no effect on the light gem would likely help.

[duzenko]

No, I'm pretty sure that the slow part in lightgem is this

gameRenderWorld->RenderScene(&m_Lightgem_rv);

Because it takes a lot of cpu time to go through the entire scene and do all the usual rendering calculation. It's not just my guess, this is what tests say.

Actual "backend" rendering is not "blazing fast" too but it definitely is only a fraction of time because virtually all geometry is culled by frontend.

As for multithreading it's Carmack's own words that eventually he ended up with backend rendering on main thread and software preprocessing (frontend) in background as opposed to his failed attempts in D3 to call OpenGL api in background. My own limited experience is the same - at least on Windows it only makes sense to have opengl calls and user input in main thread while everything else can safely go to threads.

In our case it means that potentially we could move renderer frontend (gameRenderWorld->RenderScene) to a thread but still need the backend (R_IssueRenderCommands) remain on main thread.

 

With all that said, for me personally the multithreading way, while probably most exciting from a programmer point of view, is least promising in term of end result. My own problem is low fps on a ULV processor, so simply rearranging the load in parallel will result in more throttling, lower cpu clock speed and same fps.

My problem can be solved by doing less work, so again I urge the game programmers to find a way to not do rendering preprocessing separately for main render and lightgem.

And on "normal" computers the framerate will be still limited to 60fps due to the engine internal cap.

Edited September 20, 2016 by duzenko

[nbohr1more]

I've already mentally outlined the process to reduce drawing.

 

1) Create new material flag MF_LIGHTGEM

2) Create new cull type CT_LIGHTGEM

3) Assign CT_LIGHTGEM to surfaces that do not fall under the radius of any shadow casting lights

4) suppress all surfaces with MF_NOSHADOWS and MF_NOSELFSHADOWS or CT_LIGHTGEM

5) Submit all other surfaces and MF_LIGHTGEM to the renderview.

 

Another possible area for improvement is culling all lights that don't intersect with the lightgem

If a light's radius isn't overlapping the lightgem, neither light or shadow from it will affect the result.

Hard to say if that would be worthwhile as the engine does cull lights when not in view and the cropped

view shouldn't expose too many but there are probably border cases where small light coverage in

view causes wasteful rendering.

 

Edit:

 

ai_see conditional:

 

 

 
if ( !args->GetBool( "ai_see", "1") ) // SteveL #4128
    {
        renderLight->suppressLightInViewID = DARKMOD_LG_VIEWID;
    }
 

 

reuse lg_weak code to identify light intersects:

 

 

 
if (distance > light->m_MaxLightRadius)
        {
            DM_LOG(LC_LIGHT, LT_DEBUG)LOGSTRING("%s is outside distance: %f/%f\r", light->name.c_str(), light->m_MaxLightRadius, distance);
            if(h == -1)
                h = i;
            continue;
}
 

[nbohr1more]

I created tracker 4379 to investigate this further.

[New Horizon]

Nice.

[duzenko]

Exciting!

[nbohr1more]

Having a bit of a time figuring out how to match-up renderlight to vlight.

 

After getting a little frustrated, I found out that tr_light is also checking whether a view is "inside" a light

for shadow calculations. So another redundancy can be reduced I think and we shouldn't need to add logic

to lightgem.cpp.

 
if ( (!vLight->viewInsideLight) && tr.viewDef->renderView.viewID = -1)
            {
             *ptr = vLight->next;
              light->viewCount = -1;
              continue;
            }
 

[New Horizon]

Was that tr_light checking for a view inside the light something we added? Rings a bell, but I might be thinking of something else. I thought we had a problem with the view clipping inside lights at one point, which caused incorrect readings making the lightgem go full bright sometimes...or maybe that was geometry. lol So long ago I can remember. Probably not an issue anymore since the camera is taking the snap shots from inside the diamond model.

[nbohr1more]

ViewInsideLight is there for the shadow volume code. If you are "inside" a light, then "Carmack's Reverse" is used, otherwise standard depth-pass

shadows are used. Since being inside the light is also what we are interested in, it should be a two-in-one win.

 

I was able to compile this change but saw no performance benefit. I suspect that with the lightgem camera inside the lightgem prism that the non-intersecting

lights are already being culled or that the viewport is so small that the viewport culling is doing 99% culling of the work anyway.

 

I'll do a little more validation and testing but I think it may be best to move on to culling noshadows surfaces. tr_light might also be a quicker route to that end.

[New Horizon]

Cool. Best of luck sir!

 

As soon as you have anything for me to test, just toss a line.

[nbohr1more]

Will try to return to this soon.

 

"suppressSurfaceInViewID" can be found in RenderWorld_portals.cpp with a check similar to the one in tr_light.cpp.

 

there is also some more complex checking for it under interaction.cpp, we can try a condition check at the top of

idInteraction::CreateInteraction

 

eg.

 

if (shader->Coverage() != MC_LIGHTGEM && ( !HasShadows() ) && (tr.viewDef->renderView.viewID == -1))

{

continue;

}

[nbohr1more]

Here's a compile. Place the contents of the archive into your darkmod directory:

 

https://www.dropbox.com/s/nrl5hkheik1s0yk/lg_cull.zip?dl=0

 

I didn't see any massive performance gains but the game certainly felt smoother.

 

I didn't have time to analyze my CPU usage.

 

Let me know if this improves things.