Jump to content
The Dark Mod Forums

Lightgem optimization


duzenko

Recommended Posts

I should have run a few maps with the previous build first, but so far performance is pretty steady in the 50's / 60's on this system. Actually, mostly topping out at 60fps. That's with the enhanced rendering stuff enabled. It dipped down to 30'ish in one demanding view, but otherwise it seems to be doing well. I have been testing with interleave set to 2 and haven't encountered stuttering, but the real test will be on my lap top when I get home from work. If it reduces the load enough to make tdm_lg_interleave 2 usable, that would be fantastic. Fingers crossed.

 

My work computer is:

Intel Core 2 Duo - 3.0ghz E5700

6 GB Ram

AMD Radeon HD 6800

 

Computer at home:

HP 2130 ca laptop

Core i3 330m

4GB Ram

NVIDIA GeForce G105M - 512 MB

Link to comment
Share on other sites

Thanks!

 

We really need someone with a weak AMD CPU and

a reasonably strong GPU to see if gains are greater there.

Still, AMD GPU's are usually more CPU bottlenecked so your

Initial look is promising.

Please visit TDM's IndieDB site and help promote the mod:

 

http://www.indiedb.com/mods/the-dark-mod

 

(Yeah, shameless promotion... but traffic is traffic folks...)

Link to comment
Share on other sites

I do have a weak Intel CPU and a reasonably strong GPU so I'll give this a run.

 

I've no metrics to measure besides com_showfps 1 but with tdm_lg_interleave 2 the game seemed to gain ~4 FPS at poorly performing spots (places with a lot of shadowcasting lights in KoD). tdm_lg_interleave 1 just made it run worse overall.

 

 

 

vvv EDIT: Yes, it's what I used.

Edited by Spooks
  • Like 1

My FMs: The King of Diamonds (2016) | Visit my Mapbook thread sometimes! | Read my tutorial on Image-Based Lighting Workflows for TDM!

 

 

Link to comment
Share on other sites

Did you test with the executable I uploaded a few posts back?

 

I think the animated grass map will probably be the best test case here.

Please visit TDM's IndieDB site and help promote the mod:

 

http://www.indiedb.com/mods/the-dark-mod

 

(Yeah, shameless promotion... but traffic is traffic folks...)

Link to comment
Share on other sites

I have tried it on my home laptop and I was surprised by improvements in some areas. The biggest improvements on this system came from indoor areas. I tried down by the riverside.

 

With the old lightgem calculations, I was getting around 15fps in the opening room on average, but after switching to the new exe, It was hitting 50 to 60 in there. Larger outdoor areas punished the fps. Topside of the ship I hit 9fps, but disabling the lightgem with tdm_lg_interleave 0 didn't raise the fps at all, so it looks like the new lightgem calculation isn't adding too much overhead.

 

I'll have to do some more tests to firm those findings up, but that's my initial observation.

  • Like 1
Link to comment
Share on other sites

Nice!

 

One thing I'll mention is that the tdm_lg_interleave_min setting now has an improved weighted average that is biased towards the

newest frame times. This seems to make the transition from non-interleaved to interleaved much smoother. You can try lowering

the tdm_lg_interleave_min until you hit a discomfort level with frame pacing.

Please visit TDM's IndieDB site and help promote the mod:

 

http://www.indiedb.com/mods/the-dark-mod

 

(Yeah, shameless promotion... but traffic is traffic folks...)

Link to comment
Share on other sites

Hey! So, it turns out I totally forgot about interleave_min. I had it set to 40. I dropped it down to 1 for giggles and I honestly can't say i experienced any serious stutter beyond what would normally happen at rates of 9 or so.

 

So with interleave pretty much active at all times, I am now seeing an improvement in wide open areas in Down by the River. On the top side of the ship, I averaging 13 to 19 fps....sometimes in the 20's.

 

Vast improvement for this old laptop.

 

Do you think there might be any further ways to squeeze a bit more performance out of it?

Link to comment
Share on other sites

I tried last build and in the beginning of thief's den lightgem does not appear to work. It reacts on crouch but that's it. It was working a couple of weeks ago but it was a different mission.

Last svn rev with lightgem working was 6626, and 6627 broke it.

 

BTW it's interesting that you mentioned shadows - after much struggle with my laptop I turned them off and now it's more or less playable in 720p.

I never liked them anyway but it's harder to find a place to hide without them

 

BTW2: I got this stupid idea about doing SwapBuffers before rendering. I suspect all the other opengl calls only make driver to cache the commands and actual gpu work is triggered by swapbuffers. If we could somehow tell the driver to start rendering and without waiting for this frame start calculation for the next one?

So like this:

Now:

- engine renderer frontend does culling and prepares commands for backend (a lot of cpu, gpu idle)

- backend converts them to opengl commands (a lot of cpu due to the opengl overhead, some gpu to copy data etc)

- backend calls swapbuffers and all hell breaks loose. Gpu does all the delayed rendering, driver is waiting for gpu to finish and only then presents the new frame

 

In my dreams:

- frontend does the culling for the new frame

- backend calls swapbuffers (somehow gpu has completed some or all of its work)

- backend converts new frame data to opengl calls

- backend somehow magically tells the driver to start actual rendering but not wait for it - glFlush???

Am I completely detached from this world or is it somehow possible?

Edited by duzenko
  • Like 1
Link to comment
Share on other sites

I tested rev 6627 before committing but I think I need to tweak the logic there anyway.

 

Your dream scenario sounds like Vulkan or DX12 asynchronous shaders.

 

There has been chatter on the RB-DOOM3-BFG git about adding a Vulkan backend but thus far nobody with Vulkan expertise has touched

anything with id Tech 4.

  • Like 1

Please visit TDM's IndieDB site and help promote the mod:

 

http://www.indiedb.com/mods/the-dark-mod

 

(Yeah, shameless promotion... but traffic is traffic folks...)

Link to comment
Share on other sites

I cleaned-up the logic around enabling dynamic lg interleave so that
setting tdm_lg_interleave 0 properly disables all lg processing no matter
what FPS you have (no longer need to set tdm_lg_interleave_min to 1).

As for Thief's Den, there is a window on the right as you first enter the level
that projects light even right below it were you would expect it to be obscured.
This appears to be a map bug. Otherwise lightgem processing is working as expected.

Here is a new build, other than the above I found a better place to cull all lights
outside the viewInsideLight region. I saw perhaps 1fps gain in the tunnel of NHAT
but nothing notable. Your mileage may vary:

https://dl.dropboxusercontent.com/u/17706561/lg_cull.zip

As for my little excursion into shadows?

Failure.

It wasn't a big thing, I was just gonna test out ljblade's "ARB_DEPTH_CLAMP". I tried this
before but it killed all shadows. This time I went back further in the commit log and found
some things I suspected were missing. I was able to get the code to work but it seems the
shadows were artifacted to hell due to driver issues. No real performance gain there either
so I undid the changes. I think Revelator once found a workaround for this but I can't find
his old code.


Edit: Hmm, perhaps I spoke too soon... I seem to be getting up to 10FPS more outside in Behind Closed Doors.
Not bad. (with the light culling)

  • Like 1

Please visit TDM's IndieDB site and help promote the mod:

 

http://www.indiedb.com/mods/the-dark-mod

 

(Yeah, shameless promotion... but traffic is traffic folks...)

Link to comment
Share on other sites

Last (easy) thing to try along this line of optimization.

 

Disabled ambient_world during the lightgem and replace it with shader parms like the simple ambient method:

 

https://dl.dropboxusercontent.com/u/17706561/lg_cull.zip

 

I didn't commit the code yet but if you wish to compile your own:

 

PlayerView.cpp line 868

 


 pAmbientLight->On(); // Turn on ambient light
 pAmbientLight->GetRenderLight()->suppressLightInViewID = DARKMOD_LG_VIEWID;

LightGem.cpp line 227

 


// Give the rv the current ambient light values - Not all of the other values, avoiding fancy effects.
	idVec3 ambient_color = pAmbientLight->spawnArgs.GetVector( "_color" );
	m_Lightgem_rv.shaderParms[2] = ambient_color.x * 1.5f; // Ambient R
	m_Lightgem_rv.shaderParms[3] = ambient_color.y * 1.5f; // Ambient G
	m_Lightgem_rv.shaderParms[4] = ambient_color.z * 1.5f; // Ambient B

As I can tell, that should also make the lightgem more accurate because you aren't double illuminating it

with both the real ambient and the existing parms already in the calculation.

 

Looks good to me anyway.

 

This one is a bit controversial because you can use a real textured light for ambient_world and this will

replace light texture with a flat value. The same thing happens if you use the simple ambient so I guess it's

not a huge deal but I'm thinking a toggle cvar might be the ticket.

 

Performance? I think I saw 1 more FPS in NHAT cave and wasn't able to detect further improvements elsewhere.

 

Testers?

Please visit TDM's IndieDB site and help promote the mod:

 

http://www.indiedb.com/mods/the-dark-mod

 

(Yeah, shameless promotion... but traffic is traffic folks...)

Link to comment
Share on other sites

Weird. I don't seem to be getting the improvements I saw before. Still using down by the riverside. I'm hitting maybe 30 or 40 inside now and only around 8 or 9 fps on the deck of the ship. Also seeing more stutter with tdm_lg_interleave 2 than before...I didn't notice any on some of the previous builds. Not sure what might be different.

Link to comment
Share on other sites

Alright, I'll roll that one back and I think we're done on this project.

 

Edit:

 

Here's the rollback:

 

https://dl.dropboxusercontent.com/u/17706561/lg_cull.zip

Please visit TDM's IndieDB site and help promote the mod:

 

http://www.indiedb.com/mods/the-dark-mod

 

(Yeah, shameless promotion... but traffic is traffic folks...)

Link to comment
Share on other sites

Do you think there might be any further ways to squeeze a bit more performance out of it?

Disable shadows?

Reduce screen resolution?

What does com_speeds show?

How much is GPU utilized, in %?

This particular scene on the deck of the ship could use r_skipfoglights, r_skiptranslucent

Edited by duzenko
Link to comment
Share on other sites

Rendering to a memory buffer then reading from the CPU is an expensive task.

 

The solution is to make an abstraction (table) of the light textures that stay resident in RAM and use

that rather than a scene render. We do something similar in the SEED system where we input images

and turn them into tables to bias entity placement. I'll look at this when I get a chance but it's probably

above my skill level to address.

 

We were recently discussing this topic in another thread.

 

Another possible option would be to collect single pixel samples at random player bone

coordinates and average them every 12 frames the way that T3 does.

 

Hard to say if that will be good enough for our player base.

Just aimless speculation until I or someone more skilled than myself takes a stab at it.

 

Of course, anything that reduces the CPU impact of the renderer will directly improve the lightgem performance as well.

We have some ideas on the roadmap to improve things there.

 

Due to frame rate variability, a time based update system would be more consistent I'm guessing. Like every 500 ms.

 

EDIT: lol sorry didn't realize how old this thread was, disregard. ;P

I always assumed I'd taste like boot leather.

 

Link to comment
Share on other sites

  • 2 weeks later...

Now there is one last step to do and we're done with it for a while.
I have prepared a patch to use a Pixel Pack Buffer, OpenGL 2.1 style.
It reduces the performance impact even further, better utilizes both CPU and GPU.
Framerates with and without lightgem should be roughly the same in low-fps situations.
It requires an OpenGL 2.1 class hardware, which is 2007 or later.

Diff Patch

 

Index: game/LightGem.cpp
===================================================================
--- game/LightGem.cpp	(revision 6634)
+++ game/LightGem.cpp	(working copy)
@@ -429,16 +429,16 @@
 
 			// The order is RGB. NOTE: Serp - I have moved the scale into the rgb multiplier
 			// this is to move the calculation into the precompiler (could move to .h directly)
-			m_fColVal[in] += (buffer[0] * (DARKMOD_LG_RED  * DARKMOD_LG_SCALE) +
-							  buffer[1] * (DARKMOD_LG_GREEN* DARKMOD_LG_SCALE) +
-							  buffer[2] * (DARKMOD_LG_BLUE * DARKMOD_LG_SCALE));
+			m_fColVal[in] += (buffer[0] * (DARKMOD_LG_RED  ) +
+							  buffer[1] * (DARKMOD_LG_GREEN) +
+							  buffer[2] * (DARKMOD_LG_BLUE ));
 		}
 	}
 
 	// Calculate the average for each value
 	// Could be moved to the return
-	m_fColVal[0] *= DARKMOD_LG_TRIRATIO;
-	m_fColVal[1] *= DARKMOD_LG_TRIRATIO;
-	m_fColVal[2] *= DARKMOD_LG_TRIRATIO;
-	m_fColVal[3] *= DARKMOD_LG_TRIRATIO;
+	m_fColVal[0] *= DARKMOD_LG_TRIRATIO * DARKMOD_LG_SCALE;
+	m_fColVal[1] *= DARKMOD_LG_TRIRATIO * DARKMOD_LG_SCALE;
+	m_fColVal[2] *= DARKMOD_LG_TRIRATIO * DARKMOD_LG_SCALE;
+	m_fColVal[3] *= DARKMOD_LG_TRIRATIO * DARKMOD_LG_SCALE;
 }
Index: renderer/RenderSystem.cpp
===================================================================
--- renderer/RenderSystem.cpp	(revision 6634)
+++ renderer/RenderSystem.cpp	(working copy)
@@ -293,6 +293,7 @@
 =============
 */
 idRenderSystemLocal::~idRenderSystemLocal( void ) {
+	qglDeleteBuffersARB(1, &pbo);
 }
 
 /*
@@ -1011,7 +1012,27 @@
 
 	qglReadBuffer( GL_BACK );
 
-	qglReadPixels(rc->x, rc->y, rc->width, rc->height, GL_RGB, GL_UNSIGNED_BYTE, buffer);
+	if (1) {
+		static int nbytes = 64 * 64 * 3;
+		if (!pbo) {
+			qglGenBuffersARB(1, &pbo);
+			qglBindBufferARB(GL_PIXEL_PACK_BUFFER, pbo);
+			qglBufferDataARB(GL_PIXEL_PACK_BUFFER, nbytes, NULL, GL_STREAM_READ);
+			qglBindBufferARB(GL_PIXEL_PACK_BUFFER, 0);
+		}
+		qglBindBufferARB(GL_PIXEL_PACK_BUFFER, pbo);
+		unsigned char* ptr = (unsigned char*)qglMapBufferARB(GL_PIXEL_PACK_BUFFER, GL_READ_ONLY);
+		if (ptr) {
+			memcpy(buffer, ptr, nbytes);
+			qglUnmapBufferARB(GL_PIXEL_PACK_BUFFER);
+		}
+		else {
+			// handle error
+		}
+		qglReadPixels(rc->x, rc->y, rc->width, rc->height, GL_RGB, GL_UNSIGNED_BYTE, 0);
+		qglBindBufferARB(GL_PIXEL_PACK_BUFFER, 0);
+	} else
+		qglReadPixels(rc->x, rc->y, rc->width, rc->height, GL_RGB, GL_UNSIGNED_BYTE, buffer);
 }
 
 /*
Index: renderer/tr_local.h
===================================================================
--- renderer/tr_local.h	(revision 6634)
+++ renderer/tr_local.h	(working copy)
@@ -696,6 +696,9 @@
 ** by the frontend.
 */
 class idRenderSystemLocal : public idRenderSystem {
+private:
+	GLuint pbo;
+
 public:
 	// external functions
 	virtual void			Init( void );

 


Screenshots (Intel HD5500)


Untitled.png
Untitled2.png


Binaries download link for testing (make sure you have the new materials file from post#44)

 

Edited by duzenko
  • Like 4
Link to comment
Share on other sites

This is AMAZING!!!

 

NHAT cave never goes below 20FPS now!

 

 

I was in a feeble project to attempt to move the lightgem camera inside the lg model

to take better advantage of culling but this... this is a miracle!

  • Like 4

Please visit TDM's IndieDB site and help promote the mod:

 

http://www.indiedb.com/mods/the-dark-mod

 

(Yeah, shameless promotion... but traffic is traffic folks...)

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Recent Status Updates

    • Ansome

      Finally got my PC back from the shop after my SSD got corrupted a week ago and damaged my motherboard. Scary stuff, but thank goodness it happened right after two months of FM development instead of wiping all my work before I could release it. New SSD, repaired Motherboard and BIOS, and we're ready to start working on my second FM with some added version control in the cloud just to be safe!
      · 0 replies
    • Petike the Taffer  »  DeTeEff

      I've updated the articles for your FMs and your author category at the wiki. Your newer nickname (DeTeEff) now comes first, and the one in parentheses is your older nickname (Fieldmedic). Just to avoid confusing people who played your FMs years ago and remember your older nickname. I've added a wiki article for your latest FM, Who Watches the Watcher?, as part of my current updating efforts. Unless I overlooked something, you have five different FMs so far.
      · 0 replies
    • Petike the Taffer

      I've finally managed to log in to The Dark Mod Wiki. I'm back in the saddle and before the holidays start in full, I'll be adding a few new FM articles and doing other updates. Written in Stone is already done.
      · 4 replies
    • nbohr1more

      TDM 15th Anniversary Contest is now active! Please declare your participation: https://forums.thedarkmod.com/index.php?/topic/22413-the-dark-mod-15th-anniversary-contest-entry-thread/
       
      · 0 replies
    • JackFarmer

      @TheUnbeholden
      You cannot receive PMs. Could you please be so kind and check your mailbox if it is full (or maybe you switched off the function)?
      · 1 reply
×
×
  • Create New...