Issues with directional or huge lights

[stgatilov]

I have bumped into an old issue about glitches with directional lights: 3818. At the same time, I have recently fixed a bug in interaction culling, which happens when light source illuminates many portals areas at once. If we are lucky, that bug could be the only one

It would be great to check it on some test map. But creating a testmap from scratch would be a horrible waste of time. I wonder if someone had experimented with parallel lights and vast outdoors areas and saw shadows randomly disappearing. If yes could you please share your test map?

[stgatilov]

Dragofer reported a problem with shadows, but it turned out to be an entirely different problem (5106).

Basically, TDM 2.07 introduced a horrible bug to stencil shadows. Most likely, it only affects precomputed stencil shadows for projected/directional lights. Also, it does not affect mappers having too old CPUs. If you dmapped anything with TDM 2.07 with projected/directional lights, it might be worth to re-dmap it with TDM 2.08 when it comes out.

 

P.S. Still, I would be happy to see some testmap with (global) parallel light broken

[duzenko]

Good catch on that epsilon bug. Sorry I mis-ported it from SSE.

[stgatilov]

1 hour ago, duzenko said:

Good catch on that epsilon bug. Sorry I mis-ported it from SSE.

Bugs... they happen from time to time I recall how I reviewed this code and missed it. Perhaps should add a random test on this function (i.e. compare to results of generic implementation).

Anyway, I have recalled where I saw a big outdoors map with huge light, I have found it in my installation. It was Noble Cave by AluminumHaste. Unfortunately, it has not portals outdoors, maybe because adding them broke shadows. Calling @AluminumHaste...

[Dragofer]

Thanks again for sorting this out, I was wondering what was going on there.

Btw, In Remembrance of Him by RPGista is well-known for the whole map being bathed in faint purple parallel moonlight. I don't recall seeing anything wrong when I played it, but you know what you're looking for.

[grayman]

I remember trying moonlight in the mission "Braeden Church", and it looked terrible.

I'll add a moonlight light and make a pk4 available this afternoon.

The architecture is pretty complex, with lots of outside visportals, so it should be an ok testbed.

[grayman]

Well, that didn't go well.

I added a bright parallel moonlight light to Braeden Church, and my initial test runs showed that it fails with 2.08 (before your changes), only lighting up the visportaled section where the light lives.

So I wrote up instructions to include the new map for testing, but when I ran through the instructions to make sure I had it right, the moonlight decided to start working.

The changed map works correctly in 2.06, 2.07, and 2.08. Go figure.

So I'll have to rethink this.

The only thing I can think of is that my failed tests included a new dmapping of the moonlight version, and my instructions only included adding the map file but not the proc, cm, or aas files. I can think of no reason why this would matter, but I'm going to have to revisit this. Back later.

[grayman]

Ok, when I include the dmapped files of the "moonlight" version, it now fails. (???????)

Anyway, do the following:

1. Download "Braeden Church" using the in-game downloader, then install it.

2. Go to fms/braeden_church and unzip this file there. You should now have fms/braeden_church/maps/*.

3. Start the mission.

The failure is that the only part of the map that gets painted with moonlight is the "section" (visportal-bound) containing the moonlight light entity. (The light is named "Moonlight" if you open the map in DR.)

=======================================================

Stretch task:

You can try the "it works when you don't include the dmapped files" issue (??????) by deleting everything in fms/braeden_church/maps except the *.map file. You can try this and see if the light suddenly starts working.

[AluminumHaste]

Yes, portals wreak havoc with parallel lights, I gave up on those lights long ago.

Also, the outdoor portion of that mission which was in a valley with a lot of trees didn't have visportals due to them not doing anything. It's all open, no way to close off the portals.

 

EDIT: I'll modify the map once TDM installs.

[HMart]

6 hours ago, AluminumHaste said:

Yes, portals wreak havoc with parallel lights, I gave up on those lights long ago.

Also, the outdoor portion of that mission which was in a valley with a lot of trees didn't have visportals due to them not doing anything. It's all open, no way to close off the portals.

 

EDIT: I'll modify the map once TDM installs.

yes portals for outdoor scenes are really not ideal, that is why Cryengine and such, use manual placed and automatic occluders instead, those engines also use the terrain as occlusion geometry plus techniques like GPU based occlusion culling https://bazhenovc.github.io/blog/post/gpu-driven-occlusion-culling-slides-lif/ . Something to consider to the future of TDM engine? 

Edited January 6, 2020 by HMart

[AluminumHaste]

Cryengine doesn't use bsp, very different

[stgatilov]

15 hours ago, grayman said:

The only thing I can think of is that my failed tests included a new dmapping of the moonlight version, and my instructions only included adding the map file but not the proc, cm, or aas files. I can think of no reason why this would matter, but I'm going to have to revisit this. Back later.

It depends on .proc file. If you use the old one, there is no "prelight" in the .proc file. As the result, it works. If you dmap it, then prelight version gets added to the .proc file. The engine sees that and chooses a different code path in one place, which results in incomplete lighting. I'll look into it.

[AluminumHaste]

I can no longer DMAP this file, I left it going all night and TDM just sits there not responding with the menu music playing.
Tried with a completely new download of 2.07 and SVN.

Someone else wanna give it a shot?

https://drive.google.com/open?id=1pbgamfWZ-Rcda7rlqd-0SVLW2Md3rvlW

[HMart]

14 hours ago, AluminumHaste said:

Cryengine doesn't use bsp, very different

I know that but bsp is not something that idtech 4 can use exclusively, i'm sure you know that you can create a big "room" around a totally mesh based level and it will render fine, of course it will run like garbage, because the engine will not occlude anything, but that is where LOD and occluders could come in handy.

Also not saying this should be done, is perfect or anything, it will have problems with AI AAS for example, just saying that if someone inclined wanted to modify the engine to support better "outside missions" it is not totally impossible.

[VanishedOne]

SteveL was investigating occluders, though I think they were only intended to supplement the visportal system. (The native id Tech 4 approach to open terrain is of course megatextures, but you know what the catches are with those...)

[HMart]

On 2/22/2020 at 10:11 PM, VanishedOne said:

SteveL was investigating occluders, though I think they were only intended to supplement the visportal system. (The native id Tech 4 approach to open terrain is of course megatextures, but you know what the catches are with those...)

Petty that he went away then. Btw I took a look at the Quake Wars SDK code and they mention occluders and (i assume) GPU occlusion queries in the code, so at least one version of the idtech 4 engine has this stuff, petty that Splash Damage never released the full source.

About megatexturing even if TDM system is still working, is just impractical, the kind of mega files it produces is huge, at lest for good looking terrain textures, but IMO there's no need for it, you can still make relatively big terrains using classic material system with detail textures (textures that show only near to the player), you also need to remember to cut the "terrain" in pieces so it doesn't get render all at the same time.  But imo the real problem for outdoors in TDM is not the terrain texturing support, is the inability to create good visportals for it, with a system like occluders (and the LOD system) you could at least have something to optimize outdoor levels more. 

[VanishedOne]

There's a small amount of documentation about ET:QW's occlusion handling at https://wiki.splashdamage.com/index.php/Occlusion_Tests

Yes, the ET:QW/Q4/Prey SDKs are quite a tease. If we had the full source we could find out how Q4 and ET:QW implemented material maps, for example. (Although the biggest tease is D3's own expansion, since TDM seems to have been built on vanilla gamecode: I hope to see the missing parts of xrayRenderMap ported someday, but after seeing all the changes RoE made to support slow motion I'm not holding my breath for an Arrow Time potion.)

[HMart]

Reading that link almost seems like who wrote it, wanted to give clues on how the system was made, by reading that you pretty much know how they made the occlusion system, but knowing in general how they made it is not the same has to be able to implement the same in TDM engine.  

One way to do that single material dual behavior is to support material masks, Quake Wars for example is full of material masks for many things.

For example, based on the bugtracker link you posted,  if you wanted a wood/plaster surface, and make rope arrows behave differently depending where it stroke in the texture, then you would make a mask stage that would have a black and white (or other colors) texture (akin to alpha textures) where 100% white is wood, 100% black is plaster or vice versa, then the arrow would trace the material somehow, look at the mask stage, see what pixel color it detected and behave accordingly to that.  

Edited February 25, 2020 by HMart

[VanishedOne]

Right; the problem turned out to be how to implement it in this engine: https://bugs.thedarkmod.com/view.php?id=3135#c11735

[HMart]

Yes that is always the problem.