Sikkpin Cubemap Projection

[nbohr1more]

Sikkpin has finally spilled the beans about how he got Doom 3 to use a Cubemap as a projection texture.

 

http://www.doom3world.org/phpbb2/viewtopic.php?t=24193

 

As Rich_is_Bored predicted, this required the modification of the vfp that controls all lighting (interaction.vfp?), and requires that all lights would need to use Cubemaps. Projected lights would work correctly as long as only one direction of the cubemap had a projection image while the others were black. Essentially every light would be a point-light (parallel lights are an unknown quantity at this time???).

 

Besides simply offering more accurate illumination direction from light sources, having this ability also could potentially make the use of radiance cubemaps more practical and has the implication that other 3D or n-D formats could be used instead of the current methods.

 

So the big ticket questions are:

 

1) How many Light Textures would need to be redone as Cubemaps? (Is it worth it?)

2) How many Lights are currently taking advantage of the current cylindrical lighting method where the switch would look wrong?

3) Beyond Cubemaps, could this method be extended to true 3D textures?

4) Can falloff be added back to this shader for more control than the current spherical algorithm (pending reply)?

 

I was really hoping that Sikkpin had discovered a new (hidden) native capability in Doom 3 (inferred from the _cubicLight internal image) rather than this system-wide change (akin to the problem with Parallax mapping etc), as this would make the Strombine style ambient light method as easy as:

 

1) Light scene with primary lights

2) Light scene with smaller lights to simulate radiance bounces

3) Capture grayscale Envshot ala Rich_is_Bored material replacement trick from the origin point of the primary lights

4) Remove smaller radiance lights

5) Setup lights with inverted falloff and use the envshot from step 3 as the projection Cubemap (placed in the same location as the primary lights)

 

Heck, since FPS would not matter for the purpose of collecting the envshot, you could even apply JC Dentons reflective methods and Sikkpin's SSIL and SSAO effects (or any other sophisticated lighting model that may require the z-pass hack etc) instead of Step 2.

 

I will continue to monitor the Doom3world thread for more details and other usage scenarios as they develop but I thought I would kick-off a thread about this and see what folks think. Would it be worth it to overhaul the lighting for a more accurate 3D projection method? More intuitive to mappers than the cylindrical pointlight falloff model? Or is this another thing that should wait until it can be selectively applied when GPL arrives?

 

 

Disclaimer:

 

For the record, I like Doom 3 lighting as is and I'm over-the-moon about the v1.03 improvements so this is merely a final look at providing something akin to (modern) lightmap quality ambient light for mappers who are so inclined. And a little fun speculation about pushing Doom 3 to the edge visually...

[Serpentine]

There are a few lights (i.e the desklamp) which I am interested on testing this out with.

 

I think streetlamps might be quite badass too, but performance is a question only testing and profiling can tell us, and that has to happen a long time before we can think of anything else.

[nbohr1more]

Controlling Falloff looks like it might be the big performance issue:

 

Yeah you can use the falloff texture but it'll only really work for projected lights. I mean you can make it work for point lights but it'll require you to sample the texture three times, once for each axis.

[New Horizon]

I think any further visual enhancements should wait until Doom 3 is open sourced. There is really no sense making further changes this close to the release of the code...which will probably be out in a year or so. We scaled back HDR because the performance costs were simply too high, so putting more taxing features in isn't likely at this point.

[nbohr1more]

That makes sense.

 

We'll probably still see this code in use after GPL when you will be able to apply it selectively or with a keyword.

 

(I really wish that cubicLight worked like ambientLight... I will do some futile trial and error just in case... )

Edited January 18, 2011 by nbohr1more

[rebb]

The biggest problem with changing how light projections are applied is that all existing maps would most likely have to be redone so they still look and ( since TDM is all about the light or lack of it ) play as intended.

[New Horizon]

The biggest problem with changing how light projections are applied is that all existing maps would most likely have to be redone so they still look and ( since TDM is all about the light or lack of it ) play as intended.

 

Yup...and changing all the maps to accommodate a new form of lighting just ain't gonna happen.

 

We just need to learn how to be happy with what we have, and not always chase the next great thing.

 

Thief players are still happy with the lighting in a 10 year old engine, we don't need to have all the latest gadgets for good gameplay...and gameplay is where it counts.

[Aprilsister]

We just need to learn how to be happy with what we have...

 

Well, in case you haven't noticed: I'm ecstatic.

 

Would like to see the bugs in TDM as it stands worked out, some of the promising tweaks tweaked, and the asset bank grown.

 

And heaps of FMs!

 

Smaller, sleeker, faster, more!

 

...

 

Not to say I don't enjoy reading about these adventures in experimentation... but as far as TDM goes I'm glad to see that the team is not going to be overly distracted; though they may well be curious too.

[Tels]

Apart from the "change all maps" I also see problems in that it is all-or-nothing, and apparently no falloff textures can be used and so on.

[nbohr1more]

Yeah, it looks like any change in this regard would have to wait till it could be "added as a new capability" rather than supplant the existing one. Ironically, I was hoping that this could be a performance booster but that 3 texture sample for the falloff looks like a non-starter.

 

On the plus side, Sikkpin has been made aware of Strombine lighting now (which he recognized as something similar to lighting done on some maps for Prey). He is currently considering what I was trying to accomplish with my wacky version of that idea

[nbohr1more]

This would've been fun to have during the HL2 verses Doom 3 days...

 

Sikkpin has shown that this method can be used to create HL2 style ambient cubemap lighting:

 

(Doom3world thread, latest screens):

 

http://www.doom3world.org/phpbb2/viewtopic.php?t=24193

 

[nbohr1more]

While going over all my rube-goldberg syle light painting ideas with by trying to pass non-interactive materials to interactive ones I have kept suppressing the blindingly powerful thought that this development from Sikkpin means that you could implement Volume Light Maps:

 

http://www.humus.name/index.php?page=3D&ID=47

http://www.humus.nam...p?page=3D&ID=47

 

Again, the problem here is that every light would need a 3D texture so I thought this was a silly but tantalizing frustration.

 

Then I recalled that the new Interaction shader does such things as "if no specular is stage is present, create one via math...".

 

So... could you redesign the Light interaction parser to look for BOTH a standard projection\falloff and 3D texture and then branch depending on what's available in the same way the Interaction shader branches based on specular?

 

Or, something like:

 

If the alpha-channel == 1 then use conventional light shader methods

If the alpha-channel == 0 then use volumetric lighting

 

??

 

If 3 branches could be setup, then you could have conventional, cubemap and 3D lighting depending on the specified texture?

 

Would that be too expensive?

 

Would the needed 3D texture be too large?

 

I know this is silly but sheer excitement over this long-standing holy grail being such a sliver away from being possible is just too compelling to leave floating around in my imagination.

Edited May 1, 2011 by nbohr1more

[nbohr1more]

For reference, a program that decodes UV coordinates from a bitmap:

 

http://www.doom3world.org/phpbb2/viewtopic.php?p=111453#111453

 

(A similar approach could be used if direct 3D texture decode cannot be implemented)?

[New Horizon]

I think it would simply be better to wait for D3 to go open source, rather than invest a lot of time creating work arounds.

 

When the source is available, it can all be done correctly be rewriting the renderer to support things natively.

[nbohr1more]

You are right, of course.

 

 

 

I just have this terrible curiosity over this.

 

Lunaran's Strombine map proved that an axis limited version of this is possible with no changes at all.

 

Then Sikkpin extended that to mind-blowing proportions with his Cubemap changes.

 

It would be really great to see someone make the next jump and prove that 3D textures are also possible.

 

I would even be delighted to see it built and proven to perform poorly.

 

It's the uncertainty that kills me.

[nbohr1more]

After discussing this with Sikkpin he has clarified that a shader cannot sample the same texture source with two different methods in the same shader.

 

That still leaves open the question of:

 

"What if you make the shader require ALL of the inputs (2d, Cube, 3d) and just place dummy inputs into the stages that aren't needed?"

 

That, of course, begs the question "Can you make Doom 3 light shaders accept more than one map stage?"

 

I'm almost positive that BlendLights will work that way but normal lights would not.

 

What you would have to do is encode all 3 formats into the same texture (or data structure) then decode all three in the Light shader. So it would, again, require a cumbersome asset conversion to take the projection textures and turn them into a data structure with a dummy cubemap and dummy 3D texture alongside the regular texture.

 

Add to that the complexity that you would have to either split the Y decode between the falloff and the projection shaders (and synch them...) or use some risky extrapolation from light volume surface-plane math (Sikkpin's Z-buffer extrapolation for his AO method was riskier but it worked... but it took a lot of banging against a brick wall to get it working).

 

...So this is almost put to bed...

 

After I pondered those stickier problems, I thought of another possible way to do this that wouldn't require any VFP coding antics.

 

Animated Z-Projection Texture.

 

Essentially you use the same method to paint the Z layers that a Cathode Ray Tube paints an image on the screen.

 

1) Bake your 3D texture light-map

2) Decode each Z layer into a separate image

3) String all the images together into a long chain such that they are played "Top Z image to Bottom to Bottom to Top" to create a single large z-projection texture

4) Create a Light material and set a looping scroll on the z-projection fast enough to have a static visual persistence (like a CRT) such that each "frame" is played precisely where it should hit the geometry...

 

The scroll look-up should be about the same expense as a 3D texture decode.

 

So... can scroll go fast enough to pull this off? :laugh: :laugh:

 

Yep... this is pretty-much done... I can sleep now... OCD over... kinda...

Edited May 2, 2011 by nbohr1more