
Posts
3750 
Joined

Last visited

Days Won
49
Posts posted by duzenko


1 hour ago, stgatilov said:
That would be incorrect, because BFG frustum and D3 frustum can be different geometrically. You suggest drawing D3 frustum but culling BFG frustum. In some rare cases light might disappear due to incorrect frontend culling.
I was thinking the frustum is generated from the matrix. The both style matrices define the same frustum since one is computed from the other. Currently the D3style matrix is computed from the BFG one. I was suggesting to reverse this so that the BFG matrix is computed from the D3 one.
This way, we will end up with the D3 frustum, and both matrices would be compatible with it

All well that ends well

2 hours ago, stgatilov said:
It is not so easy, because in both cases light frustum which is later used in frontend depends on the matrix row which defines falloff parameter.
This code won't work properly with D3 light matrix:
// calculate the global light bounds by inverse projecting the zero to one cube with the 'inverseBaseLightProject' idRenderMatrix::ProjectedBounds( light>globalLightBounds, light>inverseBaseLightProject, bounds_zeroOneCube, false );
Because it assumes that matrix is 3x3 projective transform, i.e. it relies on BFG definition with Z divided by W.
Doom 3 code used to set clipping planes directly from start/end parameters, so the resulting polyhedron does not need to be frustum:
/* =================== R_SetLightFrustum Creates plane equations from the light projection, positive sides face out of the light =================== */ void R_SetLightFrustum( const idPlane lightProject[4], idPlane frustum[6] ) { // we want the planes of s=0, s=q, t=0, and t=q frustum[0] = lightProject[0]; frustum[1] = lightProject[1]; frustum[2] = (lightProject[2]  lightProject[0]); frustum[3] = (lightProject[2]  lightProject[1]); // we want the planes of s=0 and s=1 for front and rear clipping planes frustum[4] = lightProject[3]; frustum[5] = lightProject[3]; frustum[5][3] = 1.0f; frustum[5][3] /= frustum[5].Normalize(); frustum[4][3] /= frustum[4].Normalize(); frustum[3][3] /= frustum[3].Normalize(); frustum[2][3] /= frustum[2].Normalize(); frustum[1][3] /= frustum[1].Normalize(); frustum[0][3] /= frustum[0].Normalize(); }
In order to restore D3 code for shading but retain BFG code with inverseBaseLightProject, one has to:
 Build D3 matrix.
 Compute 6 planes from D3 matrix using D3 code.
 Build polytope from these planes.
 Bound polytope vertices from forward/backward, i.e. determine minimal near/far clipping distances which are enough to cover the whole light volume.
 Build BFG matrix with several adjustments: a) "target" is orthogonalized like in D3, b) near/far values taken from what we obtained on step 4, c) replace "far" with "far"  "near".
 Use D3 matrix for shading, and BFG matrix for culling.
Oh dear
I thougt that local var localProject is not used beyond creating the light>lightProject
But I can now see it's used for light>baseLightProject
Let's start again
The conversion between the two styles is rather straightforward. Can't we use R_SetLightProject for D3 style matrix and then compute BFG style by swapping columns and normalizing Z?

4 hours ago, Garrett2014 said:
I alreao read about a hotfix but dont wish to do anything untill i get advice.
Not wishing to do anything is not exactly a great start for support request
Sorry for being the Negative Ned
 1
 1

3 hours ago, stgatilov said:
Theoretically, "end" should map to falloff = 1 with D3 code and to (1  f/n) with BFG code.
Yes, it has changed its meaning, as well as handling of nonorthogonal "target" and weirdlyoriented "start"/"end".
Given that behavior was silently changed to BFG several releases ago (I guess since 2.05), we have to decide which behavior should be left now so that number of broken FMs is minimalDo you have opinion on how those changes practically change mapping and gameplay?
Is there any upside/downside? (even if theoretical)
Right now I'm leaning towards reverting to D3 matrix, because this thing has dragged long enough, generated lots of noise and so far has lead to nothing other than examples of incompatibility
I'm talking about the BFG projection matrix build function only, not the render matrix stuff (which looks quite useful)

2 hours ago, stgatilov said:
So, I think the current code is close to BFG, except that someone has changed zScale from 1/(n+f) to 1/f (which most likely does not achieve the desired effect).
If we change it back to 1 / (n+f), then we'll have BFG behavior.Here are the differences between D3 and BFG:
 D3 falloff ends in W, BFG falloff ends in Z and is multiplied by (f+n). Minor postprocessing code converts the matrix from BFG convention to D3 convention.
 D3 falloff changes along "end"  "start" vector, which makes it completely independent of the frustum. BFG falloff always changes along the frustum's main direction.
 Both falloffs are 0 at "start", but D3 falloff = 1 at "end", while BFG falloff = 1 at "start" + "end".
 If "target" is not orthogonal to "right"/"up" plane, then D3 internally projects it onto plane normal and uses the projected vector instead of "target" everywhere. BFG does not care and uses "target" directly.
What next?
We have to decide which of the two behaviors we want to reproduce both in TDM and in DR?
By the way, does DarkRadiant support D3BFG maps?UPDATE: I guess I can restore both functions and allow switching between them easily...
It was me trying to fix I can't remember what now. I changed to 1 / (n+f) locally but did not commit yet
Can you please look where the "end" pos projects to with the both matrices?
QuoteBoth falloffs are 0 at "start", but D3 falloff = 1 at "end", while BFG falloff = 1 at "start" + "end".
Does it mean that the "end" changed its meaning in BFG? What should we do about it?
56 minutes ago, OrbWeaver said:In DR terms I'd vote to keep falloff in Z and perspective divisor in W, for consistency with omni lights. Otherwise we'd need to add special handling in the shader to deal with projected lights using a different texture coordinate for the falloff texture.
Note that we don't force mappers to use start/end (they have to tick another box to enable it), so rendering code needs to handle the situation where these values are unset. I think we currently assume no start means light origin, and no end means the same as the target vector.
We don't explicitly support it and I haven't done any testing myself. It might happen to work by accident though.
We don't discuss changing anything in this regard (like swapping W and Z)
Please don't worry about it
The two matrix styles coexist in TDM, they don't replace each other anywhere

@OrbWeaverLinear Z, huh? I was wrong then
 1

14 minutes ago, stgatilov said:
Looking at tdm_interaction.*.glsl, I see:
// (VS) light projection texgen var_TexLight = ( attr_Position * u_lightProjectionFalloff ).xywz; // (FS: lightColor) vec3 lightProjection = textureProj(u_lightProjectionTexture, var_TexLight.xyw).rgb; vec3 lightFalloff = texture(u_lightFalloffTexture, vec2(var_TexLight.z, 0.5)).rgb; lightColor = lightProjection * lightFalloff;
I guess these equations come from D3 shaders, and probably confirm my theory that W parameter is not divided by perspective divisor in D3. It would be great if someone could find the original shader from Doom 3 and check it.
However, I have some doubts now about BFG: perhaps Z is not divided by W there too.
Could someone please check it?We still use the D3style matrix in shaders, nothing has changed in that regard
It does not matter really which matrix style the BFG shaders use, they are equivalent, because one is built from the other
The question is about the first "original" matrix in R_DeriveLightData and which building function we want to stick with

There's a good reason that falloff does not make sense with perspective projection
The z component is crammed to have most of its range near the front plane. You won't be able to apply texture to it in a meaningful way. 
Before you get too excited, think about the fact that people don't use projected lights like that

Try the torrent
magnet:?xt=urn:btih:740C47E25BC8BC472F7D2DC61FE746E685E2B99F

18 hours ago, nbohr1more said:
I think the answer to this comes from answering these questions:
1) Is there more work to make shadow rendering efficient that can benefit from BFG code migration?
2) If so, will retaining the BFG matrix make porting BFG code easier?
3) Likewise, will the BFG matrix design make it easier to port GPU skinning code?
4) Likewise, is any portal \ light culling done in BFG more efficient and would it be easier to port?
If the BFG matrix does not assist with any code porting efforts, does not assist with any rendering efficiency designs, and causes problems with Dark Radiant compatibility then it should be reverted.
Well, I kinda like some of the functionality the render matrix offers, like easy transformation of Vec3 to Vec4

1 hour ago, stgatilov said:
Analyzed the matrix in the current code (which I guess came from BFG).
See all the formulas in PDF.Some notes:
 I explicitly extracted lengths from all three vectors: R, U, T are unit vectors from spawnargs, and W, H, D (aka width, height, and depth) are the lengths of spawnargs.
 I decomposed the matrix into three transforms: rotation (similar to modelview), projection, and some additional shift to put target point at center in output. It is easier to understand this way.
 While projection matrix in OpenGL (e.g. of gluPerspective) produces all coords in [1..1] range, this matrix produces all coordinates in [0..1] range after division. The matrix also inverts Y coordinate.
 Standard convention is used: w is divisor, range of z after division is normalized.
 "start"/"end" spawnargs define near and far clipping plane of frustum, similar to OpenGL. However, only their component along "target" vector matters, the two other components don't affect anything.
 Let n = dot(target, start), f = dot(target, end). The near clipping plane is at distance n, and far clipping plane is at distance (f + n). Quite surprisingly, "end" vector does not define far distance, it defines (far  near) difference instead.
 Frustum halfangles are: length(right) / length(target) and length(up) / length(target). Multiplying all three vectors by same coefficient does not change anything, only their length ratios matter.
 If R/U/T triple is not orthogonal, then the matrix which I called "rotation" is no longer orthogonal. Strictly speaking, it maps a parallelepiped with face normals R, U, T respectively into axisaligned box. The vectors are not axes of the local coordinate system (I guess they are called "cobasis"), they are not normals of frustum planes.
Judging from point 8, I don't think nonorthogonal R/U/T were ever intended.
However, if there is no special tools, drawing/specifying three exactly orthogonal vectors is very hard. If mapper sets almost orthogonal vectors, then the transformation will work almost as if they were orthogonal...I'd say the spawnargs should be set as follows:
 Set frustum direction into "target" vector, choose length arbitrarily.
 Specify X/Y coordinates on frustum "screen" by choosing orthogonal "right" and "up" vectors.
 Set lengths of "right" and "up" in such way that (target +/ up +/ right) vectors look through frustum corners.
 Choose "start" and "end" so that "start" lies on near frustum plane, and "start" + "end" lies on far frustum plane.
Thanks
I don't think we should not try to modify either of the two projection matrices, just pick one of them and use in both TDM and DR.
Do you have any idea what they tried to achieve with this change and why?

Once again, we need to decide if we want to switch back to the D3 matrix in TDM or switch to BFG matrix in DR
I don't think I'm qualified to make that choice despite all the time I spent on this

15 minutes ago, HMart said:
I wish I was so good at math as you guys, so I could help but alas. I can just say they used idStudio (their new editor) to do BFG maps and If you have Rage 1, it comes with idStudio, you can see if the lights still work the same there, even thou is a new editor still works mostly like Doom 3 Radiant so should be familiar.
Here someone did some tutorials for it. https://www.moddb.com/mods/ehrliasummerwheel/tutorials/ragetoolkittutorialsandupdates
Did it ever go opensource like radiant?

4 hours ago, stgatilov said:
Where is it located, by the way?
tr_lightRun.cpp in TDM
R_DeriveLightData/R_ComputeSpotLightProjectionMatrix/R_SetLightProject
Light.cpp in DR
Light::updateProjection
QuoteBut if we find that D3 math related to such light frustums makes no sense, then what?
The math in D3 was okay, I suppose. The problem is that something changed in BFG and it's not consistent any more. The end point no longer projects to the far plane. I want to understand if it was a bug in BFG, or something changed in the logic behind the anchor points. And if it did change, then it should have been reflected in whatever editor they used for BFG maps?

4 hours ago, stgatilov said:
Do you think skewed/asymmetric frustums are important?
I think you can always replace it with orthogonal/symmetric frustum without much problem...That's what I've been asking for a while but no one seems to care\know
It seems asymmetric frustums have been the "norm" in D3/DR/TDM  of course projected lights in general were taken for granted and people just used them as is without any second thoughts
If you create a projected light in DR and start dragging the anchor points around you're bound to lose symmetry soon enough
Technically it would be easier to handle if left=right and up=down but there's an issue of compatibility. Expect some angry mappers when they find out their asymmetric lights have stopped working. It can also be used for optimization if the symmetric frustum spans too much.

13 hours ago, stgatilov said:
Yeah, it is natural to assume (target, up, right) is orthogonal basis, like axes of coordinate system.
Somehow, it reminds me the story of rotation hack: everyone assumed orthogonality but nobody restricted it.I wonder what would happen if we orthogonalize (target, up, right) spawnargs with GramSchmidt in game engine when parsing light spawnargs. How much stuff would stop working?
OTOH if we want to support "asymmetric" (off center) projections then would require exactly that  nonorthogonal basis?
Like glm::frustum (btw maybe just use that?)

1 hour ago, OrbWeaver said:
I can't speak to the original game design, but in terms of DR, there is no restriction on where you can drag the right and up points, so it's quite possible to set up a projected light with nonorthogonal vectors (the frustum in this case becomes a sort of weird rhomboid shape). I guess the game is just taking this possibility into account by not assuming that the vectors are orthogonal when calculating the matrix.
In this case logically the resulting frustum center point is not what DR shows
I would suggest to fix this in DR then

Could you maybe shed some light why target, right, up aren't orthogonal?
One difference between d3 and bfg is that BFG uses the target vector from spawnargs for Z basis vector and D3 computes a normal between up and right instead.

25 minutes ago, OrbWeaver said:
I was confused when I was looking at this code (to see if I could integrate it into DR to fix projected light rendering). I wasn't sure why there were "old" and "new" conventions, but if some of this code came from BFG and some from Doom 3, this would explain the distinction.
Our shaders use W for the divider as per normal expectations, and use projective texture lookups for S and T (i.e. S/W and T/W are what gets rendered) but not for Z, which is undivided. As you say it would be possible to put the divider in Z if the shader was updated but I can't really see the point of it.
Forget about shaders, this is not about that
QuoteIn DR the Z coordinate is used for the falloff texture, so the light will become dimmer as you move further from the origin. Is this not what the game does too? Do D3 projected lights have the same brightness along the whole target vector, or do they dim based on distance?
No, the game, same as DR, uses the falloff texture. The texture decides where the light is brighter or dimmer. There's no shader math for that.
QuoteOur (DR) Z goes from 0 to 1, but this is actually wrong: the falloff texture is symmetric about the 0.5 position, so our lights are actually dim near the origin, brightest halfway along the target vector, and then dim to black. I'm trying to fix this using my limited math skills so that Z goes from 0.5 to 1, resulting in maximum brightness near the "light bulb" which dims to 0 as you reach the end of the target vector. But if the game doesn't do this, DR shouldn't either.
The texture does not have to be symmetric, if a particular texture is symmetric, that still does not mean anything. There's no really need or sense to change anything in that regard.

Added the test code for that (placed in the end of R_DeriveLightData)
Spoilerauto test = [light](idVec3 vert, int i) { idVec4 v4; light>baseLightProject.TransformPoint( vert, v4 ); auto& M2 = *(idMat4*) light>lightProject; idVec4 tmp( 0, 0, 0, 1 ); tmp.ToVec3() = vert; auto x = M2 * tmp; common>Printf( "%2d%30s%30s\n", i, v4.ToString(), x.ToString() ); v4 /= v4.w; x.ToVec3() /= x.z; common>Printf( " %30s%30s\n", v4.ToString(), x.ToString() ); }; for ( int i = 0; i < light>frustumTris>numIndexes; i++ ) { auto index = light>frustumTris>indexes[i]; auto vert = light>frustumTris>verts[index]; test( vert.xyz, i ); } ALIGNTYPE16 frustumCorners_t corners; idRenderMatrix::GetFrustumCorners( corners, light>inverseBaseLightProject, bounds_zeroOneCube ); for ( int i = 0; i < 8; i++ ) { test( idVec3( corners.x[i], corners.y[i], corners.z[i] ), i ); } idVec4 startInGlobal; lightMatrix.TransformPoint( light>parms.start, startInGlobal ); common>Printf( "start %s >> %s\n", light>parms.start.ToString(), startInGlobal.ToString() ); test( startInGlobal.ToVec3(), 0 ); idVec4 endInGlobal; lightMatrix.TransformPoint( light>parms.end, endInGlobal ); common>Printf( "end %s >> %s\n", light>parms.end.ToString(), endInGlobal.ToString() ); test( endInGlobal.ToVec3(), 1 ); idPlane d3Project[4]; R_SetLightProject( d3Project, vec3_origin, light>parms.target, light>parms.right, light>parms.up, light>parms.start, light>parms.end ); auto M1 = *(idMat4*) d3Project; auto to4 = []( idVec3 v ) { return idVec4( v.x, v.y, v.z, 1 ); }; auto st1 = M1 * to4( light>parms.start ); auto en1 = M1 * to4( light>parms.end ); common>Printf( "d3 start %s >> %s\n", light>parms.start.ToString(), st1.ToString() ); st1.ToVec3() /= st1.z; common>Printf( " >> %s\n", st1.ToString() ); common>Printf( "d3 end %s >> %s\n", light>parms.end.ToString(), en1.ToString() ); en1.ToVec3() /= en1.z; common>Printf( " >> %s\n", en1.ToString() );
Results
Unless I'm not mistaking it means that the BFG matrix does not map the end pos from the spawn args to light frustum "far" plane, unlike the D3 code
At this point we should decide if it was a bug or a feature and what their intention was when they changed the matrix.

@stgatilovThanks for looking into this
I think I got sidetracked and distracted you from the real problem
Yes, the thing I wrote about in the last post is a nonissue. Since the baseLightProject is always based on the lightProject, they'll both produce the same result
My problem is the code that builds lightProject. R_SetLightProject vs R_ComputeSpotLightProjectionMatrix will give different matrix sets and different frustums
I'm going to try another thing now. Transform local start/end coordinates (in the spawn args) to global space and see where their "projections" are
EDIT. Also note, that locally I reverted to
return 1.0f / ( zNear + zFar );
in the end of R_ComputeSpotLightProjectionMatrix but did not commit to svn as I'm still investigation this

Is the outline supposed to work for both old and new backend? I think it should though I vaguely remember something only working in new backend
Light frustums different in DR and TDM
in DarkRadiant Feedback and Development
Posted
Then what are we left with? It would be great to keep the render matrix stuff but you say it requires the BFGconstructed projection matrix? I suppose we should then change DR to use that method as well?
Probably the only change should be the end pos being relative to origin rather than start to improve compatibility with the existing maps?
Sorry, what do you mean by that? Being a frustum means that near/far planes are parallel to XY or ???