OrbWeaver Posted October 8, 2020 Report Share Posted October 8, 2020 While doing some render system refactoring I noticed that projected lights were not rendering correctly in lighting preview mode, and thought it was something I had broken until I regressed to earlier revisions and discovered that it has actually been broken for some time (in fact I tried checking out revisions from 3 years ago and the problem still reproduced). Projected lights are something of a "bug trap" because I think most mappers don't use them or the DR lighting preview very much and therefore they don't get a lot of testing. The behaviour I see is that while the rendered light projection obeys the shape of the frustum (i.e. the light_up and light_right vectors), it seems to ignore the direction of the light_target vector and always points the light straight downwards. The length of the target vector has an effect (on the size and brightness of the light outline on the floor), but the direction is ignored. Note that the problem only applies to the rendered light itself. The frustum outline appears to be correct (as it is handled with different code). I believe the problem is with how the light texture transformation is calculated in Light::updateProjection(), although I can't be sure. I will make an effort to debug this although projective texture transformations are near the limit of my mathematical abilities (I can understand the general concept and probably figure out the process step-by-step by taking it slowly), but might have to ask @greebo for assistance if the maths becomes too hard. Another approach is to try and revert the relevant code back to when (I think) it was working correctly after I initially got projected lights working, although that might have been 10 years ago or more so a straight git bisect isn't practical. 3 Quote DarkRadiant homepage ⋄ DarkRadiant user guide ⋄ OrbWeaver's Dark Ambients ⋄ Blender export scripts Link to comment Share on other sites More sharing options...
nbohr1more Posted October 8, 2020 Report Share Posted October 8, 2020 @stgatilov I mentioned this a loooong time ago but I thought it might be relevant here. If the projection volume could show a transparent representation of the effect of both the 1D and 2D projection images. With this in place authors could more easily control where light happens with the projection volume. Quote Please visit TDM's IndieDB site and help promote the mod: http://www.indiedb.com/mods/the-dark-mod (Yeah, shameless promotion... but traffic is traffic folks...) Link to comment Share on other sites More sharing options...
Obsttorte Posted October 9, 2020 Report Share Posted October 9, 2020 I can help you with the maths, too. Just in case. 1 1 Quote FM's: Builder Roads, Old Habits, Old Habits Rebuild Mapping and Scripting: Apples and Peaches Sculptris Models and Tutorials: Obsttortes Models My wiki articles: Obstipedia Texture Blending in DR: DR ASE Blend Exporter Link to comment Share on other sites More sharing options...
stgatilov Posted October 9, 2020 Report Share Posted October 9, 2020 I think projection matrix for this case is computed in R_ComputeSpotLightProjectionMatrix in tr_lightrun.cpp. On the other hand, I have no idea if DarkRadiant needs the same matrix and uses the same conventions. 1 Quote Link to comment Share on other sites More sharing options...
OrbWeaver Posted October 9, 2020 Author Report Share Posted October 9, 2020 Thanks for all the support and offers of help; I didn't realise this would get such a positive response. I will post my investigations in this thread (partly for my own sanity, partly for future documentation in case this needs to be debugged again, and partly so anyone can jump in in case I'm going completely off the rails). 51 minutes ago, stgatilov said: I think projection matrix for this case is computed in R_ComputeSpotLightProjectionMatrix in tr_lightrun.cpp. On the other hand, I have no idea if DarkRadiant needs the same matrix and uses the same conventions. I'm sure there will be some differences in the matrix DR needs, but this will still be helpful as an aid to understanding the process, so thanks for the pointer. Quote DarkRadiant homepage ⋄ DarkRadiant user guide ⋄ OrbWeaver's Dark Ambients ⋄ Blender export scripts Link to comment Share on other sites More sharing options...
greebo Posted October 9, 2020 Report Share Posted October 9, 2020 I recall working on the projected light stuff years ago. It took me quite some time (and I think 300 sheets of paper) to wrap my head around how the projection is actually working. Since knowledge is deteriorating exponentially, I'm sure I forgot 80% of it, but everything is always easier the second time. I must have left some comments for myself too in that code section - I recall the original GtkRadiant code was completely uncommented (it was pretty much a plain copy from some original id code). So, I'll try to help if I can, just post here. Quote Link to comment Share on other sites More sharing options...
Obsttorte Posted October 9, 2020 Report Share Posted October 9, 2020 Don't know if this helps, but when I was using those type of lights I always rotated them directly instead of changing the light_target etc. spawnargs. That always worked for me. EDIT: Just noticed that you were talking about an issue in DR, not TDM. Should've read more carefully. Nevertheless, the preview takes the rotation into account. So maybe it still proves useful on narrowing down the issue. Quote FM's: Builder Roads, Old Habits, Old Habits Rebuild Mapping and Scripting: Apples and Peaches Sculptris Models and Tutorials: Obsttortes Models My wiki articles: Obstipedia Texture Blending in DR: DR ASE Blend Exporter Link to comment Share on other sites More sharing options...
OrbWeaver Posted October 12, 2020 Author Report Share Posted October 12, 2020 On 10/9/2020 at 9:39 PM, Obsttorte said: Don't know if this helps, but when I was using those type of lights I always rotated them directly instead of changing the light_target etc. spawnargs. That always worked for me. EDIT: Just noticed that you were talking about an issue in DR, not TDM. Should've read more carefully. Nevertheless, the preview takes the rotation into account. So maybe it still proves useful on narrowing down the issue. That is actually very helpful, thanks. I just did a test, and rotating the light using the rotation tool works fine, including the render preview. The rotation is not applied to the target vector, but stored in a separate rotation spawnarg which is applied to the texture matrix in a separate step, and this appears to be working fine. This would explain why mappers might not have noticed the problem: since you can change the shape of the light OK, and change its direction by rotating, manual dragging of the light_target vector may be unnecessary. In summary: The location of the light works correctly. The rotation of the light works correctly. The shape of the frustum (light_up and light_right) appears to work correctly, although DR allows you to drag the points into some pretty weird configurations which may or may not correspond to valid game state. The length of the light_target vector works correctly. The direction of the light_target vector does NOT work (it is always assumed to point downwards, before rotation). Quote DarkRadiant homepage ⋄ DarkRadiant user guide ⋄ OrbWeaver's Dark Ambients ⋄ Blender export scripts Link to comment Share on other sites More sharing options...
OrbWeaver Posted October 12, 2020 Author Report Share Posted October 12, 2020 OK, let's start from the beginning (this will be a useful intellectual exercise and perhaps create some useful documentation that might help future development. A projected light works by mapping points in 3D space into texture coordinates, which are then used to sample a 2D light falloff texture and a separate 1D gradient texture resulting in a final color for each illuminated pixel. Every point within the light volume should be transformed into a triplet of texture coordinates, where the X and Y coordinates (conventionally called S and T in a shader) run from 0.0 to 1.0, and the Z coordinate runs from 0.5 at the origin to 0.0 at the target plane. The other half of the Z range (from 0.5 to 1.0) is actually behind the light origin but we clamp this to 0 to avoid the light projecting backwards as well as forwards. The projected light is defined by four vectors: The origin in 3D space (the tip of the pyramid). A light_target vector, labelled t in the diagram, which points from the origin to the center of the target plane, and is considered the light's local -Z axis. A light_right vector, labelled r in the diagram, which goes from the center of the target plane to its rightmost edge, and is considered the lights +X axis. A light_up vector, labelled u in the diagram, which goes from the center of the target plane to its uppermost edge, and is considered the light's +Y axis. Based on these vectors, and the expected resulting texture coordinates, we can assume certain properties of our projection matrix P which we need to construct: P · (0, 0, 0) = (0, 0, 0.5) The origin point must have a Z texture coordinate of 0.5, which is the brightest part of the Z falloff texture. For all x and y: P · (x, y, 0) = (0, 0, 0.5) Because of the singularity at the light origin point, any point with Z=0 should have X and Y coordinates reduced to 0. P · t = (0.5, 0.5, 0) The target vector points to the center of the projected backplane, which has texture X/Y coordinates of 0.5 and a Z coordinate of 0 (the darkest point in the Z falloff texture). P · (t + r + u) = (1, 1, 0) The "top right" corner, reached by travelling along the target vector then along both the right and up vector, should be at the (1.0, 1.0) edge of the 2D falloff texture. P · (t - r - u) = (0, 0, 0) Likewise the "bottom right" corner, reached by travelling along the target vector than backwards along the right and up vectors, should be at the (0.0, 0.0) edge of the 2D falloff texture. A reasonable approach to debugging, therefore, will be to print out the transformations of these vector combinations under our projection matrix P, and see how far off they are the expected texture coordinates. Quote DarkRadiant homepage ⋄ DarkRadiant user guide ⋄ OrbWeaver's Dark Ambients ⋄ Blender export scripts Link to comment Share on other sites More sharing options...
OrbWeaver Posted October 14, 2020 Author Report Share Posted October 14, 2020 On 10/12/2020 at 8:48 PM, OrbWeaver said: For all x and y: P · (x, y, 0) = (0, 0, 0.5) Because of the singularity at the light origin point, any point with Z=0 should have X and Y coordinates reduced to 0. Actually I think I'm wrong about this one. It's not the texture coordinates which are 0 at the origin, it's the image size. The texture coordinates are therefore effectively infinite, but this will not be achieved by actually setting the X and Y coordinates to infinity, but by setting the projective (W) texture coordinate to 0, causing the projective texture lookup (X/W, Y/W) to blow up to infinity. I think this means that the W coordinate will vary from 0 at the light origin to 1 at the light_target plane. In any case the next step will be to add some suitable debugging code to print out the matrix transforms on known vectors like the light_target, and then examine the engine code to see how it constructs the matrix. Quote DarkRadiant homepage ⋄ DarkRadiant user guide ⋄ OrbWeaver's Dark Ambients ⋄ Blender export scripts Link to comment Share on other sites More sharing options...
Obsttorte Posted October 15, 2020 Report Share Posted October 15, 2020 I think you are missing the translation Vector in your calculations. P•(0,0,0) = (0,0,0) for any matrix P. So the structure of the transformation is P•x+(0,0,0.5)=y, where x denotes the world coordinates and y the texture coordinates. I haven't looked at the DR code, but in idTech4 code it actually works like that, although it is not visible without examining the steps of the calculation as the developers haven't utilized c++ classes to replicate the mathematical notation (I've once read an interview with John Carmack where he stated, that they shifted from C to C++ during the development of Doom 3 and therefore had to familiarize with the new concepts themselves during the development, causing them to not make use of all the possibilities C++ provided). Quote FM's: Builder Roads, Old Habits, Old Habits Rebuild Mapping and Scripting: Apples and Peaches Sculptris Models and Tutorials: Obsttortes Models My wiki articles: Obstipedia Texture Blending in DR: DR ASE Blend Exporter Link to comment Share on other sites More sharing options...
OrbWeaver Posted October 15, 2020 Author Report Share Posted October 15, 2020 3 hours ago, Obsttorte said: I think you are missing the translation Vector in your calculations. P•(0,0,0) = (0,0,0) for any matrix P. So the structure of the transformation is P•x+(0,0,0.5)=y, where x denotes the world coordinates and y the texture coordinates. I think that's implied though, isn't it? We have (with added W coordinates): On 10/12/2020 at 8:48 PM, OrbWeaver said: P · (0, 0, 0, 1) = (0, 0, 0.5, 0) P · t = (0.5, 0.5, 0, 1) or P · t = (0.5, 0.5, 1, 1) So the Z coordinate must go from 0.5 at the origin to either 0 or 1 at the target point (either should look the same because the Z falloff texture is symmetrical, but I guess DR should do the same as what the game does). It is definitely going to need an offset of 0.5, resulting in a formula like: Z[tex] = 0.5 + 0.5 * (p · t̂) / ‖t‖ where t̂ is the normalised target vector and p is an arbitrary point within the volume whose Z texture coordinate we want to calculate. Quote DarkRadiant homepage ⋄ DarkRadiant user guide ⋄ OrbWeaver's Dark Ambients ⋄ Blender export scripts Link to comment Share on other sites More sharing options...
Obsttorte Posted October 15, 2020 Report Share Posted October 15, 2020 Oops. It's not really convenient to use 4 dimensions to describe a transformation in 3 dimensions, imho, so I tend to expect the transformation matrix to be 3x3. My bad. Quote FM's: Builder Roads, Old Habits, Old Habits Rebuild Mapping and Scripting: Apples and Peaches Sculptris Models and Tutorials: Obsttortes Models My wiki articles: Obstipedia Texture Blending in DR: DR ASE Blend Exporter Link to comment Share on other sites More sharing options...
OrbWeaver Posted October 15, 2020 Author Report Share Posted October 15, 2020 I think the approach I will take is to assume the D3 code is correct, and try to adapt it as best as possible to fit DR, if changes are needed. So as stgatilov helpfully pointed out, the relevant code is in R_ComputeSpotLightProjectionMatrix(). The first code block is easy to understand, although written in a slightly weird way. I wonder if this is an optimisation because calculating "inverse square root" is quicker than calculating the regular Pythagorean length using square root. const float targetDistSqr = light->parms.target.LengthSqr(); const float invTargetDist = idMath::InvSqrt(targetDistSqr); const float targetDist = invTargetDist * targetDistSqr; So we now have the length of the target vector and its inverse length which we can use to calculate a unit vector, which we then do for all three of the light vectors: const idVec3 normalizedTarget = light->parms.target * invTargetDist; const idVec3 normalizedRight = light->parms.right * (0.5f * targetDist / light->parms.right.LengthSqr()); const idVec3 normalizedUp = light->parms.up * (-0.5f * targetDist / light->parms.up.LengthSqr()); The first one is a simple normalisation of the target vector, but the right and up lines I'm a little confused about. Why is the length of the target vector multiplied into the normalised right and up vectors? Won't this mean that the "normalised" right vector will no longer have a fixed unit length (or a length of 0.5 in texture space), but a length of half the target vector? It's not obvious to me why we would want the right and up unit vectors to scale with the target length. Quote DarkRadiant homepage ⋄ DarkRadiant user guide ⋄ OrbWeaver's Dark Ambients ⋄ Blender export scripts Link to comment Share on other sites More sharing options...
Obsttorte Posted October 16, 2020 Report Share Posted October 16, 2020 In the TDM code the content of the matrix calculated in R_ComputeSpotLightProjectionMatrix() is than used to calculate the boundaries of the light volume, that are represented in world-size coordinates (but relative to the light origin). So the volume is not clamped to [-0.5...0.5] or anything like that, but is the actual volume as defined in the spawnargs. Quote FM's: Builder Roads, Old Habits, Old Habits Rebuild Mapping and Scripting: Apples and Peaches Sculptris Models and Tutorials: Obsttortes Models My wiki articles: Obstipedia Texture Blending in DR: DR ASE Blend Exporter Link to comment Share on other sites More sharing options...
OrbWeaver Posted October 16, 2020 Author Report Share Posted October 16, 2020 4 hours ago, Obsttorte said: In the TDM code the content of the matrix calculated in R_ComputeSpotLightProjectionMatrix() is than used to calculate the boundaries of the light volume, that are represented in world-size coordinates (but relative to the light origin). So the volume is not clamped to [-0.5...0.5] or anything like that, but is the actual volume as defined in the spawnargs. Sorry I don't quite get what you mean. As I understand it, a matrix can never actually clamp anything, because that would be a non-linear transformation. The purpose of the matrix is to transform the coordinate system so that the volume, which (as you rightly say) is specified in world coordinates like [128, 128, 64], ends up with texture coordinates like [0, 0, 0.5] or [1, 1, 1]. Coordinates outside the light volume would still be transformed in this way, but they would end up with texture coordinates above 1 or less than 0, which would then be clamped to black by the OpenGL edge clamping mode which treats all pixels outside the texture boundaries as black. Quote DarkRadiant homepage ⋄ DarkRadiant user guide ⋄ OrbWeaver's Dark Ambients ⋄ Blender export scripts Link to comment Share on other sites More sharing options...
Obsttorte Posted October 17, 2020 Report Share Posted October 17, 2020 15 hours ago, OrbWeaver said: As I understand it, a matrix can never actually clamp anything, because that would be a non-linear transformation That's correct. I meant your assumption that the entrees of the transformation matrix are within said range. The resulting matrix is used afterwards in R_SetLightFrustrum(...). You can take a look at R_ComputePointLightProjectionMatrix(...) (way easier code) and how it is used for the light frustrum for comparision. So the plane equations are setup so that the planes normal vector multiplied by the vector from the light origin to the point we are interested in gives us a value between -0.5 and 0.5 if we are within the light volume, at least for point lights. I'll have to write down the projection matrix for the projected lights to see how the approach is there, but it seams they are transforming the pyramid that is the light volume so that the base sides equal the height (the length of the target vector). What makes things even more complicated is that the up, right and target vector don't neccessarely have to be orthogonal to each other. Quote FM's: Builder Roads, Old Habits, Old Habits Rebuild Mapping and Scripting: Apples and Peaches Sculptris Models and Tutorials: Obsttortes Models My wiki articles: Obstipedia Texture Blending in DR: DR ASE Blend Exporter Link to comment Share on other sites More sharing options...
OrbWeaver Posted October 20, 2020 Author Report Share Posted October 20, 2020 Nope, this is beyond me. Even after trying to incorporate the D3 code into DR, all I have achieved is swapping one set of meaningless numbers for another. I could stare at this code for the next 20 years and still have no idea what it's even trying to do, much less how to fix it. Projected lights don't work correctly in DR, and they probably never have. I guess most people don't fiddle with light_target vectors very much anyway, but if they do, perhaps stgatilov's "live update" code will allow them to check the result in game without needing to bother with the DR renderer. Quote DarkRadiant homepage ⋄ DarkRadiant user guide ⋄ OrbWeaver's Dark Ambients ⋄ Blender export scripts Link to comment Share on other sites More sharing options...
HMart Posted October 21, 2020 Report Share Posted October 21, 2020 This particular sample of OpenGL_4.0 cookbook is useful in any way? Sorry if is not I'm a noob on this stuff. ps- I have the full pdf available if you want it, not sure if I can post here thou... OpenGL_4.0_Shading_Language_Cookbook.pdf Quote Link to comment Share on other sites More sharing options...
OrbWeaver Posted October 22, 2020 Author Report Share Posted October 22, 2020 7 hours ago, HMart said: This particular sample of OpenGL_4.0 cookbook is useful in any way? Sorry if is not I'm a noob on this stuff. Thanks for the suggestion, but it doesn't really help: that is a very different way of implementing a spotlight, which is conical in shape and implemented entirely with math in the shader, whereas our spotlights are squar(-ish) pyramids defined by three vectors and implemented by using a matrix to map two falloff textures. The problem is that while I understand the general idea and parts of the mathematical process, it all seems to fall apart when I get down to the nuts and bolts and look at the matrix and vectors themselves. I won't abandon this work entirely but maybe poke at it from time to time in between other tasks; perhaps I'll gain a greater understanding of how the maths works while working on other areas of the renderer. 1 Quote DarkRadiant homepage ⋄ DarkRadiant user guide ⋄ OrbWeaver's Dark Ambients ⋄ Blender export scripts Link to comment Share on other sites More sharing options...
greebo Posted October 23, 2020 Report Share Posted October 23, 2020 I'm probably stating the obvious, but when I worked on that topic, the key for me was not to look at the source and target of the transformation, but to figure out the steps in between - like 1) move coordinate system to origin, 2) scale, 3) rotate coordinate system to match up the direction, 4) move it to target location, etc. and then often reverse all these steps to get the actual matrix. Once that is working, one could look into making things more efficient by combining transformations or other tricks, like making use of the affine-ness of the involved transformations. One more thing that helps is to make use of the DR renderer and/or the console to get visualisations of what is happening. As you say, the vectors and matrices tend to be unhelpful when it comes to debugging stuff. Sometimes other things like the DR frontend renderer is also applying transformations on top of what you come up with, which adds complexity. Quote Link to comment Share on other sites More sharing options...
OrbWeaver Posted October 25, 2020 Author Report Share Posted October 25, 2020 On 10/23/2020 at 1:45 PM, greebo said: 1) move coordinate system to origin, 2) scale, 3) rotate coordinate system to match up the direction, 4) move it to target location, etc. and then often reverse all these steps to get the actual matrix. That part is fine — I understand the bits of code which subtract the origin, handle the rotation key, scale the light radius into a [-0.5, 0.5] box (for omni lights) etc. But with projection it's a whole new ball game, because we have three distinct vectors (which are not necessarily orthogonal), both a Z and a W coordinate which must vary along the target vector (but with different ranges), and the target vector can also be "sheared" sideways and affect the X and Y coordinates as well. I think the mistake the current DR code makes is to try to map the projected volume into an AABB (before doing the projective divide part), which means that Z axis always points downwards and the target vector's X and Y components don't have any effect. This would be consistent with the behaviour whereby the length of the target vector affects the light falloff but moving the target point around in the X/Y plane makes no difference. I guess this is why the Doom 3 code actually uses the initial matrix to project the target vector into the new space and calculate offsets in the X and Y dimension: // project the target vector localProject.TransformPoint(light->parms.target, projectedTarget); // what happens to the target vector's X coordinate, plus 0.5 const float ofs0 = 0.5f - projectedTarget[0] / projectedTarget[3]; // final X transformation includes offset and varies with the target vector // as well (this enables shear I guess) -- the normalised target vector is // in row 3 of the matrix here. localProject[0][0] += ofs0 * localProject[3][0]; localProject[0][1] += ofs0 * localProject[3][1]; localProject[0][2] += ofs0 * localProject[3][2]; localProject[0][3] += ofs0 * localProject[3][3]; // what happens to the target vector's Y coordinate, plus 0.5 const float ofs1 = 0.5f - projectedTarget[1] / projectedTarget[3]; // final Y transformation as above: offset plus variation along the target // vector localProject[1][0] += ofs1 * localProject[3][0]; localProject[1][1] += ofs1 * localProject[3][1]; localProject[1][2] += ofs1 * localProject[3][2]; localProject[1][3] += ofs1 * localProject[3][3]; Quote One more thing that helps is to make use of the DR renderer and/or the console to get visualisations of what is happening. The ironic thing is that we already have a perfectly good visualisation of the light shape in the form of the wireframe frustum. We even have a method Frustum::getProjectionMatrix() which should give us the matrix corresponding to the frustum, but my initial naive attempt to just use this method did not succeed. I'm kind of torn between the two approaches really: try to figure out exactly what D3 is doing and adapt the code into DR (on the assumption that ID knew what they were doing and their code is probably both correct and pretty well optimised), or try to get it working using our own code (on the basis that it is more likely to be closer to how DR is expecting the matrix to be formed, whereas ID's code might have very different assumptions). Quote DarkRadiant homepage ⋄ DarkRadiant user guide ⋄ OrbWeaver's Dark Ambients ⋄ Blender export scripts Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.