MirceaKitsune

Was going to ask why in the world such a nice effect isn't included in vanilla but PeterSpy answered that. Can't wait for TDM to finally have PBR with this effect and more! Let us dream for 2.12

This is a consequence of not yet having a proper reflection system: Windows use a static blurry cubemap to simulate some level of reflection. This will probably be improved later someday, for now you'll have to use a window without any reflection at all or with the mirror pass for it. Don't know if there's one by default, if not it should be easy to make a custom material.

Can't help wondering if anyone still uses a 32-bit CPU or OS on a daily basis: I think that's about 15-20 years old now? Would be surprised if TDM still runs at decent FPS even on low settings, especially recent FM's with a lot of detail. Part of me finds it odd we still build one, though there's likely a few 32-bit players out there and no one should be left out.

Got it, and that's great news! I thought the initial plan was to create a new type of map light that captures a cubemap at map start and reflects it. I was thinking that would be better than existing skybox lights, but very inaccurate and unable to capture moving objects in reflections. A global grid based solution will be complex but I do believe worth it in the end. Also am I correct to assume this can tackle both reflections and global illumination? Technically you can project the captured cubemap mirrored in global space for a reflection, as a cube light for GI. Will definitely be amazing if both could be tackled at the same time with a common implementation.

Just realized a flaw in my proposal: Each probes view range can't be just its distance from the neighboring probe, as that distance will also be the maximum distance of the reflection itself. At a grid size of 64 for instance, a probe in front of a wall would only reflect at most 64 units behind that wall. So reflection distance should be minimum the grid distance doubled to avoid gaps, but also higher if you want your reflection to fade further. This would mean some bad news for performance unfortunately. An similar yet different alternative would be for each point to capture and retain a single color, acting as a 3D pixel / point of light altogether. This would automatically emanate light and show up in specular channel without requiring material changes, tackling both GI and reflectivity at once... the grid size would be the resolution of the result in this case. But now we run into the issue that points behind the player need to be updated... in this case it would be best to calculate and remember all voxels inside a room the moment its portal is opened, only updating ones within radius of a light or object when there's movement near it.

Last night we chatted on Discord about Vulkan support and PBR, bringing up a system for adding proper reflections once more. I suggested screenspace reflections but it was argued reflection probes would still be better than SSR in our case, a ticket for that is already open but I'm uncertain whether it's the best way: A manual approach would need new entities to be placed by the mapper... this requires extra effort and would exclude old FM's that are no longer updated, while the result will also be inaccurate and static meaning you won't see an AI reflected as they walk on a shiny metal plate for instance. If PBR with realistic graphics can be a hope for 2.12 or later, we'll definitely want to do it right rather than using a limited / limiting system. A technique came to mind that might just work for our engine and setup. I wanted to share it here before I forget the specifics; This might already be a common practice and even have a definition, for the purpose of this thread I'll just describe it as I originally imagined it and feel this would work for our engine. The idea is we'd use reflection probes but in an automated fashion: A probe is automatically spawned in every valid area (within bounds) in the player's view, at a given grid unit size. For example: If the grid scale is 16, a probe may exist at position '0 -48 16' another at ''0 -48 32' and so on. Every point projects its result on all surfaces in its radius which contain a specular channel masking it, the best alternative presently available till we were to convert all textures for PBR support. The cool thing is the same cubemap can also be projected as a light source, allow for global illumination in addition to just reflections! This would be similar to how the Irradiance Volume works in Blender / Eevee except each dot renders a little cubemap from its perspective. I already know what everyone is rightfully thinking: This is going to kill performance! After all each probe needs to produce a render from its perspective, and being a 360* panorama it will open portals in all directions. Normally that would be insane, but I thought of various ways in which the impact could be greatly minimized to very bearable amounts. The frame buffer of each probe will be at a very small resolution by default since much detail shouldn't be needed. Even 64x64 per cube face might do. Each probe only needs a draw distance double its grid size, given it only has to see as much as is necessary to fill the gaps between it and its neighbors. So if the grid size is set to 64, each probe would only have a draw distance of 128 to cover the space in between its neighbors, nothing beyond that would exist to it. Only probes the player can see would ever be spawned and calculated; If the view frustum doesn't overlap the virtual cube who's corners touch that probe's neighbors, the probe is dropped from memory. Probes are also only spawned in a valid visible space, never out of bounds including rooms culled by portals. A draw distance after which probes are removed or not spawned can also be included. This would further help by making any probe further than X distance be ignored, slowly fading away as to not noticeably pop in and out of existence. Reflections / GI are a discrete effect you'll only see up close. Similar to lights and shadows, the result of a probe should be cached and not calculated unless necessary. This means that unless something moves within radius of that probe its cubemap won't be rendered again. Probes would only be updated either when they first come into the player's view, or if something touching their cube has moved. Note that particles and lights with animated textures would have to count as you may see them in a detailed reflection, candles and torches would force constant updates per-frame for probes they intersect. If with all that performance is still affected, frame skipping is also a solution: Reflection probes can update at a lower frame rate to further decrease their impact. If you have a 60 Hz monitor and are running TDM at 60 FPS max, reflections could run at 30 / 20 / 10 FPS without looking out of place. They could in fact be defined as a fraction of your average framerate, so for the FPS you get you can decide whether it's going to be 1/2 or 1/4 or so on of that... this would have the added advantage of exponentially gaining back FPS the lower your FPS goes. There are several reasons why I believe this would be better than mappers manually placing new probe entities: Extra work is required for the mapper, who needs to figure out how to cover each area in probe lights. Every piece of the map would need to be encompassed in a reflection / GI probe otherwise you won't get shiny surfaces or bounce lighting which will look out of place. Most existing FM's will never be updated to use this: Only maps created or updated after the feature is introduced would benefit, anyone playing old missions will get boring visuals without reflections / GI which will be inconsistent to new ones. I strongly believe this should be done as an universal effect like SSAO. Single large probes will produce inaccurate results; The larger a cubemap is, the more drift and a fake results you get with distance from its center. This can be mitigated by using parallax corrected cubemaps which should be used for automated cubemaps too... none the less you get a single point of vision for a large room which makes the result inaccurate the further you go... with an automated approach you could have many probes in a dense grid (if your hardware allows it) for a much more accurate result at any position and angle. What are your thoughts on this solution, do you think it's realistic and can work out? I do believe it should be either this or screenspace reflections the way they're done in Godot or Blender / Eevee. If SSR isn't the right choice for our engine, reflections and global illumination alike could be captured using a global grid of capture points shining within their respective areas.

https://www.twitch.tv/videos/1714572974 Replayed on stream on TDM 2.11 Beta 5, was fun to go through it again

Interesting. I tried again to be sure: In my case on Wayland even with a fresh config directory DR will always have the issue on Wayland (or X11 under Wayland in the cases I described).

In my case I believe I tried a fresh config and same thing. It's interesting that on Linux it seems to be strictly an X11 vs Wayland issue, whereas on Windows the exact same thing happens at random: Likely means there's a bug somewhere in mouse input handling that's only reached in a few rare circumstances triggered in different ways by the OS.

I forgot which FM I saw it on. I know it improved by some 5 FPS or more. If I see it again I'll try taking comparison screenshots.

As a general note, 32bit increases FPS quite a bit over 64bit on some FM's for me. It's the only reason why I also use it. Only disadvantage is some ugly color banding in dark areas.

Sounds very interesting! Let us know how it goes. Something I was thinking about: It would be fun to see how TDM maps would look with raytracing... granted our engine will surely never have such a thing unless a complete technological miracle happened, importing it to Blender and rendering in Cycles could give us an interesting view As for such scripts I write them when needed which isn't all that often. Without Google (or DuckDuckGo in my case) I wouldn't be able to do it in Bash's case, both as it's more complex and I use it more rarely compared to JavaScript.

Still very relevant since a solution to our lack of reflections hasn't been found, this is the type of thread revival I find most welcome. We've been talking about PBR (whether in general or for TDM) and I've been intrigued on whether we can get proper reflections using the specular maps of textures as (reverse) roughness. With the release of 2.11 not far away I've been wanting to hope some sort of solution might make it in time even if that seems unlikely. https://bugs.thedarkmod.com/view.php?id=5239 There's currently an open ticket on implementing light probes that can automatically capture a cubemap from a given position. The technique used here would definitely come in handy for such a thing! An issue with those is mappers would need to add them manually: I'd really prefer an universal solution like SSR so it can work on all FM's without requiring the mapper to enable it, same way SSAO came as an universal improvement. Thus my first idea was rendering a realtime cubemap from the player's POV each frame or every X frames... that would brutalize performance however, especially as it would open all portals around the player to capture a 360* panorama (goodbye view culling). Also it's worth remembering we're dealing with two independent yet interconnected issues here: Ambient lighting and reflection. Ideally a single solution can tackle both, lighting walls with the ambiance of the room as well as showing the reflected view that moves around with the camera. If we were to do it right maybe we could use a smart cubemap to achieve bounce lighting / realtime irradiance... I'm probably going so far off there, still who knows

It's alright, and that is normal: You won't see a huge difference on everything, just a few surfaces typically smooth metal and stone. The goal of the experiment was to see how close I could get to realistic reflections that would match what you get from a PBR engine... obviously this has nothing to do with PBR itself, just a comparison to its feel. Like in real life only some surfaces will produce reflections: You won't see sharp reflection on your walls for instance, but will see a light one on your door knob. The trick in my approach is only those materials with a specular map are given a cubemap reflection, the reflection is masked by the specular map and works as a supplement to it; This ensures every material is only as shiny as intended by the texture pack, which makes the effect barely noticeable on surfaces with low specularity. Making everything shiny could have looked prettier in the moment but would be fake and unrealistic. Also worth noting: There are three default /env/gen# cubemaps. gen1 is very low intensity (almost unnoticeable), gen2 is the strongest (makes everything bright), gen3 is somewhere in the middle (looks best so I used it). You can change the script to gen2 but apart from making the universal cubemap too obvious it might make everything too bright by default. I prefer Python myself, Bash is weird and uses strange syntax. It's what Linux has by default so I got used to thinking it's best for stuff like editing files, especially as I also need to zip / unzip archives. Something like this is easy to do though, if there was real need it could be easily redone in Python or Lua or NodeJS. That would be ideal and what I wanted, either something like this with light probes or actual SSR (screen space reflections). Unfortunately it's not currently possible with existing capabilities: I spent the first day trying to trick the material into using the _currentRender or _scratch pass to fake reflections, obviously it won't work. My approach is admittedly meant to be a crutch till we have that, for those of us who prefer seeing some kind of shine rather than just the boring specular reflections... both approaches are ugly and unrealistic but at the moment it's a pick between which is the least ugly.

Here's a bunch of comparisons from other FM's which I did recently. Hard to find spots where the results are obvious in an image, it's very subtle but very good where you do see it. Some bonus ones better showing the effect on guards and ice and some models: Also here's the pk4 for TDM 2.11 Beta 5 in case anyone has issues with the script, just drop it in the base darkmod directory. It should be rebuilt after every update of course. Z-tdm_materials_cubemap.pk4

I recently saw discussion about PBR materials being added to Doom3BFG with folks also talking about it on Discord. One of the things I always wanted from PBR is proper reflections, especially in a way that works on all FM's new and old without requiring changes (eg: new light entities). Lack of proper reflections is one of the biggest limitations we still have, leaving us with mere boring specular reflections lacking any detail. While currently we can't have things like metallicity or per-pixel roughness, not even the ability to use the skybox or player camera feed as a reflection map, we do have a generic cubemap used only on windows and a few special textures. So the thought itched me: What if we could make every material with a specular map also blend a cubemap reflection? I've done Linux batch scripts for complex tasks before, so after lots of searching (and dealing with DOS era line terminations getting in the way) I managed to create a bash script that will do just that! This 50 line script will detect all materials with a specular map, inject a customized cubemap reflection, then repack everything. It scans every material in TDM thus it changes all map textures models and entities alike, everything gets modified to benefit from this. Modifications are NOT made to the official pk4 files, instead a single pk4 named Z-tdm_materials_cubemap.pk4 is generated to override the old defs, you can revert at any time simply by removing this one file. The cubemaps are subtly blended in without using extra vertex / fragment shaders which should have minimal performance impact, they also respect the bump map of the material and are deformed by it... each cubemap is masked by the specular map which gives it a close feel to PBR, materials without a specularity texture are considered rough and remain unchanged. Simply download the script and use it in your TDM folder, you'll need either Linux or a bash environment on Windows (untested): material_cubemaps.sh #!/bin/bash # Add cubemap reflections to all TheDarkMod materials containing specular maps # Use sub(/\r$/,"") to fix DOS line termination, https://stackoverflow.com/questions/45772525/why-does-my-tool-output-overwrite-itself-and-how-do-i-fix-it # Unpack all materials mkdir "temp" for f in *.pk4; do unzip -o $f "materials/*" -d "temp" done mv "temp/materials" "temp/mtr" mkdir "temp/materials" cd "temp/mtr" # Inject cubemap code into all materials with specular maps, the reflection is masked by specular intensity # First ensure the file contains at least one definition that need to be modified to avoid needless repacking for f in *.mtr; do awk '{ sub(/\r$/,"") if($1 == "specularmap" && $2 != "_black" && $2 != "") exit !f }' "$f" if [ $? == 1 ]; then awk '{ sub(/\r$/,"") print $0 if($1 == "specularmap" && $2 != "_black" && $2 != "") { print "" print " // Cubemap reflection for specularity" print " {" print " maskcolor" print " map makealpha(" $2 ")" print " }" print " {" print " blend gl_dst_alpha, gl_one" print " maskalpha" print " cubeMap env/gen3" print " texgen reflect" print " }" } }' "$f" > "../materials/$f" fi done # Pack modified materials cd ".." zip -r "../Z-tdm_materials_cubemap.pk4" "materials" rm -r "../temp" At the moment I haven't done a full comparison and only tried it out on a map I'm working on: It's possible I might do my next TDM stream with this on which will allow others to see it better. From what I'm noticing it's pretty much perfect: Very subtle and doesn't disrupt visually, it does improve realism in a lot of cases as you move around and see the shine... given the texture is almost always masked and distorted by bump you don't feel the reflection is ugly and fake but it feels natural. Currently this is here as a mod for players that wish to use it... not gonna lie part of me is tempted to suggest it be considered for 2.11, it's definitely an improvement to having no reflections at all until a better method is found. Please try it out and share your own thoughts and images, below are a few screenshots I took during my first test to confirm it works as expected.

A little thing I'd like to see changed: When renaming a readable definition in the Readable Editor, a copy of the old definition is created instead of moving the old one. This leads to clutter that needs to be fixed manually in a text editor if the mapper wants such duplication removed. It would be appreciated if a way to properly rename readable paths was supported. Here's what I mean: Add and select a readable entity on the map, such as atdm:readable_immobile_paper01. Open the Readable Editor and type something, then under XData Name save it under an unique name like readables/myfm/note_foo. Click the Save and Close button to store any changes. Open the Readable Editor again and under XData Name rename your entry to something else like readables/myfm/note_bar. Save and Close once more. Issue: If you open darkmod/fms/myfm/xdata/myfm.xd you will find that both readables/myfm/note_foo and readables/myfm/note_bar exist and duplicate the same text body, instead of the first being renamed into the second. A problem I can see is that a readable definition might be used by multiple entities on the map, so when renaming we must ensure every entity with the same xdata_contents is changed to point to the new path. Just in case a prompt might be desired if a rename of the XData path is detected, asking whether you want to copy or move the definition and warn that if any other maps use it they'll need to be pointed to the new name manually.

Keyhole peek crash bug: I'm sorry for not reporting it earlier! I first noticed it years ago in fact, but always thought it's just a fluke, especially as I never thought it could be platform specific. Can confirm and hopefully will be fixed before 2.11 is out.

After doing more testing and practically using DR under Wayland, I uncovered more bizarre functionality to this issue. Even when using the parameter GDK_BACKEND=x11 to run Radiant through X11, the same input issue will still occur in some cases. First of all the model chooser will always have it: If you choose to add a new model or edit the model of a selected entity, right-clicking in the 3D view of the Choose Model window to look at the mesh from different angles causes the mouse to wildly throw the camera around only allowing for small movements. Then I noticed the 2D view in the main window will sometimes start doing it too, though unlike the model browser it doesn't always have the problem and something triggers it (on X11 backend, full Wayland always does). Finally I was able to see what it is: After pressing X to select the clipper tool and cutting a brush, click-dragging the 2D view always produces the same exaggerated movement. Switching away from he clipper doesn't fix it once the issue has been triggered, right now I need to constantly restart DarkRadiant after using the clipper to cut a brush or I can't control the 2D view any more.

Looks as wonderful as it gets! I take it we're using particles that don't face the camera, which is the best approach considering they're the least performance intensive option. Only issue I remember with particle grass was particles not respond to lighting accordingly... one of the videos shows a guard with a lamp though, the grass lights up properly so I take it that's been solved as well Only extra feature I'd say is needed would be a system to use patch terrain as the emission source. I take it that doesn't work yet and you can't get a particle distribution like the seed brush / entity... or does it? If not they'll have to be placed on flat terrain, or each entity aligned to bumps and holes in the ground manually.

How many arrows were there in total? We have numbers 0 to 9 so 10 in total. 1 is Blackjack 2 is Sword and the other 8 would be arrows: We have the standard arrow, water, fire, moss, gas, noisemaker, rope, vine... yeah that would be all 8 then, I thought there would be 6 or 7. I guess it could go on another keybind like - or = or not have a specific bind at all. I was thinking about it, haven't released new FM's but would like to include this arrow when I do. Problem is I'd need to add it as a mod, and this requires a custom player def that might get broken later due to how weapon slots are defined since the Doom 3 days (also why we have a 16 weapon limit though that we're far from breaching). Like I said it can work alongside the flashbomb but is rather different in its latest form so I don't think there's a conflict. Mappers can either place only the bomb, only the arrow, or both as they don't really obsolete each other.

Thank you as well for the kind words! I believe the last update had indeed turned it into a flare arrow as well, after other players offered feedback and suggested this would make it even more useful which I agree with. If I remember correctly, the latest version causes the same effect as the flash bomb if the arrow is shot in a guard's face, otherwise you can also shoot it at walls to illuminate the surrounding area for some time. I'd still be very happy if this could be added to vanilla, given it feels like it will nicely complete the set of arrows with special functions we already have by adding new functionality which definitely has a purpose. With all due respect for the poor and misunderstood vine arrow, that still exists although to this day I never used one despite playing the vast majority of FM's... I'm sure the flash arrow would have a place especially with circumstantial inventive to use it It's a map item so no one who considers it useless needs to have it on their map. I believe the initial complaint was that it's too similar to the flash bomb: The goal was to have an arrow version of it, but now that I added the flashlight functionality to it the arrow also serves an unique purpose making it much more different. With 2.11 in beta and probably releasing in a few months, I would kindly ask the team to take a look and see if adding this is a possibility. I especially don't want it to remain a mod because it requires duplicating and editing the player def to add the new arrow to the weapon list of the player, meaning FM's doing it may break the player def later down the road by forcing an outdated player entity. I presume nothing broke and the same defs / scripts / etc should still be working so feel free to give it a try! I believe 1.3 was the last pk3 containing the new functionality including a fairy-dust like effect:

God I wish. PBR in TheDarkMod sounds too good to be true: I accepted the idea graphics may never improve past a certain point. Hearing about this now though... the need hurts! I know there's at least one challenge to proper PBR: Many textures will need a metallicity map they were never given upon design. The good news is that's particularly ones who combine metal with non-metal surfaces... those that are all metal or non-metal can use a simple yes / no shader setting I imagine. Specular textures would need to be inverted and used for roughness which shouldn't be a big deal. Main benefit I believe would be having proper reflections; We still use a simple specularity system, anything else currently requires a cubemap blended in with marginally better light probes being considered. Oh: When I clicked reply on your post I saw December 2021 but somehow my brain interpreted it as being last month forgetting it's 2023 already. Already wrote my reply at that point and since it still feels relevant why not bump this: Would definitely love an answer as to whether porting this system from BFG may be an option

Video pleaaaase! I'm also curious to see a performance assessment. Fingers crossed this makes it in time for 2.11 so the new version benefits from having it integrated, which will obviously mean more maps using it may happen sooner. Makes me wish we had a wind system similar to Red Eclipse, where wind can be an entity affecting all particles and plants with compatible procedural deformations. That's another story of course, good to be happy for what we have now

I'm working on a bunch of maps and have a new one more than halfway done. Finishing a map to completion is a difficult task, it's why I always wished DarkRadiant could automate more things though it's doing the best it can.