DarkRadiant 2.10.0 pre-release test

[greebo]

Here's the DR 2.10.0pre3 build for every interested mapper to test. It introduces a couple of new features that might increase the average mapper life quality.

Feature: Patch Merging/Welding

Feature: Maps can be opened from PK4s

Selecting File > Open Map from Project... one can select a .map file that is saved in any PK4 of the current project or from a completely separate PK4 somewhere in the file system:

And one can select the font size of the OrthoView grid in the preferences now:

For more things that have changed or fixed, see the list below.

Download Windows Portable x64: https://drive.google.com/file/d/1Q4iCbF--lZYu5p57GWkNM2xaI7vI8LWU/view?usp=sharing
Download Windows Installer x64: https://drive.google.com/file/d/1rKc6ansoAY7Subf70GV5XYsyQTrl7b6D/view?usp=sharing

Linux folks need to compile this stuff from source, instructions for various distributions are on the wiki.

If you happen to run into a crash, please record a crashdump: How to record a crashdump

Changes since 2.9.0 can be seen on the Bugtracker changelog, here's the summary:

#5436: Feature request: User control of font size of "size" numbers on ortho screen
#5382: Merge patch connect patches
#5108: Allow DR to open .maps that are inside .pk4s
#5309: Creating a model entity creates a "phantom light" near the world origin
#5425: Automatic map saving can crash DarkRadiant
#5097: Layer toggle controls can get out of sync (showing wrong icons)
#5424: Up/down strafe movement not working in camera free look mode
#5426: Can't build Radiant on current arch due to pybind breaking change
#5429: EntityClassManager vs. VirtualFileSystem initialisation order might prevent .def file parsing
#5430: World geometry is blue and cannot be changed when no worldspawn entityDef is defined
#5442: Expose LayerManager interface to Python scripts

Changes since 2.10.0pre1

#5382: Some patch welding improvements to have the merged patches face in the expected direction
#5364: Light colours in the ortho view can be overridden by the colour specified in the scheme
#5452: Update pybind11 version to 2.6
#5448: 16-Bit PNGs and 8-Bit Greyscale PNGs with Alpha channel can be loaded correctly (again)
#5440: DarkRadiant now warns before saving in case the map file has been modified in the meantime

Changes since 2.10.0pre2

#5451: Creating/renaming/deleting a layer didn't flag the map as modified
#5443: Selected light bounds not updated when editing light_radius spawnarg
#5406: Add dropdown list of all map entity names when editing a "Name of single entity" of an Objective component
#5406: Double-clicking an objective will open the editor. Objectives Dialog will automatically select the first objective entity now.
#5444: Brushes were not selectable in orthoview when none of its faces were facing upwards

Thanks for testing, as always!

[IZaRTaX]

Oh yes I'm so excited to try it! Thank you Greebo

[IZaRTaX]

Merging patch function works better than I expected what a very good job you've done here !

[greebo]

20 minutes ago, IZaRTaX said:

Merging patch function works better than I expected what a very good job you've done here !

Thanks, glad you like it.

[OrbWeaver]

Something weird happened when I tested the patch welding (which overall looks like a great feature). I made two rectangular patches in the top view, welded them together and the patch seemed to disappear from the 3D view. It turns out the orientation was flipped, so I had to move the camera down and look up at it to see the texture. Is it possible that the welding code is either flipping or ignoring (resulting in undefined behaviour) the patch normals?

[IZaRTaX]

3 minutes ago, OrbWeaver said:

Something weird happened when I tested the patch welding (which overall looks like a great feature). I made two rectangular patches in the top view, welded them together and the patch seemed to disappear from the 3D view. It turns out the orientation was flipped, so I had to move the camera down and look up at it to see the texture. Is it possible that the welding code is either flipping or ignoring (resulting in undefined behaviour) the patch normals?

Press CTRL+I to invert patch that'll fix the issue

[greebo]

3 hours ago, OrbWeaver said:

Is it possible that the welding code is either flipping or ignoring (resulting in undefined behaviour) the patch normals?

I saw this happen a lot too, the merging code is not caring about orientation as of now. The algorithm is merely looking to match up any edge of the two candidates in any direction - I guess the reversal happens in the case the code needs to flip around the other patch for its vertices to match.

But I agree that if both patches are facing upwards, the result should be facing upwards too, it shouldn't counter-intuitively reverse things. It's either a bug in the code or I have to add some steps to preserve the orientation, I just need to look into it.

edit: found a way to improve things, the patch orientation should be preserved now.

 

[grayman]

I'm experiencing serious lag when moving around in the map.

Any settings that can be used to speed this up?

[greebo]

To be honest, I haven't looked into performance at all. I take it that 2.8.0 or 2.9.0 didn't have these lags?

It should probably be visible in the profiler, if it's that bad.

[greebo]

Can't spot anything huge in the profiler for the orthoview, that's dominated by the display list and patch wireframe rendering, as usual.

But the camera renderer has a quite large portion of light intersection handling going on (this is not in lighting mode, just regular fullbright):

Maybe that step isn't all that necessary when doing regular fullbright rendering?

[OrbWeaver]

9 minutes ago, greebo said:

But the camera renderer has a quite large portion of light intersection handling going on (this is not in lighting mode, just regular fullbright):

Maybe that step isn't all that necessary when doing regular fullbright rendering?

No, it shouldn't be necessary at all. That seems like an oversight on my part. We only need to calculate the light intersections in the lighting preview mode.

Even when we are in lighting mode, that 35% worries me (since it happens in every frame). I did do some performance testing with the new lighting code, but didn't see any significant regressions. Maybe the performance characteristics are very different on Windows vs Linux.

[greebo]

There's a lot of this going on, it evaluates the Light Entity AABB for every check afresh, this takes alone 5% of all time spent on all cam rendering:

Also, it seems includePoint() is really expensive. That check could be refactored to check the two radii first and choose the larger one. But I think most of it can be saved by buffering the m_doom3AABB member - it doesn't have to be recalculated for each test.

Maybe we don't need localAABB() at all and can use lightAABB() instead? I think that one is cheaper.

 

[greebo]

VectorLightList is also doing a lot of reallocations (4.8% of the render footprint) - while this might be more strongly pronounced in the STL Debug build I've been profiling than it would be in a release build, I guess it'd be better to initialise the vector with a larger capacity to start with.

[grayman]

I wasn’t able to get back to the computer last night after filtering out items one by one. when I filtered out lights, performance went back to the quicker 2.08 speed.

So I’m glad to see that you’re focused on lighting.

I don’t use advanced camera-view lighting, btw. Just the basics.

[OrbWeaver]

9 hours ago, greebo said:

Maybe we don't need localAABB() at all and can use lightAABB() instead? I think that one is cheaper.

Yes, that seems plausible. We need to be careful though because there are at least two different situations where AABBs are needed — lighting calculations and the selection system — and the AABBs they need are not identical. But I'm sure there is a bit of redundancy from the legacy code that we could simplify.

9 hours ago, greebo said:

VectorLightList is also doing a lot of reallocations (4.8% of the render footprint) - while this might be more strongly pronounced in the STL Debug build I've been profiling than it would be in a release build, I guess it'd be better to initialise the vector with a larger capacity to start with.

Also a good suggestion. We could default initialise it with a "sensible" initial light count like 16, or even try std::list which should have constant time insertions at the expense of slow random access (which is not needed).

In any case, as of a8a19bec7bce6 I have disabled the call to calculateLightIntersections() when in non-lit mode, which should save a few wasted cycles.

[peter_spy]

No love for this one, eh? https://bugs.thedarkmod.com/view.php?id=5364

I guess this one needs more votes

[greebo]

Glad you asked, I'm running out of work anyway

[OrbWeaver]

On 12/14/2020 at 11:10 AM, greebo said:

Maybe we don't need localAABB() at all and can use lightAABB() instead? I think that one is cheaper.

This is implemented in e649657d774fc05b.

I actually removed Light::intersectsAABB entirely; now all of the intersectsLight methods just do a simple AABB intersection test between Light::lightAABB and Node::worldAABB, which is much cheaper than the complex long-winded code in Light::intersectsAABB.

I now see relatively little time spent in the lighting intersection calculation, and a much larger amount of time spent in the backend render method of Winding. I suspect this could be improved by the use of OpenGL VBOs to store the winding data, rather than submitting it repeatedly using slow, deprecated OpenGL functions.

[greebo]

Nice!

[jonri]

Built and starts up fine for me so far on Linux, but I did notice the PKGBUILD hasn't been updated for cmake yet.  I can fix that up for you guys tonight if you want.

[OrbWeaver]

Sure, I don't run whatever distro uses PKGBUILD so I never touch it myself, but I'll accept patches.

[OrbWeaver]

On 12/15/2020 at 6:12 PM, peter_spy said:

No love for this one, eh? https://bugs.thedarkmod.com/view.php?id=5364

I guess this one needs more votes

I'm happy to work on this one assuming @greebo hasn't started on it himself. I also preferred the single-colour light volumes so I would definitely like an option to disable the per-light colouring, and this ought to be pretty easy to implement.

[jonri]

I've noticed that the python scripts are no longer found (on Linux at least, it might be fine on Windows):

CMake will install the scripts to PKGDATADIR:

# Install scripts
install(DIRECTORY install/scripts DESTINATION ${PKGDATADIR}
        FILES_MATCHING PATTERN "*.py")

ScriptingSystem.cpp looks for the scripts in PKGLIBDIR:

#if defined(POSIX) && defined(PKGLIBDIR)
	_scriptPath = std::string(PKGLIBDIR) + "/scripts/";
#else
	_scriptPath = ctx.getRuntimeDataPath() + "scripts/";
#endif

Changing the install location to ${PKGLIBDIR} should take care of this.

[jonri]

6 hours ago, HeresJonny said:

Built and starts up fine for me so far on Linux, but I did notice the PKGBUILD hasn't been updated for cmake yet.  I can fix that up for you guys tonight if you want.

 

4 hours ago, OrbWeaver said:

Sure, I don't run whatever distro uses PKGBUILD so I never touch it myself, but I'll accept patches.

 

I went to take a look and realized there has to be a release tag in Github first.  I'll let Greebo take care of it, he's already been maintaining that file consistently anyways

[greebo]

I usually update that file when I'm getting close to a release. Good to know that this file is actually noticed by someone though.