Can't compile latest GIT version

[AluminumHaste]

I get errors:

Severity    Code    Description    Project    File    Line    Suppression State
Error    C1083    Cannot open include file: 'gtest/gtest.h': No such file or directory    Tests    C:\src\darkradiant_src\test\Basic.cpp    6    
Error    C1083    Cannot open include file: 'gtest/gtest.h': No such file or directory (compiling source file ..\..\..\test\Brush.cpp)    Tests    C:\src\darkradiant_src\test\RadiantTest.h    3    
Error    C1083    Cannot open include file: 'gtest/gtest.h': No such file or directory (compiling source file ..\..\..\test\Camera.cpp)    Tests    C:\src\darkradiant_src\test\RadiantTest.h    3    
Error    C1083    Cannot open include file: 'gtest/gtest.h': No such file or directory (compiling source file ..\..\..\test\ColourSchemes.cpp)    Tests    C:\src\darkradiant_src\test\RadiantTest.h    3    
Error    C1083    Cannot open include file: 'gtest/gtest.h': No such file or directory (compiling source file ..\..\..\test\CSG.cpp)    Tests    C:\src\darkradiant_src\test\RadiantTest.h    3    
Error    C1083    Cannot open include file: 'gtest/gtest.h': No such file or directory (compiling source file ..\..\..\test\Favourites.cpp)    Tests    C:\src\darkradiant_src\test\RadiantTest.h    3    
Error    C1083    Cannot open include file: 'gtest/gtest.h': No such file or directory (compiling source file ..\..\..\test\Entity.cpp)    Tests    C:\src\darkradiant_src\test\RadiantTest.h    3    

 

Indeed that folder doesn't exist, it also isn't in the repo

https://github.com/codereader/DarkRadiant/tree/master/test

Is it supposed to be part of Windows and I didn't set up VS2022 properly?
I've re-download the dependencies using the python script.

Am I doing something wrong?

[greebo]

The master branch on Github should be compiling fine with the dependencies that it downloads using the Python script (either automatically or manually, as you did).

I haven't touched VS2022 yet, as there's only just a Preview release of it. It's not unlikely that the new version is causing the trouble.

[AluminumHaste]

Is the gtest.h part of the dependencies?

[greebo]

No, I suppose it's part of the Visual Studio test adapter that can be selected in the VS Installer:  https://docs.microsoft.com/en-us/visualstudio/test/how-to-use-google-test-for-cpp?view=vs-2019

Entirely possible that the compilation guide is lacking that piece of information. It's only used for the unit tests though, so you could in theory unload the Tests project before compiling.

[AluminumHaste]

I checked the installer for individual components, but I don't see that I'm missing anything obvious.

[AluminumHaste]

I do have Test Adapter for Google Test installed at least VS's installer shows that it's on.

[greebo]

Well, maybe the path of the header file got changed, who knows.

I'll have a look at VS2022 sooner or later, but right now I don't feel like it's urgent since it's not even released yet.

[AluminumHaste]

Np,  I'm doing a repair right now as I uninstalled the google test adapter, but it's always back in there when I check

[AluminumHaste]

Ok I did a repair and now the Google Test Adapter is showing as not installed.

So I ran the installer again, checked the box to install the GTest adapter, it shows as installed now but still fails to find it.

 

What's weird, is that gtest.h EXISTS in the darkradiant source folder tree.

C:\src\darkradiant_src\tools\msvc\packages\Microsoft.googletest.v140.windesktop.msvcstl.static.rt-dyn.1.8.1\build\native\include\gtest\gtest.h
C:\src\darkradiant_src\tools\msvc\packages\Microsoft.googletest.v140.windesktop.msvcstl.static.rt-dyn.1.8.1\build\native\include\gtest\internal\custom\gtest.h
 

 

[AluminumHaste]

Welp, not anymore, I deleted those files and now I can't get them back lol.

Oh well, it compiled fine if I exclude the test project from the build.

[AluminumHaste]

Interesting, if I use the NuGet package manager, it shows that the google test is not installed, so installed it, then there was an update, added it to the solution.

Still fails with can't find source file.

20>C:\src\darkradiant_src\test\RadiantTest.h(3,10): fatal error C1083: Cannot open include file: 'gtest/gtest.h': No such file or directory (compiling source file ..\..\..\test\FileTypes.cpp)
20>C:\src\darkradiant_src\test\RadiantTest.h(3,10): fatal error C1083: Cannot open include file: 'gtest/gtest.h': No such file or directory (compiling source file ..\..\..\test\ImageLoading.cpp)

 

 

C:\Users\XXXXXX\.nuget\packages\microsoft.googletest.v140.windesktop.msvcstl.static.rt-dyn\1.8.1.4\build\native\include\gtest\internal\custom\gtest.h
C:\Users\XXXXXX\.nuget\packages\microsoft.googletest.v140.windesktop.msvcstl.static.rt-dyn\1.8.1.4\build\native\include\gtest\gtest.h
C:\src\darkradiant_src\tools\msvc\packages\Microsoft.googletest.v140.windesktop.msvcstl.static.rt-dyn.1.8.1.4\build\native\include\gtest\internal\custom\gtest.h
C:\src\darkradiant_src\tools\msvc\packages\Microsoft.googletest.v140.windesktop.msvcstl.static.rt-dyn.1.8.1.4\build\native\include\gtest\gtest.h

 

I tried including the packages folder in the includes for the tests project but it still couldn't find that source file.

I ended up copying the gtest folder from "C:\src\darkradiant_src\tools\msvc\packages\Microsoft.googletest.v140.windesktop.msvcstl.static.rt-dyn.1.8.1.4\build\native\include\" over to "C:\src\darkradiant_src\test" folder and that got much further.

Got all the way to the end and Vector.cpp fails to compile with the same damn error:

1>Vector.cpp
1>C:\src\darkradiant_src\test\math\Quaternion.cpp(1,10): fatal error C1083: Cannot open include file: 'gtest/gtest.h': No such file or directory
1>C:\src\darkradiant_src\test\math\Plane3.cpp(1,10): fatal error C1083: Cannot open include file: 'gtest/gtest.h': No such file or directory
1>C:\src\darkradiant_src\test\math\Vector.cpp(1,10): fatal error C1083: Cannot open include file: 'gtest/gtest.h': No such file or directory
1>C:\src\darkradiant_src\test\math\Matrix4.cpp(1,10): fatal error C1083: Cannot open include file: 'gtest/gtest.h': No such file or directory

[greebo]

Don't bang your head against this, unless you really want to. You don't strictly need the unit test project to use DR you compiled from the sources.

Sounds like VS2022 has some minor difference in the google test handling, the DR solution probably has to be adjusted for that.

[duzenko]

DR does not start for me when compiled from master. It just exits immediately

@stgatilov@OrbWeaver@greebo Anything obvious I could have missed?

The exception message is "Cannot find the main module in any of the paths: C:/dev/tdm/drsrc/install/modules/; C:/dev/tdm/drsrc/install/plugins/"

EDIT: Apparently I need to build all the projects manually. I assumed they were included as project dependencies

Spoiler

 

[greebo]

When you hit Build Solution, everything should be compiled automatically, including the Download of windows dependencies. That's also what happens on the build server too, something must be different in your working copy. This is the same for every configuration:

The DarkRadiantCore module is what was missing, but I can't see why it shouldn't build when you hit Build Solution. Did you just hit F5 to run the UI binary? That might perhaps do it.

[stgatilov]

I think he built one project only (most likely DarkRadiant), and some core modules did not build.

You can say that DarkRadiant project depends on SomeCoreModule project in the solution, so that the latter one will build automatically if you build the former one.

UPDATE: On my daily job, there are several hundred projects in the solution, and some of the projects are intended for dynamic loading (i.e. you can load them in some tests which need them but not load in the others). So if I build only one project that I need, I have to later track down more projects that are used in the particular test, since dependencies are missing. And if I build the whole solution, then I can just go get some tea/coffee chat or pull out my sword... Unfortunately, inter-project dependencies are often not enough.

[greebo]

Yeah, I've seen a few of those mega-solutions, they are rarely nice to handle. They also tend to cause problems with plugins like ReSharper, which severly slow down your VS installation.

23 minutes ago, stgatilov said:

You can say that DarkRadiant project depends on SomeCoreModule project in the solution, so that the latter one will build automatically if you build the former one.

I can set the DarkRadiant (UI) project to rely on the DarkRadiantCore project, but I deliberately chose not to do that since I want the builds to work in parallel - and technically it wouldn't be true, the UI project really doesn't need the core project to compile and link. Setting the dependency would make the solution wait for the largest core project to have finished before it starts building the second-largest project - waste of time. I always hit Ctrl-Shift-B to build the solution, and I figured everybody would do the same - but I might be wrong on that assumption!

[OrbWeaver]

52 minutes ago, greebo said:

I can set the DarkRadiant (UI) project to rely on the DarkRadiantCore project, but I deliberately chose not to do that since I want the builds to work in parallel - and technically it wouldn't be true, the UI project really doesn't need the core project to compile and link.

Would it work to have a top level "meta project" which doesn't build anything itself but just depends on everything that is needed (including both UI and Core module projects), and have this meta project as the default? This would presumably ensure that by default you build everything, but if you choose to build just the UI it doesn't require the core module (since there is no actual build-time dependency).

I'm not an expert on Visual Studio projects of course so this may be a completely daft or unachievable idea.

[stgatilov]

1 hour ago, OrbWeaver said:

Would it work to have a top level "meta project" which doesn't build anything itself but just depends on everything that is needed (including both UI and Core module projects), and have this meta project as the default?

It is pointless: just tell people to build the whole solution.
In fact, most people usually do it by default.

[greebo]

1 hour ago, OrbWeaver said:

Would it work to have a top level "meta project" which doesn't build anything itself but just depends on everything that is needed

That's what effectively the solution and its configuration represents. It defines what projects should be built, like shown in the screenie above.

DR doesn't do this, but one can define more configurations than just "Debug" and "Release", and specify which projects should be built and which should be skipped.

IIRC, the TDM solution is using Configurations, to skip the compilation of the MayaImport DLL by default (which needs the SDK installed to compile and link successfully). It's a bit like specifying all these ./configure switches --enable-darkmod-plugins, just not as freely combinable.

[duzenko]

Alright, that makes sense

I want to look into the Surface Inspector window and make it resize (down) nicely

Biker complained that this is a regression in the latest version

Is that ok?

[greebo]

That's already fixed, at least I'd like to believe that: https://bugs.thedarkmod.com/view.php?id=5706

But I appreciate your readiness to look into it, helping hands are always welcome.

[duzenko]

5 hours ago, greebo said:

That's already fixed, at least I'd like to believe that: https://bugs.thedarkmod.com/view.php?id=5706

But I appreciate your readiness to look into it, helping hands are always welcome.

I just created three UI/UX bugs in the tracker. While I'm practically zero skilled in DR they feel like immediate improvements.

OTOH they could be intended operation? I can work on each of them if green lighted

[greebo]

I've confirmed two of them right away, feel free to have a go at them!

Not so sure about the Ctrl+S one - I can confirm that this is not intended behaviour, but I'm not sure if there's an easy non-hacky fix for that.

[duzenko]

On 9/9/2021 at 5:27 PM, greebo said:

I can set the DarkRadiant (UI) project to rely on the DarkRadiantCore project, but I deliberately chose not to do that since I want the builds to work in parallel - and technically it wouldn't be true, the UI project really doesn't need the core project to compile and link. Setting the dependency would make the solution wait for the largest core project to have finished before it starts building the second-largest project - waste of time. I always hit Ctrl-Shift-B to build the solution, and I figured everybody would do the same - but I might be wrong on that assumption!

Apologies for going in circles around this, but

What is the reason again to build projects in parallel?

Other than ambiguity in project dependencies, there seems to also exist RAM consumption issue on modern many-core CPU's. I mean I understand that a semi-pro programmer is expected to have "enough" RAM (and I do) but it's still a real minus. On my day job I've seen a colleague to have 8GB RAM and 20+GB committed because of many garbage-collected java/dart/javascript processes running in background. Surely we don't want the compiler to start swapping pages to disk in the middle of build on a "regular" laptop?

Are you trying to utilize CPU cores in this way? But e.g. the notools darkmod build is just one project and it loads all cores nicely. And please @stgatilov correct my on this assumption but isn't building multiple projects theoretically slower because of all the data the CPU has to keep pumping along the L3cache <> RAM bus?

[stgatilov]

33 minutes ago, duzenko said:

What is the reason again to build projects in parallel?

Are you trying to utilize CPU cores in this way?

Yes, exactly.

Of course, there is /MP to compile cpp files in parallel, but it does not always fill the CPU in practice, so building several projects at once is not useless.

Quote

Other than ambiguity in project dependencies, there seems to also exist RAM consumption issue on modern many-core CPU's. I mean I understand that a semi-pro programmer is expected to have "enough" RAM (and I do) but it's still a real minus.

That's the real of problem of Visual Studio: given many projects and cpp files to build on T-core hyperthreaded CPU, it builds 2T projects simultaneously, and spawns 2T compilers per project to build cpp files in parallel. This gives you 4T^2 compilers at once, e.g. 144 on a six-core machine.

Normally, compiler does not take too much RAM, but 1) overly c-plus-plus-y projects need more RAM (Eigen, I'm looking at you), and 2) 144 instances eat gigabytes.
I even had to increase amount of RAM from 16 GB to 32 GB on my work machine, because when it depletes its RAM with 144 memory-hungry processes, remote desktop just stops responding.

Anyway, if you have such problems on your machine, you can always limit the parallelism in Visual Studio settings.

Quote

And please @stgatilov correct my on this assumption but isn't building multiple projects theoretically slower because of all the data the CPU has to keep pumping along the L3cache <> RAM bus?

I don't see how can RAM suffer from too many requests. Also, even with 144 processes, only 12 of them are executed at once during each time quantum on 6-core hyperthreaded CPU, all the rest are sleeping. You don't get more cache pressure or RAM pressure beyond the number of threads your CPU supports, and the additional processes can kick in if the primary ones go to sleeping for some weird reason.