Can't compile latest GIT version

[greebo]

 

34 minutes ago, duzenko said:

What is the reason again to build projects in parallel?

It's all about overall compilation duration for me. And yes, it is indeed a bit self-centered, since I want to use my time effectively when working on DR. I don't want to cut down my own free time just because low-end machines might get into trouble when building from source. Or do you?

On a more general note, I don't explicitly force the Visual C++ compiler to do all builds in parallel, I just enable it by setting it the flags and defining the dependencies. I shift the responsibility to not overload the machine to VC++, which I think is respecting the amount of cores and memory, or is it not?

[duzenko]

14 minutes ago, greebo said:

 

It's all about overall compilation duration for me. And yes, it is indeed a bit self-centered, since I want to use my time effectively when working on DR. I don't want to cut down my own free time just because low-end machines might get into trouble when building from source. Or do you?

On a more general note, I don't explicitly force the Visual C++ compiler to do all builds in parallel, I just enable it by setting it the flags and defining the dependencies. I shift the responsibility to not overload the machine to VC++, which I think is respecting the amount of cores and memory, or is it not?

That's what I wonder actually

is there a switch somewhere that allows/forbids to build projects in parallel? I'd benchmark that out of curiosity

I think there should be one in .sln properties, but it's not easily accessible when opening the .cmake in VS2019 like I do

EDIT. Found it in global IDE options

[stgatilov]

58 minutes ago, greebo said:

 

On a more general note, I don't explicitly force the Visual C++ compiler to do all builds in parallel, I just enable it by setting it the flags and defining the dependencies. I shift the responsibility to not overload the machine to VC++, which I think is respecting the amount of cores and memory, or is it not?

Unfortunately no.

First of all, you do set /MP flag in project settings, which means "unlimited parallelism", and results in T or 2T threads per project. If you don't set this flag on the project, then cpp-s of one project will build strictly sequentally. You can also set /MP 2 to say "build 2 cpp files in parallel". So you do force MSVC to spawn specific number of threads per project (and exact number depends on machine).
It is quite stupid that parallel builds are controlled via build settings of the project, instead of settings in .user file or user-specific settings in IDE. But that's how things work, as far as I know...

Second, Visual Studio does not control number of threads properly, just see my example about 144 compiler processes on 12-thread CPU. There Is no way to say "spawn 12 parallel processes", you can control independently 1) number of parallel projects in IDE settings, and 2) number of compiler processes per project in project settings. The total number of projects can rise to the product of these two numbers.

Perhaps something has changed recently or will change in future, I don't know.

Here is related question BTW: https://stackoverflow.com/questions/45379427/how-to-limit-the-number-of-parallel-cl-exe-processes-during-the-visual-studio-so

[duzenko]

Results here. What's puzzling me the most is relatively low CPU usage during the first minute. After that it goes to 100% and stays there until a few seconds before the end. I wonder what's going on during that first minute - maybe there's some potential in improving the overall time.

[stgatilov]

4 minutes ago, duzenko said:

Results here. What's puzzling me the most is relatively low CPU usage during the first minute. After that it goes to 100% and stays there until a few seconds before the end. I wonder what's going on during that first minute - maybe there's some potential in improving the overall time.

It can be precompiled header at the beginning, and linking at the end.

[duzenko]

13 minutes ago, stgatilov said:

It can be precompiled header at the beginning, and linking at the end.

Not sure how it fits the design pattern of parallelism.

E.g. I might have precompiled headers compiling in 2 projects (out of 4 allowed in settings)

That still leaves plenty of room for other 2 projects to load all cores twice.

Even one non-PCH process should be able to load all remaining thread pool slots. (And looking in the build output window, it says it's sifting through the .cpp's at least most of that time)

And I'm not even sure all the P*2T instances are pooling together, considering they're all running in separate processes. Are they talking to each other or something?

It's scary to live in MSVC world!