Limit CPU threads for compilation (Windows)

[CodeHD]

Is there a way to limit the number of CPU threads during compilation?
Currently, it makes the system quite unresponsive for a few minutes. I can live with 8 min compile time instead of 5, if I can use the computer in the meanwhile.

[Fox]

Is this not the line:
set CPUCOUNT=/maxcpucount
https://github.com/LuxCoreRender/Window ... 64.bat#L16

[CodeHD]

if I set something like set CPUCOUNT=4, it just aborts after a few seconds. would that be correct syntax though? I can't find anything about this via google.

[Dade]

if I set something like set CPUCOUNT=4, it just aborts after a few seconds. would that be correct syntax though? I can't find anything about this via google.

It is a command line option to add, just:

Code: Select all

cmake-build-x64.bat 4

It should work looking at the .bat.

[acasta69]

The syntax is similar to the msbuild one:

Code: Select all

cmake-build-x64.bat /cpucount:n

It limits the number of msbuild threads to n.
Note that msbuild can still use several processes at compile time, using up to 100% cpu.
But link processes, the ones that eat up a lot of memory and probably make system unresponsive, are then limited.
On my 16GB PC I usually use 2 threads, with 4 I have no benefit.

[CodeHD]

Thanks for the help, now I was able to improve performance and also understand the relevant part of the script. (one problem in my earlier attempts was that - working mostly with python - I misinterpreted the brackets in the comment "/cpucount[:n] specifies ..." )

The scaling of msbuild still remains a mystery to me. On my 6-core i5:
no option: 6 msbuild and 6 linkers spawn, system very unresponsive, compile time ~4:30
/cpucount:2 : 6 msbuild and 3 linkers, system slightly laggy at times but workable, compile time ~5:00
/cpucount:1: 1 msbuild and 1 linker, everything smooth, compile time ~8:30

[Fox]

On my machine the linker one in Linux, with all cores and threats, crashes sound driver, when youtube music plays in background, but also the video freezes. My graphics card does not have hardware youtube video decoder.

In windows it's okay, some lag still.

[acasta69]

The scaling of msbuild still remains a mystery to me.

I suspect it is for everyone outside MS
CL.exe launches its own processes to use the available CPU threads, but it's the only compiler front end to behave like that, and msbuild relies on that.
So for example the Clang integration in Visual Studio is extremely slow because not enough compilation processes are used.
(There was work done in Clang to include that functionality, but I think it's not been released yet.)
They say they would like Visual Studio to be the IDE for as many as possible toolchains, but non-MS ones have a big penalty.

Sorry for the OT...

[alpistinho]

When I am on Windows I always disable the LTCG flag (and also the Gy too, I think).
The issue is the linking-time optimization.

[acasta69]

When I am on Windows I always disable the LTCG flag (and also the Gy too, I think).
The issue is the linking-time optimization.

Indeed, that's what makes the link stage so slow and memory-hungry.
But, with that disabled, the resulting executable is visibly slower.