Page 1 of 2
Limit CPU threads for compilation (Windows)
Posted: Fri Apr 10, 2020 8:31 pm
by CodeHD
Is there a way to limit the number of CPU threads during compilation?
Currently, it makes the system quite unresponsive for a few minutes. I can live with 8 min compile time instead of 5, if I can use the computer in the meanwhile.
Re: Limit CPU threads for compilation (Windows)
Posted: Fri Apr 10, 2020 10:15 pm
by Fox
Re: Limit CPU threads for compilation (Windows)
Posted: Fri Apr 10, 2020 10:42 pm
by CodeHD
if I set something like set CPUCOUNT=4, it just aborts after a few seconds. would that be correct syntax though? I can't find anything about this via google.
Re: Limit CPU threads for compilation (Windows)
Posted: Fri Apr 10, 2020 11:27 pm
by Dade
CodeHD wrote: Fri Apr 10, 2020 10:42 pm
if I set something like set CPUCOUNT=4, it just aborts after a few seconds. would that be correct syntax though? I can't find anything about this via google.
It is a command line option to add, just:
It should work looking at the .bat.
Re: Limit CPU threads for compilation (Windows)
Posted: Sat Apr 11, 2020 6:17 am
by acasta69
The syntax is similar to the msbuild one:
It limits the number of msbuild threads to n.
Note that msbuild can still use several processes at compile time, using up to 100% cpu.
But link processes, the ones that eat up a lot of memory and probably make system unresponsive, are then limited.
On my 16GB PC I usually use 2 threads, with 4 I have no benefit.
Re: Limit CPU threads for compilation (Windows)
Posted: Sat Apr 11, 2020 11:09 am
by CodeHD
Thanks for the help, now I was able to improve performance and also understand the relevant part of the script. (one problem in my earlier attempts was that - working mostly with python - I misinterpreted the brackets in the comment "/cpucount[:n] specifies ..."

)
The scaling of msbuild still remains a mystery to me. On my 6-core i5:
no option: 6 msbuild and 6 linkers spawn, system very unresponsive, compile time ~4:30
/cpucount:2 : 6 msbuild and 3 linkers, system slightly laggy at times but workable, compile time ~5:00
/cpucount:1: 1 msbuild and 1 linker, everything smooth, compile time ~8:30
Re: Limit CPU threads for compilation (Windows)
Posted: Sat Apr 11, 2020 11:41 am
by Fox
On my machine the linker one in Linux, with all cores and threats, crashes sound driver, when youtube music plays in background, but also the video freezes. My graphics card does not have hardware youtube video decoder.
In windows it's okay, some lag still.
Re: Limit CPU threads for compilation (Windows)
Posted: Sat Apr 11, 2020 12:03 pm
by acasta69
CodeHD wrote: Sat Apr 11, 2020 11:09 am
The scaling of msbuild still remains a mystery to me.
I suspect it is for everyone outside MS
CL.exe launches its own processes to use the available CPU threads, but it's the only compiler front end to behave like that, and msbuild relies on that.
So for example the Clang integration in Visual Studio is extremely slow because not enough compilation processes are used.
(There was work done in Clang to include that functionality, but I think it's not been released yet.)
They say they would like Visual Studio to be the IDE for as many as possible toolchains, but non-MS ones have a big penalty.
Sorry for the OT...
Re: Limit CPU threads for compilation (Windows)
Posted: Sat Apr 11, 2020 12:26 pm
by alpistinho
When I am on Windows I always disable the LTCG flag (and also the Gy too, I think).
The issue is the linking-time optimization.
Re: Limit CPU threads for compilation (Windows)
Posted: Sat Apr 11, 2020 2:05 pm
by acasta69
alpistinho wrote: Sat Apr 11, 2020 12:26 pm
When I am on Windows I always disable the LTCG flag (and also the Gy too, I think).
The issue is the linking-time optimization.
Indeed, that's what makes the link stage so slow and memory-hungry.
But, with that disabled, the resulting executable is visibly slower.