Limit CPU threads for compilation (Windows)
Limit CPU threads for compilation (Windows)
Is there a way to limit the number of CPU threads during compilation?
Currently, it makes the system quite unresponsive for a few minutes. I can live with 8 min compile time instead of 5, if I can use the computer in the meanwhile.
Currently, it makes the system quite unresponsive for a few minutes. I can live with 8 min compile time instead of 5, if I can use the computer in the meanwhile.
Re: Limit CPU threads for compilation (Windows)
Is this not the line:
set CPUCOUNT=/maxcpucount
https://github.com/LuxCoreRender/Window ... 64.bat#L16
set CPUCOUNT=/maxcpucount
https://github.com/LuxCoreRender/Window ... 64.bat#L16
Re: Limit CPU threads for compilation (Windows)
if I set something like set CPUCOUNT=4, it just aborts after a few seconds. would that be correct syntax though? I can't find anything about this via google.
Re: Limit CPU threads for compilation (Windows)
It is a command line option to add, just:
Code: Select all
cmake-build-x64.bat 4
Re: Limit CPU threads for compilation (Windows)
The syntax is similar to the msbuild one:
It limits the number of msbuild threads to n.
Note that msbuild can still use several processes at compile time, using up to 100% cpu.
But link processes, the ones that eat up a lot of memory and probably make system unresponsive, are then limited.
On my 16GB PC I usually use 2 threads, with 4 I have no benefit.
Code: Select all
cmake-build-x64.bat /cpucount:n
Note that msbuild can still use several processes at compile time, using up to 100% cpu.
But link processes, the ones that eat up a lot of memory and probably make system unresponsive, are then limited.
On my 16GB PC I usually use 2 threads, with 4 I have no benefit.
Re: Limit CPU threads for compilation (Windows)
Thanks for the help, now I was able to improve performance and also understand the relevant part of the script. (one problem in my earlier attempts was that - working mostly with python - I misinterpreted the brackets in the comment "/cpucount[:n] specifies ..." )
The scaling of msbuild still remains a mystery to me. On my 6-core i5:
no option: 6 msbuild and 6 linkers spawn, system very unresponsive, compile time ~4:30
/cpucount:2 : 6 msbuild and 3 linkers, system slightly laggy at times but workable, compile time ~5:00
/cpucount:1: 1 msbuild and 1 linker, everything smooth, compile time ~8:30
The scaling of msbuild still remains a mystery to me. On my 6-core i5:
no option: 6 msbuild and 6 linkers spawn, system very unresponsive, compile time ~4:30
/cpucount:2 : 6 msbuild and 3 linkers, system slightly laggy at times but workable, compile time ~5:00
/cpucount:1: 1 msbuild and 1 linker, everything smooth, compile time ~8:30
Re: Limit CPU threads for compilation (Windows)
On my machine the linker one in Linux, with all cores and threats, crashes sound driver, when youtube music plays in background, but also the video freezes. My graphics card does not have hardware youtube video decoder.
In windows it's okay, some lag still.
In windows it's okay, some lag still.
Re: Limit CPU threads for compilation (Windows)
I suspect it is for everyone outside MS
CL.exe launches its own processes to use the available CPU threads, but it's the only compiler front end to behave like that, and msbuild relies on that.
So for example the Clang integration in Visual Studio is extremely slow because not enough compilation processes are used.
(There was work done in Clang to include that functionality, but I think it's not been released yet.)
They say they would like Visual Studio to be the IDE for as many as possible toolchains, but non-MS ones have a big penalty.
Sorry for the OT...
- alpistinho
- Developer
- Posts: 198
- Joined: Thu Jul 05, 2018 11:38 pm
- Location: Rio de Janeiro
Re: Limit CPU threads for compilation (Windows)
When I am on Windows I always disable the LTCG flag (and also the Gy too, I think).
The issue is the linking-time optimization.
The issue is the linking-time optimization.
Re: Limit CPU threads for compilation (Windows)
Indeed, that's what makes the link stage so slow and memory-hungry.alpistinho wrote: ↑Sat Apr 11, 2020 12:26 pm When I am on Windows I always disable the LTCG flag (and also the Gy too, I think).
The issue is the linking-time optimization.
But, with that disabled, the resulting executable is visibly slower.