Luxcore and memory
Forum rules
Please upload a testscene that allows developers to reproduce the problem, and attach some images.
Please upload a testscene that allows developers to reproduce the problem, and attach some images.
Re: Luxcore and memory
I will check back on this, thanks for the input. This memory usage is my biggest concerns for now.
Re: Luxcore and memory
By the way, this setting is not exposed in the addon yet, it is always enabled (if you have an OpenCL build).
However, I have done all my tests on this matter so far with a Non-OpenCL build on Linux, so the OpenCL imagepipeline was not enabled.
Re: Luxcore and memory
Alright so here are some results of path vs tilepath ram usage:
scene in blender before rendering:
3.3GB mostly empty scene and cycles at any resolution gives below 4GB RAM usage
luxcore:
cpu+PATH:
256x256: 5.8 GB
denoiser: 5.9 GB
2048x2048: 8 GB
denoiser: 10.3 GB
cpu+TILEPATH:
256x256: 5.8 GB
denoiser: 5.9 GB
2048x2048: 7.2 GB
denoiser: 9.5 GB
OpenCL+PATH(using also CPU):
256x256: 6.3 GB
denoiser: 6.3 GB
2048x2048: 9.8 GB
denoiser: 12.1 GB
OpenCL+TILEPATH(using also CPU):
256x256: 6.3 GB
denoiser: 6.3 GB
2048x2048: 8.9 GB
denoiser: 11.2 GB
OpenCL+PATH(GPU only/GTX1060-6GB):
256x256: 5.2 GB
denoiser: 5.2 GB
2048x2048: 8 GB
denoiser: 10.3 GB
OpenCL+TILEPATH(GPU only/GTX1060-6GB):
256x256: 5.2 GB
denoiser: 5.2 GB - not working, black image as denoised result
2048x2048: 7.8 GB
denoiser: 10 GB - not working, black image as denoised result
So in conclusion, not only luxcore uses too much memory in general, even at low resolution, the memory usage skyrockets at 2K resolution(that's not even hi-res that's for mobile viewing) in a mostly empty scene with no textures and no hdri. Also even at 2K the denoiser performance is terribly slow. The differences between Path and TilePath were in all cases minimal and it's bad vs bad.
If you suspect problem could be elsewhere, like in the scene or luxcore usage itself. It is more or less the same version of the scene I gave Simon few days ago. Since I'm noob with luxcore please do check yourselves, I might be doing something wrong in the end.
scene in blender before rendering:
3.3GB mostly empty scene and cycles at any resolution gives below 4GB RAM usage
luxcore:
cpu+PATH:
256x256: 5.8 GB
denoiser: 5.9 GB
2048x2048: 8 GB
denoiser: 10.3 GB
cpu+TILEPATH:
256x256: 5.8 GB
denoiser: 5.9 GB
2048x2048: 7.2 GB
denoiser: 9.5 GB
OpenCL+PATH(using also CPU):
256x256: 6.3 GB
denoiser: 6.3 GB
2048x2048: 9.8 GB
denoiser: 12.1 GB
OpenCL+TILEPATH(using also CPU):
256x256: 6.3 GB
denoiser: 6.3 GB
2048x2048: 8.9 GB
denoiser: 11.2 GB
OpenCL+PATH(GPU only/GTX1060-6GB):
256x256: 5.2 GB
denoiser: 5.2 GB
2048x2048: 8 GB
denoiser: 10.3 GB
OpenCL+TILEPATH(GPU only/GTX1060-6GB):
256x256: 5.2 GB
denoiser: 5.2 GB - not working, black image as denoised result
2048x2048: 7.8 GB
denoiser: 10 GB - not working, black image as denoised result
So in conclusion, not only luxcore uses too much memory in general, even at low resolution, the memory usage skyrockets at 2K resolution(that's not even hi-res that's for mobile viewing) in a mostly empty scene with no textures and no hdri. Also even at 2K the denoiser performance is terribly slow. The differences between Path and TilePath were in all cases minimal and it's bad vs bad.
If you suspect problem could be elsewhere, like in the scene or luxcore usage itself. It is more or less the same version of the scene I gave Simon few days ago. Since I'm noob with luxcore please do check yourselves, I might be doing something wrong in the end.
Re: Luxcore and memory
no it's memory consumption during rendering phase vs denoiser phase...
I assumed disabled denoiser won't affect rendering itself.
Re: Luxcore and memory
Collecting and saving the statistics for denoising requires an HUGE amount of data per pixel. If you disable the denoiser, the memory usage will go down a lot.
Re: Luxcore and memory
damn, I did not expect that... I'm gonna do more tests tomorrow morning, but...
Using denoiser is pretty important so I hope that something can be done about that.
Now I'm really curious what will the results be without denoiser.
Re: Luxcore and memory
Alright, so here is the updated test - denoiser means denoiser phase during rendering and denoiser disabled means... disabled.
scene in blender before rendering:
3.3GB mostly empty scene and cycles at any resolution gives below 4GB RAM usage usage
luxcore:
cpu+PATH:
256x256: 5.8 GB
denoiser: 5.9 GB
denoiser disabled: 4.5 GB
2048x2048: 8 GB
denoiser: 10.3 GB
denoiser disabled: 5.5 GB
cpu+TILEPATH:
256x256: 5.8 GB
denoiser: 5.9 GB
denoiser disabled: 4.5 GB
2048x2048: 7.2 GB
denoiser: 9.5 GB
denoiser disabled: 6 GB
OpenCL+PATH(using also CPU):
256x256: 6.3 GB
denoiser: 6.3 GB
denoiser disabled: 6.3 GB
2048x2048: 9.8 GB
denoiser: 12.1 GB
denoiser disabled: 7.4 GB
OpenCL+TILEPATH(using also CPU):
256x256: 6.3 GB
denoiser: 6.3 GB
denoiser disabled: 6.3GB
2048x2048: 8.9 GB
denoiser: 11.2 GB
denoiser disabled: 7.8 GB
OpenCL+PATH(GPU only/GTX1060-6GB):
256x256: 5.2 GB
denoiser: 5.2 GB
denoiser disabled: 5.2 GB
2048x2048: 8 GB
denoiser: 10.3 GB
denoiser disabled: 5.7 GB
OpenCL+TILEPATH(GPU only/GTX1060-6GB):
256x256: 5.2 GB
denoiser: 5.2 GB - not working, black image as denoised result
denoiser disabled: 5.2 GB
2048x2048: 7.8 GB
denoiser: 10 GB - not working, black image as denoised result
denoiser disabled: 6.7GB
I hope I didn't make this somewhat confusing
scene in blender before rendering:
3.3GB mostly empty scene and cycles at any resolution gives below 4GB RAM usage usage
luxcore:
cpu+PATH:
256x256: 5.8 GB
denoiser: 5.9 GB
denoiser disabled: 4.5 GB
2048x2048: 8 GB
denoiser: 10.3 GB
denoiser disabled: 5.5 GB
cpu+TILEPATH:
256x256: 5.8 GB
denoiser: 5.9 GB
denoiser disabled: 4.5 GB
2048x2048: 7.2 GB
denoiser: 9.5 GB
denoiser disabled: 6 GB
OpenCL+PATH(using also CPU):
256x256: 6.3 GB
denoiser: 6.3 GB
denoiser disabled: 6.3 GB
2048x2048: 9.8 GB
denoiser: 12.1 GB
denoiser disabled: 7.4 GB
OpenCL+TILEPATH(using also CPU):
256x256: 6.3 GB
denoiser: 6.3 GB
denoiser disabled: 6.3GB
2048x2048: 8.9 GB
denoiser: 11.2 GB
denoiser disabled: 7.8 GB
OpenCL+PATH(GPU only/GTX1060-6GB):
256x256: 5.2 GB
denoiser: 5.2 GB
denoiser disabled: 5.2 GB
2048x2048: 8 GB
denoiser: 10.3 GB
denoiser disabled: 5.7 GB
OpenCL+TILEPATH(GPU only/GTX1060-6GB):
256x256: 5.2 GB
denoiser: 5.2 GB - not working, black image as denoised result
denoiser disabled: 5.2 GB
2048x2048: 7.8 GB
denoiser: 10 GB - not working, black image as denoised result
denoiser disabled: 6.7GB
I hope I didn't make this somewhat confusing
Re: Luxcore and memory
As you see, the denoiser requires to store a lot of data for each pixel (and, no, it is not avoidable, only mitigabile with the use of same already existing parameter). However it is a fixed cost related only to the number of pixel not the complexity of the scene (polygons count, materials, image maps, etc.)
What it can be avoided is the use of a local Film for each thread in order to reduce the increase of memory used with the increase of rendering threads (if you try to do a PATHCPU 2048x2048 rendering with 1 threads and with 10, you will se a HUGE increment in memory usage).
Re: Luxcore and memory
8192x8192:Dade wrote: ↑Sat Aug 11, 2018 6:24 pmAs you see, the denoiser requires to store a lot of data for each pixel (and, no, it is not avoidable, only mitigabile with the use of same already existing parameter). However it is a fixed cost related only to the number of pixel not the complexity of the scene (polygons count, materials, image maps, etc.)
What it can be avoided is the use of a local Film for each thread in order to reduce the increase of memory used with the increase of rendering threads (if you try to do a PATHCPU 2048x2048 rendering with 1 threads and with 10, you will se a HUGE increment in memory usage).
2.5GB scene in blender
cpu+path+logpower
denoiser disabled:
1 thread: 9.7 GB
8 thread: 20 GB
cycles: 3.7 GB