Luxcore and memory

Use this forum for general user support and related questions.
Forum rules
Please upload a testscene that allows developers to reproduce the problem, and attach some images.
User avatar
lacilaci
Donor
Donor
Posts: 1969
Joined: Fri May 04, 2018 5:16 am

Re: Luxcore and memory

Post by lacilaci »

I will check back on this, thanks for the input. This memory usage is my biggest concerns for now.
User avatar
B.Y.O.B.
Developer
Developer
Posts: 4146
Joined: Mon Dec 04, 2017 10:08 pm
Location: Germany
Contact:

Re: Luxcore and memory

Post by B.Y.O.B. »

Dade wrote: Fri Aug 10, 2018 4:33 pm Side note: you can reduce Film memory usage by disabling OpenCL image pipeline (property film.opencl.enable). Indeed, some complex image pipeline will run a lot slower if using the CPU.
By the way, this setting is not exposed in the addon yet, it is always enabled (if you have an OpenCL build).

However, I have done all my tests on this matter so far with a Non-OpenCL build on Linux, so the OpenCL imagepipeline was not enabled.
User avatar
lacilaci
Donor
Donor
Posts: 1969
Joined: Fri May 04, 2018 5:16 am

Re: Luxcore and memory

Post by lacilaci »

Alright so here are some results of path vs tilepath ram usage:

scene in blender before rendering:
3.3GB mostly empty scene and cycles at any resolution gives below 4GB RAM usage

luxcore:

cpu+PATH:
256x256: 5.8 GB
denoiser: 5.9 GB

2048x2048: 8 GB
denoiser: 10.3 GB

cpu+TILEPATH:
256x256: 5.8 GB
denoiser: 5.9 GB

2048x2048: 7.2 GB
denoiser: 9.5 GB

OpenCL+PATH(using also CPU):
256x256: 6.3 GB
denoiser: 6.3 GB

2048x2048: 9.8 GB
denoiser: 12.1 GB

OpenCL+TILEPATH(using also CPU):
256x256: 6.3 GB
denoiser: 6.3 GB

2048x2048: 8.9 GB
denoiser: 11.2 GB

OpenCL+PATH(GPU only/GTX1060-6GB):
256x256: 5.2 GB
denoiser: 5.2 GB

2048x2048: 8 GB
denoiser: 10.3 GB

OpenCL+TILEPATH(GPU only/GTX1060-6GB):
256x256: 5.2 GB
denoiser: 5.2 GB - not working, black image as denoised result

2048x2048: 7.8 GB
denoiser: 10 GB - not working, black image as denoised result


So in conclusion, not only luxcore uses too much memory in general, even at low resolution, the memory usage skyrockets at 2K resolution(that's not even hi-res that's for mobile viewing) in a mostly empty scene with no textures and no hdri. Also even at 2K the denoiser performance is terribly slow. The differences between Path and TilePath were in all cases minimal and it's bad vs bad.

If you suspect problem could be elsewhere, like in the scene or luxcore usage itself. It is more or less the same version of the scene I gave Simon few days ago. Since I'm noob with luxcore please do check yourselves, I might be doing something wrong in the end.
User avatar
Dade
Developer
Developer
Posts: 5672
Joined: Mon Dec 04, 2017 8:36 pm
Location: Italy

Re: Luxcore and memory

Post by Dade »

lacilaci wrote: Sat Aug 11, 2018 7:12 am cpu+PATH:
256x256: 5.8 GB
denoiser: 5.9 GB
Is this this denoiser enabled Vs. disabled ?
Support LuxCoreRender project with salts and bounties
User avatar
lacilaci
Donor
Donor
Posts: 1969
Joined: Fri May 04, 2018 5:16 am

Re: Luxcore and memory

Post by lacilaci »

Dade wrote: Sat Aug 11, 2018 10:08 am
lacilaci wrote: Sat Aug 11, 2018 7:12 am cpu+PATH:
256x256: 5.8 GB
denoiser: 5.9 GB
Is this this denoiser enabled Vs. disabled ?
no it's memory consumption during rendering phase vs denoiser phase...

I assumed disabled denoiser won't affect rendering itself.
User avatar
Dade
Developer
Developer
Posts: 5672
Joined: Mon Dec 04, 2017 8:36 pm
Location: Italy

Re: Luxcore and memory

Post by Dade »

lacilaci wrote: Sat Aug 11, 2018 10:14 am I assumed disabled denoiser won't affect rendering itself.
Collecting and saving the statistics for denoising requires an HUGE amount of data per pixel. If you disable the denoiser, the memory usage will go down a lot.
Support LuxCoreRender project with salts and bounties
User avatar
lacilaci
Donor
Donor
Posts: 1969
Joined: Fri May 04, 2018 5:16 am

Re: Luxcore and memory

Post by lacilaci »

Dade wrote: Sat Aug 11, 2018 10:58 am
lacilaci wrote: Sat Aug 11, 2018 10:14 am I assumed disabled denoiser won't affect rendering itself.
Collecting and saving the statistics for denoising requires an HUGE amount of data per pixel. If you disable the denoiser, the memory usage will go down a lot.
damn, I did not expect that... I'm gonna do more tests tomorrow morning, but...
Using denoiser is pretty important so I hope that something can be done about that.
Now I'm really curious what will the results be without denoiser.
User avatar
lacilaci
Donor
Donor
Posts: 1969
Joined: Fri May 04, 2018 5:16 am

Re: Luxcore and memory

Post by lacilaci »

Alright, so here is the updated test - denoiser means denoiser phase during rendering and denoiser disabled means... disabled.

scene in blender before rendering:
3.3GB mostly empty scene and cycles at any resolution gives below 4GB RAM usage usage

luxcore:

cpu+PATH:
256x256: 5.8 GB
denoiser: 5.9 GB
denoiser disabled: 4.5 GB

2048x2048: 8 GB
denoiser: 10.3 GB
denoiser disabled: 5.5 GB

cpu+TILEPATH:
256x256: 5.8 GB
denoiser: 5.9 GB
denoiser disabled: 4.5 GB

2048x2048: 7.2 GB
denoiser: 9.5 GB
denoiser disabled: 6 GB

OpenCL+PATH(using also CPU):
256x256: 6.3 GB
denoiser: 6.3 GB
denoiser disabled: 6.3 GB

2048x2048: 9.8 GB
denoiser: 12.1 GB
denoiser disabled: 7.4 GB

OpenCL+TILEPATH(using also CPU):
256x256: 6.3 GB
denoiser: 6.3 GB
denoiser disabled: 6.3GB

2048x2048: 8.9 GB
denoiser: 11.2 GB
denoiser disabled: 7.8 GB

OpenCL+PATH(GPU only/GTX1060-6GB):
256x256: 5.2 GB
denoiser: 5.2 GB
denoiser disabled: 5.2 GB

2048x2048: 8 GB
denoiser: 10.3 GB
denoiser disabled: 5.7 GB

OpenCL+TILEPATH(GPU only/GTX1060-6GB):
256x256: 5.2 GB
denoiser: 5.2 GB - not working, black image as denoised result
denoiser disabled: 5.2 GB

2048x2048: 7.8 GB
denoiser: 10 GB - not working, black image as denoised result
denoiser disabled: 6.7GB

I hope I didn't make this somewhat confusing :D
User avatar
Dade
Developer
Developer
Posts: 5672
Joined: Mon Dec 04, 2017 8:36 pm
Location: Italy

Re: Luxcore and memory

Post by Dade »

lacilaci wrote: Sat Aug 11, 2018 11:30 am cpu+PATH:
256x256: 5.8 GB
denoiser: 5.9 GB
denoiser disabled: 4.5 GB

2048x2048: 8 GB
denoiser: 10.3 GB
denoiser disabled: 5.5 GB
As you see, the denoiser requires to store a lot of data for each pixel (and, no, it is not avoidable, only mitigabile with the use of same already existing parameter). However it is a fixed cost related only to the number of pixel not the complexity of the scene (polygons count, materials, image maps, etc.)

What it can be avoided is the use of a local Film for each thread in order to reduce the increase of memory used with the increase of rendering threads (if you try to do a PATHCPU 2048x2048 rendering with 1 threads and with 10, you will se a HUGE increment in memory usage).
Support LuxCoreRender project with salts and bounties
User avatar
lacilaci
Donor
Donor
Posts: 1969
Joined: Fri May 04, 2018 5:16 am

Re: Luxcore and memory

Post by lacilaci »

Dade wrote: Sat Aug 11, 2018 6:24 pm
lacilaci wrote: Sat Aug 11, 2018 11:30 am cpu+PATH:
256x256: 5.8 GB
denoiser: 5.9 GB
denoiser disabled: 4.5 GB

2048x2048: 8 GB
denoiser: 10.3 GB
denoiser disabled: 5.5 GB
As you see, the denoiser requires to store a lot of data for each pixel (and, no, it is not avoidable, only mitigabile with the use of same already existing parameter). However it is a fixed cost related only to the number of pixel not the complexity of the scene (polygons count, materials, image maps, etc.)

What it can be avoided is the use of a local Film for each thread in order to reduce the increase of memory used with the increase of rendering threads (if you try to do a PATHCPU 2048x2048 rendering with 1 threads and with 10, you will se a HUGE increment in memory usage).
8192x8192:

2.5GB scene in blender

cpu+path+logpower

denoiser disabled:
1 thread: 9.7 GB
8 thread: 20 GB

cycles: 3.7 GB
Post Reply