Page 1 of 4

Luxcore and memory

Posted: Thu Aug 09, 2018 9:49 am
by lacilaci
Ok,

So with 2.1alpha3 I tried few hi-res 8192x8192 renderings and managed to fit the scene in the ram (~22GB).
I was able to use even opencl however not with denoiser (got this error - clEnqueueWriteBuffer) seems cause too big framebuffer for a 6GB gpu.

I still found the memory consumtion too big so I switched to cycles quickly and it would seem that not only I was able to render at 8K with gpu(with 2.8 it would be gpu+cpu) and denoiser, but the scene took only ~5GB ram.

Now I'm not comparing performance, gpu rendering is still only useful for cars and products nothing more complex will fit into any mainstream gpus and while smaller scenes are great to be able to render on both cpu+gpu. Indirect caching can make cpu effective enough for complex scenes rendered at high resolution.

What I'm worried about is that this is mostly empty scene with almost no textures. and if I take the 4X difference in memory consumption I've observed now it would mean that the full project that took ~14-16GB ram on cycles would require ~64GB ram? And this is still relatively small project.

EDIT:
I initially thought I'd get denoiser working even at 8K using only cpu. But it took 15min to get one sample done so I can't test that(not now at least)
Also, got this error on render end "vector<T> too long"

Re: Luxcore and memory

Posted: Thu Aug 09, 2018 9:58 am
by Sharlybg
any possibility to upload your scene so i could try ?

Is your cycles test is set in progressive rendering intead of tile rendering ?

Re: Luxcore and memory

Posted: Thu Aug 09, 2018 10:21 am
by lacilaci
Sharlybg wrote: Thu Aug 09, 2018 9:58 am any possibility to upload your scene so i could try ?

Is your cycles test is set in progressive rendering intead of tile rendering ?
You can try on any scene? Do you not see the differences in your case?

Also I did use tile rendering for cycles, I don't see reason not to use it.

EDIT:
To make sure, I created completely different scene and got this:

luxcore: 3.6GB-rendering and 5.3GB-denoising
cycles: 1.4GB

again, huge differences. This was just a piece of furniture (although pretty heavy one)

Re: Luxcore and memory

Posted: Thu Aug 09, 2018 10:57 am
by Sharlybg
Also I did use tile rendering for cycles, I don't see reason not to use it.
because it reduce memory usage. So try them both with 1 big tile aka progressive rendering. no HDRI or sky light just test with area light so we can easily see where the difference is. ;)

Re: Luxcore and memory

Posted: Thu Aug 09, 2018 10:59 am
by B.Y.O.B.
I have also observed that LuxCore uses a lot of RAM.
I suspect the framebuffer (because I also tested with an almost empty scene), however there's also the possibility that I'm making some mistake in the addon code (e.g. creating unnecessary film outputs or something).
I've created this issue to keep track of this problem: https://github.com/LuxCoreRender/BlendL ... issues/151

Re: Luxcore and memory

Posted: Thu Aug 09, 2018 11:11 am
by lacilaci
Sharlybg wrote: Thu Aug 09, 2018 10:57 am
Also I did use tile rendering for cycles, I don't see reason not to use it.
because it reduce memory usage. So try them both with 1 big tile aka progressive rendering. no HDRI or sky light just test with area light so we can easily see where the difference is. ;)
Ok, so I did enable progresive mode on cycles and it made almost no difference in memory consumption(+1GB difference), except it being slower.

But even if it did, it doesn't matter. I've worked with big scenes using corona (has no tiled sampler) and memory was only issue with very very big and complex scenes, making whole 3Ds max pain to use.
Luxcore is taking so much I can't imagine practical usage in complex projects without running out of memory halfway through.

Re: Luxcore and memory

Posted: Thu Aug 09, 2018 11:12 am
by lacilaci
B.Y.O.B. wrote: Thu Aug 09, 2018 10:59 am I have also observed that LuxCore uses a lot of RAM.
I suspect the framebuffer (because I also tested with an almost empty scene), however there's also the possibility that I'm making some mistake in the addon code (e.g. creating unnecessary film outputs or something).
I've created this issue to keep track of this problem: https://github.com/LuxCoreRender/BlendL ... issues/151
Can I try rendering the exported scene outside of blender? Would that help to see if that's the issue?

Re: Luxcore and memory

Posted: Thu Aug 09, 2018 12:00 pm
by B.Y.O.B.
lacilaci wrote: Thu Aug 09, 2018 11:12 am Can I try rendering the exported scene outside of blender? Would that help to see if that's the issue?
I already tried that (see the github issue).
In Blender: 3.5 GiB
In luxcoreui: 2.7 GiB

So if we subtract the RAM Blender needs (about 330 MiB) then pyluxcore needs about 3.2 GiB of memory when run in Blender.
There is a difference (there might be several unrelated problems here that all contribute to the high RAM usage), but the 2.7 GiB luxcoreui needs standalone are also a lot compared to the 70 MiB Cycles needs.

Re: Luxcore and memory

Posted: Thu Aug 09, 2018 12:28 pm
by lacilaci
B.Y.O.B. wrote: Thu Aug 09, 2018 12:00 pm
lacilaci wrote: Thu Aug 09, 2018 11:12 am Can I try rendering the exported scene outside of blender? Would that help to see if that's the issue?
I already tried that (see the github issue).
In Blender: 3.5 GiB
In luxcoreui: 2.7 GiB

So if we subtract the RAM Blender needs (about 330 MiB) then pyluxcore needs about 3.2 GiB of memory when run in Blender.
There is a difference (there might be several unrelated problems here that all contribute to the high RAM usage), but the 2.7 GiB luxcoreui needs standalone are also a lot compared to the 70 MiB Cycles needs.
Interesting, but can the framebuffer be so big? Do you think cycles somehow offloads it from vram or ram completely? One another thing in this regard is also navigating blender and uv/image editor while rendering.
With cycles even 8K rendering on gpu was no problem zooming in and paning etc.. Luxcore on other hand I couldn't do anything but wait with 8K even using cpu

Re: Luxcore and memory

Posted: Thu Aug 09, 2018 12:30 pm
by Sharlybg
the 2.7 GiB luxcoreui needs standalone are also a lot compared to the 70 MiB Cycles needs.
:shock: