Luxcore and memory

Use this forum for general user support and related questions.
Forum rules
Please upload a testscene that allows developers to reproduce the problem, and attach some images.
User avatar
lacilaci
Donor
Donor
Posts: 1969
Joined: Fri May 04, 2018 5:16 am

Luxcore and memory

Post by lacilaci »

Ok,

So with 2.1alpha3 I tried few hi-res 8192x8192 renderings and managed to fit the scene in the ram (~22GB).
I was able to use even opencl however not with denoiser (got this error - clEnqueueWriteBuffer) seems cause too big framebuffer for a 6GB gpu.

I still found the memory consumtion too big so I switched to cycles quickly and it would seem that not only I was able to render at 8K with gpu(with 2.8 it would be gpu+cpu) and denoiser, but the scene took only ~5GB ram.

Now I'm not comparing performance, gpu rendering is still only useful for cars and products nothing more complex will fit into any mainstream gpus and while smaller scenes are great to be able to render on both cpu+gpu. Indirect caching can make cpu effective enough for complex scenes rendered at high resolution.

What I'm worried about is that this is mostly empty scene with almost no textures. and if I take the 4X difference in memory consumption I've observed now it would mean that the full project that took ~14-16GB ram on cycles would require ~64GB ram? And this is still relatively small project.

EDIT:
I initially thought I'd get denoiser working even at 8K using only cpu. But it took 15min to get one sample done so I can't test that(not now at least)
Also, got this error on render end "vector<T> too long"
User avatar
Sharlybg
Donor
Donor
Posts: 3101
Joined: Mon Dec 04, 2017 10:11 pm
Location: Ivory Coast

Re: Luxcore and memory

Post by Sharlybg »

any possibility to upload your scene so i could try ?

Is your cycles test is set in progressive rendering intead of tile rendering ?
Support LuxCoreRender project with salts and bounties

Portfolio : https://www.behance.net/DRAVIA
User avatar
lacilaci
Donor
Donor
Posts: 1969
Joined: Fri May 04, 2018 5:16 am

Re: Luxcore and memory

Post by lacilaci »

Sharlybg wrote: Thu Aug 09, 2018 9:58 am any possibility to upload your scene so i could try ?

Is your cycles test is set in progressive rendering intead of tile rendering ?
You can try on any scene? Do you not see the differences in your case?

Also I did use tile rendering for cycles, I don't see reason not to use it.

EDIT:
To make sure, I created completely different scene and got this:

luxcore: 3.6GB-rendering and 5.3GB-denoising
cycles: 1.4GB

again, huge differences. This was just a piece of furniture (although pretty heavy one)
User avatar
Sharlybg
Donor
Donor
Posts: 3101
Joined: Mon Dec 04, 2017 10:11 pm
Location: Ivory Coast

Re: Luxcore and memory

Post by Sharlybg »

Also I did use tile rendering for cycles, I don't see reason not to use it.
because it reduce memory usage. So try them both with 1 big tile aka progressive rendering. no HDRI or sky light just test with area light so we can easily see where the difference is. ;)
Support LuxCoreRender project with salts and bounties

Portfolio : https://www.behance.net/DRAVIA
User avatar
B.Y.O.B.
Developer
Developer
Posts: 4146
Joined: Mon Dec 04, 2017 10:08 pm
Location: Germany
Contact:

Re: Luxcore and memory

Post by B.Y.O.B. »

I have also observed that LuxCore uses a lot of RAM.
I suspect the framebuffer (because I also tested with an almost empty scene), however there's also the possibility that I'm making some mistake in the addon code (e.g. creating unnecessary film outputs or something).
I've created this issue to keep track of this problem: https://github.com/LuxCoreRender/BlendL ... issues/151
User avatar
lacilaci
Donor
Donor
Posts: 1969
Joined: Fri May 04, 2018 5:16 am

Re: Luxcore and memory

Post by lacilaci »

Sharlybg wrote: Thu Aug 09, 2018 10:57 am
Also I did use tile rendering for cycles, I don't see reason not to use it.
because it reduce memory usage. So try them both with 1 big tile aka progressive rendering. no HDRI or sky light just test with area light so we can easily see where the difference is. ;)
Ok, so I did enable progresive mode on cycles and it made almost no difference in memory consumption(+1GB difference), except it being slower.

But even if it did, it doesn't matter. I've worked with big scenes using corona (has no tiled sampler) and memory was only issue with very very big and complex scenes, making whole 3Ds max pain to use.
Luxcore is taking so much I can't imagine practical usage in complex projects without running out of memory halfway through.
User avatar
lacilaci
Donor
Donor
Posts: 1969
Joined: Fri May 04, 2018 5:16 am

Re: Luxcore and memory

Post by lacilaci »

B.Y.O.B. wrote: Thu Aug 09, 2018 10:59 am I have also observed that LuxCore uses a lot of RAM.
I suspect the framebuffer (because I also tested with an almost empty scene), however there's also the possibility that I'm making some mistake in the addon code (e.g. creating unnecessary film outputs or something).
I've created this issue to keep track of this problem: https://github.com/LuxCoreRender/BlendL ... issues/151
Can I try rendering the exported scene outside of blender? Would that help to see if that's the issue?
User avatar
B.Y.O.B.
Developer
Developer
Posts: 4146
Joined: Mon Dec 04, 2017 10:08 pm
Location: Germany
Contact:

Re: Luxcore and memory

Post by B.Y.O.B. »

lacilaci wrote: Thu Aug 09, 2018 11:12 am Can I try rendering the exported scene outside of blender? Would that help to see if that's the issue?
I already tried that (see the github issue).
In Blender: 3.5 GiB
In luxcoreui: 2.7 GiB

So if we subtract the RAM Blender needs (about 330 MiB) then pyluxcore needs about 3.2 GiB of memory when run in Blender.
There is a difference (there might be several unrelated problems here that all contribute to the high RAM usage), but the 2.7 GiB luxcoreui needs standalone are also a lot compared to the 70 MiB Cycles needs.
User avatar
lacilaci
Donor
Donor
Posts: 1969
Joined: Fri May 04, 2018 5:16 am

Re: Luxcore and memory

Post by lacilaci »

B.Y.O.B. wrote: Thu Aug 09, 2018 12:00 pm
lacilaci wrote: Thu Aug 09, 2018 11:12 am Can I try rendering the exported scene outside of blender? Would that help to see if that's the issue?
I already tried that (see the github issue).
In Blender: 3.5 GiB
In luxcoreui: 2.7 GiB

So if we subtract the RAM Blender needs (about 330 MiB) then pyluxcore needs about 3.2 GiB of memory when run in Blender.
There is a difference (there might be several unrelated problems here that all contribute to the high RAM usage), but the 2.7 GiB luxcoreui needs standalone are also a lot compared to the 70 MiB Cycles needs.
Interesting, but can the framebuffer be so big? Do you think cycles somehow offloads it from vram or ram completely? One another thing in this regard is also navigating blender and uv/image editor while rendering.
With cycles even 8K rendering on gpu was no problem zooming in and paning etc.. Luxcore on other hand I couldn't do anything but wait with 8K even using cpu
User avatar
Sharlybg
Donor
Donor
Posts: 3101
Joined: Mon Dec 04, 2017 10:11 pm
Location: Ivory Coast

Re: Luxcore and memory

Post by Sharlybg »

the 2.7 GiB luxcoreui needs standalone are also a lot compared to the 70 MiB Cycles needs.
:shock:
Support LuxCoreRender project with salts and bounties

Portfolio : https://www.behance.net/DRAVIA
Post Reply