To stress memory usage a bit a setup a test scene with 250 trees with particle system. Thanks to B.Y.O.B. I have been able to reduce time creating a tree group. Most tricky thing is the orientation, I think the behaviour of particle system regarding rotation is strange for a long time. One has to rotate an object or a group until one can use it properly for randomzied rotation around veritcal axis of a an upright standing object on the emitter plane.
I don't see much unexpected memory usage nor with opencl path neither with opencl tile path.
But rendering 2xHD resolution blender is not very responsive, means I can wait several seconds until blender reacts to mouse inputs if blender shows any reaction at all. And this even with an overal CPU usage of less than 2%.
The tough blender response is not the case rendering HD resolution only.
For the texture of the leaf I made a simple sketch to avoid copy right issues.
Luxcore and memory
Forum rules
Please upload a testscene that allows developers to reproduce the problem, and attach some images.
Please upload a testscene that allows developers to reproduce the problem, and attach some images.
- FarbigeWelt
- Donor
- Posts: 1046
- Joined: Sun Jul 01, 2018 12:07 pm
- Location: Switzerland
- Contact:
Re: Luxcore and memory
Light and Word designing Creator - www.farbigewelt.ch - aka quantenkristall || #luxcorerender
MacBook Air with M1
MacBook Air with M1
Re: Luxcore and memory
In memory it's still just one tree not 250 if it's in the particle system.FarbigeWelt wrote: ↑Sun Aug 12, 2018 5:39 pm To stress memory usage a bit a setup a test scene with 250 trees with particle system. Thanks to B.Y.O.B. I have been able to reduce time creating a tree group. Most tricky thing is the orientation, I think the behaviour of particle system regarding rotation is strange for a long time. One has to rotate an object or a group until one can use it properly for randomzied rotation around veritcal axis of a an upright standing object on the emitter plane.
I don't see much unexpected memory usage nor with opencl path neither with opencl tile path.
But rendering 2xHD resolution blender is not very responsive, means I can wait several seconds until blender reacts to mouse inputs if blender shows any reaction at all. And this even with an overal CPU usage of less than 2%.
The tough blender response is not the case rendering HD resolution only.
Hazel 2.zip
250 Hazels OpenCL path tile .jpg
For the texture of the leaf I made a simple sketch to avoid copy right issues.
And as David said it's not scene complexity but multiple films and high resolution+denoiser issue. Although, even in the ideal case of using lowres no denoiser and single core, luxcore still uses quite a bit more ram than cycles.
But I guess it is because cycles is much better inegrated into blender.
- FarbigeWelt
- Donor
- Posts: 1046
- Joined: Sun Jul 01, 2018 12:07 pm
- Location: Switzerland
- Contact:
Re: Luxcore and memory
[quote=lacilaci post_id=5403 time=1534131748 user_id=211
b) But I guess it is because cycles is much better inegrated into blender.
[/quote]
a) Are you sure? I made a test with 1 M particles. Referenced object: a single rectangle with an image texture glossy material. An image with 50x255. During export process task manager reported 28 GB for blender. It took luxcore about half an hour to build all duplicate before it threw some errors in console and killed blender.
Yes, increasing resolution takes mem. 2HD took about 1.6 GB, 4HD about 5.6 GB. Opening 32 bit float in PS takes 1 GB, after conversion to 16 bit, PS takes 3.6 GB, why ever. Maybe related to undo stack. It still allocated tjis ram after closing the file.
Maybe there should be one result layer for large resolutions and smaller render layers per core like every 1/cores pixel and displaying only render of a core until end conditions are reached or esc has been pressed.
b) Why do you think so?
a) In memory it's still just one tree not 250 if it's in the particle system.FarbigeWelt wrote: ↑Sun Aug 12, 2018 5:39 pm To stress memory usage a bit a setup a test scene with 250 trees with particle system.
I don't see much unexpected memory usage nor with opencl path neither with opencl tile path.
b) But I guess it is because cycles is much better inegrated into blender.
[/quote]
a) Are you sure? I made a test with 1 M particles. Referenced object: a single rectangle with an image texture glossy material. An image with 50x255. During export process task manager reported 28 GB for blender. It took luxcore about half an hour to build all duplicate before it threw some errors in console and killed blender.
Yes, increasing resolution takes mem. 2HD took about 1.6 GB, 4HD about 5.6 GB. Opening 32 bit float in PS takes 1 GB, after conversion to 16 bit, PS takes 3.6 GB, why ever. Maybe related to undo stack. It still allocated tjis ram after closing the file.
Maybe there should be one result layer for large resolutions and smaller render layers per core like every 1/cores pixel and displaying only render of a core until end conditions are reached or esc has been pressed.
b) Why do you think so?
Light and Word designing Creator - www.farbigewelt.ch - aka quantenkristall || #luxcorerender
MacBook Air with M1
MacBook Air with M1
Re: Luxcore and memory
lacilaci is right, the point of instances is that the whole mesh is only loaded into memory once.FarbigeWelt wrote: ↑Mon Aug 13, 2018 10:57 am a) Are you sure? I made a test with 1 M particles. Referenced object: a single rectangle with an image texture glossy material. An image with 50x255. During export process task manager reported 28 GB for blender. It took luxcore about half an hour to build all duplicate before it threw some errors in console and killed blender.
The instances just need a transformation matrix and some extra info (e.g. material), so they need very little memory.
However they still need some memory (for example, a matrix is 4 * 4 * sizeof(float) = 64 Bytes (a float is 4 Bytes).
So if you instance 100,000 trees, you need the memory for the base mesh plus 100,000 * 64 = 6,400,000 Bytes = 6.4 MB for the instances.
However an instance is more than just the matrix, so you will need a bit more.
You should not test instances with simple meshes: A quad is just two triangles, so about 6 vertices * sizeof(float) = 24 Bytes. With a simple mesh, you save almost nothing by using instancing compared to duplicating the 24 Bytes around.
Anyway, this thread is not about instancing, it's about film memory usage (at least I assume that it is causing most of the issues).
- FarbigeWelt
- Donor
- Posts: 1046
- Joined: Sun Jul 01, 2018 12:07 pm
- Location: Switzerland
- Contact:
Re: Luxcore and memory
Thanks for the explanation, which makes sense.
I forgot to ask, what is about light groups? Does memory usage grow with their number?
Light and Word designing Creator - www.farbigewelt.ch - aka quantenkristall || #luxcorerender
MacBook Air with M1
MacBook Air with M1
Re: Luxcore and memory
Yes, each light group's contribution needs a separate buffer.FarbigeWelt wrote: ↑Mon Aug 13, 2018 12:19 pm I forgot to ask, what is about light groups? Does memory usage grow with their number?
I think the memory usage of a light group is width * height * 3 * sizeof(float) Bytes.
Re: Luxcore and memory
It is "4 *" (i.e. the weight) for eye path buffer and "3 *" for light path buffer (used only by BIDIRCPU and LIGHTCPU, in conjunction with the eye path buffer).B.Y.O.B. wrote: ↑Mon Aug 13, 2018 12:27 pmYes, each light group's contribution needs a separate buffer.FarbigeWelt wrote: ↑Mon Aug 13, 2018 12:19 pm I forgot to ask, what is about light groups? Does memory usage grow with their number?
I think the memory usage of a light group is width * height * 3 * sizeof(float) Bytes.