LuxCoreRender Forums

Is there any way to do the scene Parse and/or DefineMesh from different threads at the same time?
I tried to use Mutex and Critical Section before this functions call but the performance suffers from all synchronization processes and in most cases, I didn't get any benefit from multi threads, in the best case I achieved 25% of performance.
On the other hand for the extensive process part from 3dsmax, I achieved at least 4 to 8 times better performance. considering those parts did not require too much synchronization and waiting for writing variables.
I tried to compare a few scene translation part performance with BlendLuxCore and because of the nature of the BlendLuxCore that mostly runs by the python language, the 3dsmax version has better performance, for example in my case, translation of almost 174000 objects (most of them are low poly) take around 160 seconds with Core i9 9900k and take no more than 8% of CPU usage.
Of course, the MaxToLux code needs to be optimized and it's not finished yet, as I started to code not so far and more focus on adding features that optimizing the code, and try to make it less complicated at this point, with optimized code may come better performance.
Long story short, my question is:
Is there any other solution available that anyone tried before to multi-thread these parts? or, any suggestions to put me in the right direction?

TAO wrote: ↑Thu Jan 28, 2021 2:26 am Is there any way to do the scene Parse and/or DefineMesh from different threads at the same time?

No, it is not thread safe.

TAO wrote: ↑Thu Jan 28, 2021 2:26 am Is there any other solution available that anyone tried before to multi-thread these parts? or, any suggestions to put me in the right direction?

Are you defining 174000 "different" objects ? Or are they mostly copies of the same object ?

Can you post a test scene ?

There may be many ways to accelerate the process but I need to see where is the bottleneck. For instance, you could merge the meshes by material in order to reduce the object count, etc.

Dade wrote: ↑Thu Jan 28, 2021 12:00 pm

TAO wrote: ↑Thu Jan 28, 2021 2:26 am Is there any way to do the scene Parse and/or DefineMesh from different threads at the same time?

No, it is not thread safe.

TAO wrote: ↑Thu Jan 28, 2021 2:26 am Is there any other solution available that anyone tried before to multi-thread these parts? or, any suggestions to put me in the right direction?

Are you defining 174000 "different" objects ? Or are they mostly copies of the same object ?

Can you post a test scene ?

There may be many ways to accelerate the process but I need to see where is the bottleneck. For instance, you could merge the meshes by material in order to reduce the object count, etc.

ost of them are different objects but I also have around 57000 instance objects that half of them are identical instance and the rest are instanced plus adding a few modifiers on top of them (I think that makes them a different object and not instance, not sure), I try to use instancing but I still don't know how to detect these objects are instanced from each other in 3dsmax, especially when I have a different modifier stack on top of a few of them.
One of the issues I faced is I don't know how to decide which objects are instanced and which is not.
The scene is too huge to be uploaded. around 1 GB.
But I think I can make a sample from a few objects, so, maybe you can help me to better understand instancing proccess.

Here is a simple scene to start with.
In this scene, all of these objects are instances from the 3dsmax point of view.

Object 2 is an instance from object 1 and they are identical.
Object 3 is a copy of object 2, exactly the same plus on top of that add a UV modifier to change the UV scale.
Object 4 is an instance of object 1 plus mesh modifier on top of it. (reference object)
if object one change all other objects will change, if any modifier change on object 3 and 4 will not affect the other objects. Are these objects considering instance at all?

As I wrote elsewhere, you need to use the DuplicateObject method to create a large number of instances, it is the correct way to create them.

You need also to detect instances to save memory: otherwise you are duplicating the meshes and using an insane amount of memory.

The scene you posted created 4 different meshes while 1 and 2 should share the same mesh (not the others).

Thank you for the point. as I can see instance objects should be exactly the same and in my scene, there is about 26000 instance object that can speed up the project with instancing.
Still, I have a hard time using it in the correct way.