Light emitting instances

[Dade]

Yes, Cycles starts to render immediately with this scene.

Cycles access directly internal Blender structures and doesn't support scene editing: in LuxCore you can edit/delete one of the 18,000,000 light sources by name (so it needs to set up the required structures to access the named object).

[Dade]

I have optimized the mesh light parsing code.

Step #1

The first step was to discover how slow is boost::unordered_map structure (aka hash table). I replaced boost::unordered_map with robin_hood::unordered_flat_map (https://github.com/martinus/robin-hood-hashing).

This reduced the parsing time from 32secs to 19secs.

Step #2

I than discovered that computing hash of object names was taking a long time due to the extreme length of the names used by BlendLuxCore. It uses names like "139760923179016_1397609231759440_0_2147483647_2147483647_2147483647_2147483647_2147483647_2147483647_0_0_0_0_0_0_0_00dupli4460" ... is a bit long, isn't it ?

I worked around this problem computing the hash on a pre-encoded version of the name.

This reduced the parsing time from 19secs to 15secs.

Step #3

I than discovered that converting the triangle light index to string was taking a significative amount of time (because some generic code to handle locale problems when converting floating point numbers). I added an optimized and dedicated path fro this case.

This reduced the parsing time from 15secs to 10secs.

So now the scene parsing is more than 3 times faster. Note: this code is on "for_v2.6" branch.

[TAO]

Great job, 3 times faster that means a lot.
Is replacing boost::unordered_map with robin_hood::unordered_flat_map have any benefit in other cases, or do you have any suggestion for using it, if there is i be able to use it in the 3dsmax plugin, or is not necessarily.

[daros]

Great! Very nice!

[Sharlybg]

It indeed great improvement
Still afraid that if you have 3X the number of particles you're back to the same situation
Don't get me wrong This is anyway a great improvement.

Cycles access directly internal Blender structures and doesn't support scene editing: in LuxCore you can edit/delete one of the 18,000,000 light sources by name (so it needs to set up the required structures to access the named object).

In Cycles i'm able to changes everything In Realtime:

__instances number
__Emitter geometry
__Emitter Color

So if that same behaviour is possible and improve speed We should go this way (Artistically I mean i don't feel restricted with the current cycles behaviour). Maybe we can't access the blender structures like cycles but if there is anything we can mimic by making a given sacrifice maybe it is worth the sacrifice .........

[Dade]

Is replacing boost::unordered_map with robin_hood::unordered_flat_map have any benefit in other cases, or do you have any suggestion for using it, if there is i be able to use it in the 3dsmax plugin, or is not necessarily.

It is an hash table with 18,000,000 of entries in this scene, it is not a very common case. I don't think there are many other places where it is worth using. 99% of the scenes will have just <10 entries.

[Dade]

I introduced (partial) multi-thread support in light sources pre-processing so the time required has gone from 35secs to 8secs:

Code: Select all

# 8secs light preprocessing:
#   [LuxRays][16.892] EmbreeAccel build time: 19ms
#   [LuxCore][20.773] Light step #1 preprocessing time: 3.88031secs
#   [LuxCore][22.160] Light step #2 preprocessing time: 1.3868secs
#   [LuxCore][22.716] Light step #3 preprocessing time: 0.556359secs
#   [LuxCore][22.716] Light step #4 preprocessing time: 3.48091e-05secs
#   [LuxCore][25.295] Light step #5 preprocessing time: 2.57913secs
#   [LuxCore][25.295] Light total preprocessing time: 8.40269secs

[Sharlybg]

I introduced (partial) multi-thread support in light sources pre-processing so the time required has gone from 35secs to 8secs:

Code: Select all

# 8secs light preprocessing:
#   [LuxRays][16.892] EmbreeAccel build time: 19ms
#   [LuxCore][20.773] Light step #1 preprocessing time: 3.88031secs
#   [LuxCore][22.160] Light step #2 preprocessing time: 1.3868secs
#   [LuxCore][22.716] Light step #3 preprocessing time: 0.556359secs
#   [LuxCore][22.716] Light step #4 preprocessing time: 3.48091e-05secs
#   [LuxCore][25.295] Light step #5 preprocessing time: 2.57913secs
#   [LuxCore][25.295] Light total preprocessing time: 8.40269secs

[Sharlybg]

I introduced (partial) multi-thread support in light sources pre-processing so the time required has gone from 35secs to 8secs:

Code: Select all

# 8secs light preprocessing:
#   [LuxRays][16.892] EmbreeAccel build time: 19ms
#   [LuxCore][20.773] Light step #1 preprocessing time: 3.88031secs
#   [LuxCore][22.160] Light step #2 preprocessing time: 1.3868secs
#   [LuxCore][22.716] Light step #3 preprocessing time: 0.556359secs
#   [LuxCore][22.716] Light step #4 preprocessing time: 3.48091e-05secs
#   [LuxCore][25.295] Light step #5 preprocessing time: 2.57913secs
#   [LuxCore][25.295] Light total preprocessing time: 8.40269secs

gdg.gif (2.78 MiB) Viewed 3012 times

[marcatore]

fantastic job!!