Page 2 of 6
Re: OpenVDB support
Posted: Mon Apr 09, 2018 9:42 pm
by Dade
B.Y.O.B. wrote: ↑Mon Apr 09, 2018 9:29 pm
By the way can we accelerate the export of the normal point cache with C++?
Have to investigate. I think we would need a C++ function "DefineDensityGridData()" (like DefineMesh etc.) because the main bottleneck seems to be that we currently pass a HUGE array to a property.
Yes but isn't this problem totally solved by OpenVDB ?
Re: OpenVDB support
Posted: Mon Apr 09, 2018 9:47 pm
by B.Y.O.B.
Yes, but OpenVDB is not the default and most users will still use the old point cache.
But yes, if it turns out to be too much work then I'll just add a label to the UI that says "use OpenVDB for fast smoke rendering".
Re: OpenVDB support
Posted: Tue Apr 10, 2018 4:07 pm
by Dade
I merged all branches and now OpenVDB is part of the main LuxCoreRender branch.
I'm going to add the support for half storage in order to reduce the memory footprint.
Don't we need support for 3xFloat data too ? To express density grids with RGB values (not just a single scalar)
Re: OpenVDB support
Posted: Tue Apr 10, 2018 4:10 pm
by B.Y.O.B.
Dade wrote: ↑Tue Apr 10, 2018 4:07 pm
Don't we need support for 3xFloat data too ? To express density grids with RGB values (not just a single scalar)
Would be great!
I think Blender supports smoke sims with mixing different colors and has a color_grid, but currently we don't support it.
edit: had a look and the Blender color grid is 4 times the size of the density grid.
So I guess it is a one dimensional float array with values r, g, b, a, r, g, b, a, ... (note the alpha component).
Obviously it is not documented in the API.
https://docs.blender.org/api/current/bp ... color_grid
Re: OpenVDB support
Posted: Tue Apr 10, 2018 6:56 pm
by Dade
B.Y.O.B. wrote: ↑Tue Apr 10, 2018 4:10 pm
Dade wrote: ↑Tue Apr 10, 2018 4:07 pm
Don't we need support for 3xFloat data too ? To express density grids with RGB values (not just a single scalar)
Would be great!
I think Blender supports smoke sims with mixing different colors and has a color_grid, but currently we don't support it.
edit: had a look and the Blender color grid is 4 times the size of the density grid.
So I guess it is a one dimensional float array with values r, g, b, a, r, g, b, a, ... (note the alpha component).
Obviously it is not documented in the API.
https://docs.blender.org/api/current/bp ... color_grid
I will add the support for RGB values.
Re: OpenVDB support
Posted: Wed Apr 11, 2018 1:20 pm
by Dade
I have added the support for DensityGrid storage type with the ".storage" property. The supported storage are the same of image maps:
- byte
- half
- float
- auto
The default value (and auto) is half format. The "half" format cut in half the memory requirements and it is now the default. The "byte" format may not have enough resolution for some application but it is very effective to reduce the memory usage. This is the bunny rendered with byte storage:
It uses 1/4 of memory required by float storage.
I have also added the support for vector data (i.e. RGB). They can be defined with the ".data3" property instead of ".data". Indeed the number of expected values for ".data3" is nx * ny * nz * 3. OpenVDB supports automatically detect if the grid is scalar or vector.
Re: OpenVDB support
Posted: Wed Apr 11, 2018 6:37 pm
by B.Y.O.B.
B.Y.O.B. wrote: ↑Mon Apr 09, 2018 9:29 pm
I have started to work on support in BlendLuxCore however apparently it is non-trivial to find out the path of the cache files...
I can't find the relevant Python attribute in Blender or in the documentation. Have to look further.
Found it!
object.modifiers["Smoke"].domain_settings.point_cache.filepath
Now I have to figure out where exactly to find the cache (because if the filepath is empty, I think a folder is created in the save location of the .blend file, and it can have a name but it does not have to, and I have to account for smoke sims linked in through a library etc. pp.)
To be honest it would be much more elegant to have a fast way to transfer Blender's grid data to LuxCore.
It would even save RAM if we could just specify a pointer to the start of the array in Blender's memory (no need to copy the data).
(But then again this only works for CPU renders...)
I'm sure the OpenVDB support will be used by someone else someday, but for Blender it is a bit complicated.
I should have investigated before you started to implement it...
Re: OpenVDB support
Posted: Wed Apr 11, 2018 7:07 pm
by Dade
How do you access the Blender gird data ? Is it a Python list of scalar float ? How is it if they are RGB values ?
Re: OpenVDB support
Posted: Thu Apr 12, 2018 6:42 am
by B.Y.O.B.
From Python, we can access this class:
https://docs.blender.org/api/current/bp ... inSettings
It contains the various grids: density_grid, color_grid, ... (but only in Python - in the C code behind the scenes they are somewhere else)
They are of type "bpy_prop_array", which is some kind of Blender-defined Python data structure:
https://github.com/sobotka/blender/blob ... na.c#L6011
Another way would be to get a pointer to the
SmokeDomainSettings struct.
But it seems complicated to retrieve the smoke grids from this struct (requires mutex locking, and the memory has to be copied anyway):
https://github.com/sobotka/blender/blob ... oke.c#L235
I will look into how to construct the correct path to the OpenVDB files, it should be doable.
Re: OpenVDB support
Posted: Thu Apr 12, 2018 11:16 am
by Dade
I have added a new set of methods to Property Python object: AddAllBool, AddAllInt, AddUnsignedLongLong and AddAllFloat. It is a fast path for adding large Python array (or list) to a property (like with density grid ".data" field). It should be as fast as having a dedicated method but it is a general solution we can use in other place too.
This code:
Code: Select all
size = 2000000
a = array('f', [i for i in range(size)])
prop = pyluxcore.Property("test.array", [])
# Test Add
start = time.clock()
for i in range(size):
prop.Add([a[i]])
end = time.clock()
print("Add test: %.2gs" % (end-start))
prop = pyluxcore.Property("test.array", [])
# Test AddAll
start = time.clock()
prop.AddAllFloat(a)
end = time.clock()
print("AddAll test: %.2gs" % (end-start))
The result is:
Code: Select all
Add test: 3.2s
AddAll test: 0.042s
So it is 76 times faster. It should work "bpy_prop_array" object so you can add the complete grid with just a single "AddAllFloat()" call.