Yes but isn't this problem totally solved by OpenVDB ?B.Y.O.B. wrote: ↑Mon Apr 09, 2018 9:29 pm By the way can we accelerate the export of the normal point cache with C++?
Have to investigate. I think we would need a C++ function "DefineDensityGridData()" (like DefineMesh etc.) because the main bottleneck seems to be that we currently pass a HUGE array to a property.
OpenVDB support
Re: OpenVDB support
Re: OpenVDB support
Yes, but OpenVDB is not the default and most users will still use the old point cache.
But yes, if it turns out to be too much work then I'll just add a label to the UI that says "use OpenVDB for fast smoke rendering".
But yes, if it turns out to be too much work then I'll just add a label to the UI that says "use OpenVDB for fast smoke rendering".
Re: OpenVDB support
I merged all branches and now OpenVDB is part of the main LuxCoreRender branch.
I'm going to add the support for half storage in order to reduce the memory footprint.
Don't we need support for 3xFloat data too ? To express density grids with RGB values (not just a single scalar)
I'm going to add the support for half storage in order to reduce the memory footprint.
Don't we need support for 3xFloat data too ? To express density grids with RGB values (not just a single scalar)
Re: OpenVDB support
Would be great!
I think Blender supports smoke sims with mixing different colors and has a color_grid, but currently we don't support it.
edit: had a look and the Blender color grid is 4 times the size of the density grid.
So I guess it is a one dimensional float array with values r, g, b, a, r, g, b, a, ... (note the alpha component).
Obviously it is not documented in the API. https://docs.blender.org/api/current/bp ... color_grid
Re: OpenVDB support
I will add the support for RGB values.B.Y.O.B. wrote: ↑Tue Apr 10, 2018 4:10 pmWould be great!
I think Blender supports smoke sims with mixing different colors and has a color_grid, but currently we don't support it.
edit: had a look and the Blender color grid is 4 times the size of the density grid.
So I guess it is a one dimensional float array with values r, g, b, a, r, g, b, a, ... (note the alpha component).
Obviously it is not documented in the API. https://docs.blender.org/api/current/bp ... color_grid
Re: OpenVDB support
I have added the support for DensityGrid storage type with the ".storage" property. The supported storage are the same of image maps:
- byte
- half
- float
- auto
The default value (and auto) is half format. The "half" format cut in half the memory requirements and it is now the default. The "byte" format may not have enough resolution for some application but it is very effective to reduce the memory usage. This is the bunny rendered with byte storage:
It uses 1/4 of memory required by float storage.
I have also added the support for vector data (i.e. RGB). They can be defined with the ".data3" property instead of ".data". Indeed the number of expected values for ".data3" is nx * ny * nz * 3. OpenVDB supports automatically detect if the grid is scalar or vector.
- byte
- half
- float
- auto
The default value (and auto) is half format. The "half" format cut in half the memory requirements and it is now the default. The "byte" format may not have enough resolution for some application but it is very effective to reduce the memory usage. This is the bunny rendered with byte storage:
It uses 1/4 of memory required by float storage.
I have also added the support for vector data (i.e. RGB). They can be defined with the ".data3" property instead of ".data". Indeed the number of expected values for ".data3" is nx * ny * nz * 3. OpenVDB supports automatically detect if the grid is scalar or vector.
Re: OpenVDB support
Found it!
object.modifiers["Smoke"].domain_settings.point_cache.filepath
Now I have to figure out where exactly to find the cache (because if the filepath is empty, I think a folder is created in the save location of the .blend file, and it can have a name but it does not have to, and I have to account for smoke sims linked in through a library etc. pp.)
To be honest it would be much more elegant to have a fast way to transfer Blender's grid data to LuxCore.
It would even save RAM if we could just specify a pointer to the start of the array in Blender's memory (no need to copy the data).
(But then again this only works for CPU renders...)
I'm sure the OpenVDB support will be used by someone else someday, but for Blender it is a bit complicated.
I should have investigated before you started to implement it...
Re: OpenVDB support
From Python, we can access this class: https://docs.blender.org/api/current/bp ... inSettings
It contains the various grids: density_grid, color_grid, ... (but only in Python - in the C code behind the scenes they are somewhere else)
They are of type "bpy_prop_array", which is some kind of Blender-defined Python data structure: https://github.com/sobotka/blender/blob ... na.c#L6011
Another way would be to get a pointer to the SmokeDomainSettings struct.
But it seems complicated to retrieve the smoke grids from this struct (requires mutex locking, and the memory has to be copied anyway):
https://github.com/sobotka/blender/blob ... oke.c#L235
I will look into how to construct the correct path to the OpenVDB files, it should be doable.
It contains the various grids: density_grid, color_grid, ... (but only in Python - in the C code behind the scenes they are somewhere else)
They are of type "bpy_prop_array", which is some kind of Blender-defined Python data structure: https://github.com/sobotka/blender/blob ... na.c#L6011
Another way would be to get a pointer to the SmokeDomainSettings struct.
But it seems complicated to retrieve the smoke grids from this struct (requires mutex locking, and the memory has to be copied anyway):
https://github.com/sobotka/blender/blob ... oke.c#L235
I will look into how to construct the correct path to the OpenVDB files, it should be doable.
Re: OpenVDB support
I have added a new set of methods to Property Python object: AddAllBool, AddAllInt, AddUnsignedLongLong and AddAllFloat. It is a fast path for adding large Python array (or list) to a property (like with density grid ".data" field). It should be as fast as having a dedicated method but it is a general solution we can use in other place too.B.Y.O.B. wrote: ↑Thu Apr 12, 2018 6:42 am From Python, we can access this class: https://docs.blender.org/api/current/bp ... inSettings
It contains the various grids: density_grid, color_grid, ... (but only in Python - in the C code behind the scenes they are somewhere else)
They are of type "bpy_prop_array", which is some kind of Blender-defined Python data structure: https://github.com/sobotka/blender/blob ... na.c#L6011
This code:
Code: Select all
size = 2000000
a = array('f', [i for i in range(size)])
prop = pyluxcore.Property("test.array", [])
# Test Add
start = time.clock()
for i in range(size):
prop.Add([a[i]])
end = time.clock()
print("Add test: %.2gs" % (end-start))
prop = pyluxcore.Property("test.array", [])
# Test AddAll
start = time.clock()
prop.AddAllFloat(a)
end = time.clock()
print("AddAll test: %.2gs" % (end-start))
Code: Select all
Add test: 3.2s
AddAll test: 0.042s