Hi guys, I've written a script to render a bunch of .ply files (between 300 and 500). It is a loop that does basically this:
1 - Create an object from a .ply
2 - Render the scene
3 - Save it as .png
4 - Delete the object create in step 1
5 - Go back to step 1
(If anyone wants to see the script, it is attached)
Everything works fine for a while. I always manage to render a couple dozen of images. But it never goes all the way up to the last image. It hangs somewhere in the middle of the way. Always while sampling. The last thing I see written is "refreshing film" and no image is produced anymore.
The point where it freezes is random, so there is nothing wrong with a specific .ply file.
I would appreciate any kind of help.
My system:
Ubuntu 18.04
Blender 2.82a
Luxcorerender 2.3
GPUs: two GeForce RTX 2080 Ti
Rendering freezing
Forum rules
Please upload a testscene that allows developers to reproduce the problem, and attach some images.
Please upload a testscene that allows developers to reproduce the problem, and attach some images.
-
- Posts: 7
- Joined: Mon May 25, 2020 6:26 pm
Rendering freezing
- Attachments
-
- blender_script.zip
- (1.83 KiB) Downloaded 160 times
Re: Rendering freezing
It may be some kind of memory leak or similar, I try to run the script tomorrow.
Have you tried with LuxCoreRender v2.4 ? Have tried to run on a single GPU (the problem may be related to the use of multiple GPUs) ?
Have you tried with LuxCoreRender v2.4 ? Have tried to run on a single GPU (the problem may be related to the use of multiple GPUs) ?
-
- Posts: 7
- Joined: Mon May 25, 2020 6:26 pm
Re: Rendering freezing
Hi, @Dade. Thanks for the reply.
I also suspect of a memory leak. Maybe I'm not deleting everything that is necessary after using the object. But I don't know what that would be.
I've tried running the script on another machine with just one GPU. Same problem.
I haven't tried 2.4 yet. Thanks for this tip.
Its very kind of you to take a look at the script. To make your life easier, I've attached some .ply files that you could use to generate meaningful images.
You can execute the script using:
I also suspect of a memory leak. Maybe I'm not deleting everything that is necessary after using the object. But I don't know what that would be.
I've tried running the script on another machine with just one GPU. Same problem.
I haven't tried 2.4 yet. Thanks for this tip.
Its very kind of you to take a look at the script. To make your life easier, I've attached some .ply files that you could use to generate meaningful images.
You can execute the script using:
Code: Select all
blender --background --python blender_script.py -- path/to/ply/dir path/to/an/output/dir 0
- Attachments
-
- in.zip
- (1.21 MiB) Downloaded 177 times
-
- Posts: 7
- Joined: Mon May 25, 2020 6:26 pm
Re: Rendering freezing
I noticed a pattern:
This freezing happens always when I'm doing something else that is CPU intensive (In my case, running CFD simulations).
During the freeze, the Blender process is highly demanding the CPU and the GPUs are almost blowing fire out the air oulets. Nevertheless, no rendering progress is observed. I waited for 30 minutes and nothing. Before the freeze, I get one image every 20 seconds (aprox.).
If I'm not running anything else on the computer, i.e. just rendering, then I can make it render all the images.
I'm really puzzled.
This freezing happens always when I'm doing something else that is CPU intensive (In my case, running CFD simulations).
During the freeze, the Blender process is highly demanding the CPU and the GPUs are almost blowing fire out the air oulets. Nevertheless, no rendering progress is observed. I waited for 30 minutes and nothing. Before the freeze, I get one image every 20 seconds (aprox.).
If I'm not running anything else on the computer, i.e. just rendering, then I can make it render all the images.
I'm really puzzled.
Re: Rendering freezing
I'm trying to replicate the problem but I get a couple of errors (with latest v2.4 sources):
The first error may be the result of BlendLuxCore not expecting to run in background mode.
The second, get_list_of_ply_files() is defined after its usage (I don;t know if this is allowed in Python).
Code: Select all
david@Desktop-3900x:~/tmp/blenderscript$ ls -l
total 1260
-rw-rw-r-- 1 david david 4940 May 25 20:46 blender_script.py
-rw-r--r-- 1 david david 1875 May 27 15:31 blender_script.zip
drwxr-xr-x 2 david david 4096 May 25 19:48 in
-rw-r--r-- 1 david david 1268554 May 27 15:31 in.zip
drwxr-xr-x 2 david david 4096 May 27 15:33 out
david@Desktop-3900x:~/tmp/blenderscript$ ~/opt/blender-2.82a-linux64/blender --background --python blender_script.py -- in out
Blender 2.82 (sub 7) (hash 375c7dc4caf4 built 2020-03-12 05:30:40)
Read prefs: /home/david/.config/blender/2.82/config/userpref.blend
found bundled python: /home/david/opt/blender-2.82a-linux64/2.82/python
Exception in module register(): /home/david/.config/blender/2.82/scripts/addons/BlendLuxCore/__init__.py
Traceback (most recent call last):
File "/home/david/opt/blender-2.82a-linux64/2.82/scripts/modules/addon_utils.py", line 382, in enable
mod.register()
File "/home/david/.config/blender/2.82/scripts/addons/BlendLuxCore/__init__.py", line 86, in register
auto_load.register()
File "/home/david/.config/blender/2.82/scripts/addons/BlendLuxCore/auto_load.py", line 36, in register
module.register()
File "/home/david/.config/blender/2.82/scripts/addons/BlendLuxCore/operators/keymaps.py", line 9, in register
keymap = wm.keyconfigs.addon.keymaps.new(name="Node Editor", space_type="NODE_EDITOR")
AttributeError: 'NoneType' object has no attribute 'keymaps'
Traceback (most recent call last):
File "/home/david/tmp/blenderscript/blender_script.py", line 9, in <module>
ply_files = get_list_of_ply_files(ply_dir=argv[0])
NameError: name 'get_list_of_ply_files' is not defined
Blender quit
david@Desktop-3900x:~/tmp/blenderscript$
The second, get_list_of_ply_files() is defined after its usage (I don;t know if this is allowed in Python).
-
- Posts: 7
- Joined: Mon May 25, 2020 6:26 pm
Re: Rendering freezing
Hi Dade,
Interesting here, using v2.3, I can run on background without problems. But I'll try to run it on foreground to see if something changes.
Regarding the function definition error, I'm sorry. I've made some last minutes changes on the code to make it more readable before posting here. But I ended up making this mistake. Just copy the "if __name__" block to the end of the file and it should work. Like this:
Interesting here, using v2.3, I can run on background without problems. But I'll try to run it on foreground to see if something changes.
Regarding the function definition error, I'm sorry. I've made some last minutes changes on the code to make it more readable before posting here. But I ended up making this mistake. Just copy the "if __name__" block to the end of the file and it should work. Like this:
Code: Select all
import bpy
import os, sys, math
from glob import glob
def get_list_of_ply_files(ply_dir):
query = os.path.join(ply_dir, '*.ply')
ply_files = glob(query)
return sorted(ply_files)
def delete_everything():
bpy.ops.object.select_all(action='SELECT')
bpy.ops.object.delete(use_global=False)
for m in bpy.data.materials:
bpy.data.materials.remove(m)
for c in bpy.data.cameras:
bpy.data.cameras.remove(c)
for l in bpy.data.lights:
bpy.data.lights.remove(l)
def setup_scene():
# Luxcore engine options
bpy.context.scene.render.engine = 'LUXCORE'
bpy.context.scene.luxcore.config.device = 'OCL'
bpy.context.scene.luxcore.denoiser.enabled = True
bpy.context.scene.luxcore.opencl.use_native_cpu = False
# Rendering stop criterium (halt condition)
bpy.context.scene.luxcore.halt.enable = True
bpy.context.scene.luxcore.halt.use_samples = True
bpy.context.scene.luxcore.halt.samples = 1000
# Output image properties
x_res = 150
bpy.context.scene.render.resolution_x = x_res
bpy.context.scene.render.resolution_y = int(x_res * 5.5)
bpy.context.scene.render.image_settings.color_mode = 'RGBA'
# Turn-off world's natural light
bpy.context.scene.world.luxcore.light = 'none'
# Setup compositor
bpy.context.scene.use_nodes = True
render_layers = bpy.context.scene.node_tree.nodes['Render Layers']
composite = bpy.context.scene.node_tree.nodes['Composite']
links = bpy.context.scene.node_tree.links
links.remove(links[0])
links.new(render_layers.outputs['Alpha'], composite.inputs['Alpha'])
links.new(render_layers.outputs['DENOISED'], composite.inputs['Image'])
def create_light():
bpy.ops.object.light_add(
type='AREA',
location=(-0.01, 0, 0),
rotation=(0, -math.pi/2., 0.),
)
backlight = bpy.context.active_object
backlight.name = 'Backlight'
backlight.data.luxcore.is_laser = True
def create_camera():
bpy.ops.object.camera_add(
location=(.02, 0., 2.75E-3),
rotation=(math.pi/2., 0., math.pi/2.),
)
camera = bpy.context.active_object
camera.data.type = 'ORTHO'
camera.data.ortho_scale = 0.0055
camera.data.luxcore.use_clipping = False
camera.data.luxcore.imagepipeline.transparent_film = True
bpy.context.scene.camera = camera
def create_material():
material = bpy.data.materials.new(name="water")
material.luxcore.node_tree = bpy.data.node_groups.new(
name="water_node_tree",
type="luxcore_material_nodes",
)
material.luxcore.node_tree.fake_user = True
nodes = material.luxcore.node_tree.nodes
mat_output = nodes.new("LuxCoreNodeMatOutput")
mat_output.location = (200, 0)
mat_specs = nodes.new("LuxCoreNodeMatGlass")
mat_specs.location = (0, 0)
mat_specs.rough = True
mat_specs.inputs["Roughness"].default_value = 0.02
mat_specs.inputs["IOR"].default_value = 1.33
mat_specs.inputs["Opacity"].default_value = 1.0
material.luxcore.node_tree.links.new(
mat_specs.outputs["Material"],
mat_output.inputs["Material"])
return material
def create_fluid(ply_file_path, material, angle, smooth=True):
# Import ply geometry
bpy.ops.import_mesh.ply(filepath=ply_file_path)
fluid = bpy.context.active_object
fluid.rotation_euler[2] = angle
if smooth:
# Corrective smooth modifier (tries to keep volume constant)
fluid.modifiers.new(name="smoother", type='CORRECTIVE_SMOOTH')
fluid.modifiers["smoother"].factor = 0.5
fluid.modifiers["smoother"].iterations = 30
fluid.modifiers["smoother"].smooth_type = 'LENGTH_WEIGHTED'
fluid.modifiers["smoother"].use_only_smooth = True
fluid.modifiers["smoother"].use_pin_boundary = True
bpy.ops.object.modifier_apply(modifier="smoother")
# Smooth shader
bpy.ops.object.shade_smooth()
# Apply water material
fluid.data.materials.append(material)
return fluid
def delete_fluid(fluid):
bpy.ops.object.select_all(action='DESELECT')
fluid.select_set(True)
bpy.ops.object.delete()
def render_scene(file_name):
bpy.context.scene.render.filepath = file_name
bpy.ops.render.render(write_still=True)
if __name__ == "__main__":
# Read command line arguments
argv = sys.argv
argv = argv[argv.index("--") + 1:] # get all args after "--"
ply_files = get_list_of_ply_files(ply_dir=argv[0])
render_dir = argv[1]
angle = float(argv[2])
# Stuff that is common for all the images
delete_everything()
setup_scene()
create_camera()
create_light()
material = create_material()
for ply_file in ply_files:
fluid = create_fluid(ply_file, material, angle)
image_path = os.path.join(render_dir, fluid.name + '.png')
render_scene(image_path)
delete_fluid(fluid)