Rendering freezing

[gui-miotto]

Hi guys, I've written a script to render a bunch of .ply files (between 300 and 500). It is a loop that does basically this:

1 - Create an object from a .ply
2 - Render the scene
3 - Save it as .png
4 - Delete the object create in step 1
5 - Go back to step 1

(If anyone wants to see the script, it is attached)

Everything works fine for a while. I always manage to render a couple dozen of images. But it never goes all the way up to the last image. It hangs somewhere in the middle of the way. Always while sampling. The last thing I see written is "refreshing film" and no image is produced anymore.
The point where it freezes is random, so there is nothing wrong with a specific .ply file.

I would appreciate any kind of help.

My system:
Ubuntu 18.04
Blender 2.82a
Luxcorerender 2.3
GPUs: two GeForce RTX 2080 Ti

[Dade]

It may be some kind of memory leak or similar, I try to run the script tomorrow.

Have you tried with LuxCoreRender v2.4 ? Have tried to run on a single GPU (the problem may be related to the use of multiple GPUs) ?

[gui-miotto]

Hi, @Dade. Thanks for the reply.

I also suspect of a memory leak. Maybe I'm not deleting everything that is necessary after using the object. But I don't know what that would be.

I've tried running the script on another machine with just one GPU. Same problem.
I haven't tried 2.4 yet. Thanks for this tip.

Its very kind of you to take a look at the script. To make your life easier, I've attached some .ply files that you could use to generate meaningful images.

You can execute the script using:

Code: Select all

blender --background --python blender_script.py -- path/to/ply/dir path/to/an/output/dir 0

[gui-miotto]

I noticed a pattern:

This freezing happens always when I'm doing something else that is CPU intensive (In my case, running CFD simulations).
During the freeze, the Blender process is highly demanding the CPU and the GPUs are almost blowing fire out the air oulets. Nevertheless, no rendering progress is observed. I waited for 30 minutes and nothing. Before the freeze, I get one image every 20 seconds (aprox.).

If I'm not running anything else on the computer, i.e. just rendering, then I can make it render all the images.

I'm really puzzled.

[Dade]

I'm trying to replicate the problem but I get a couple of errors (with latest v2.4 sources):

Code: Select all

david@Desktop-3900x:~/tmp/blenderscript$ ls -l 
total 1260
-rw-rw-r-- 1 david david    4940 May 25 20:46 blender_script.py
-rw-r--r-- 1 david david    1875 May 27 15:31 blender_script.zip
drwxr-xr-x 2 david david    4096 May 25 19:48 in
-rw-r--r-- 1 david david 1268554 May 27 15:31 in.zip
drwxr-xr-x 2 david david    4096 May 27 15:33 out
david@Desktop-3900x:~/tmp/blenderscript$ ~/opt/blender-2.82a-linux64/blender --background --python blender_script.py -- in out
Blender 2.82 (sub 7) (hash 375c7dc4caf4 built 2020-03-12 05:30:40)
Read prefs: /home/david/.config/blender/2.82/config/userpref.blend
found bundled python: /home/david/opt/blender-2.82a-linux64/2.82/python
Exception in module register(): /home/david/.config/blender/2.82/scripts/addons/BlendLuxCore/__init__.py
Traceback (most recent call last):
  File "/home/david/opt/blender-2.82a-linux64/2.82/scripts/modules/addon_utils.py", line 382, in enable
    mod.register()
  File "/home/david/.config/blender/2.82/scripts/addons/BlendLuxCore/__init__.py", line 86, in register
    auto_load.register()
  File "/home/david/.config/blender/2.82/scripts/addons/BlendLuxCore/auto_load.py", line 36, in register
    module.register()
  File "/home/david/.config/blender/2.82/scripts/addons/BlendLuxCore/operators/keymaps.py", line 9, in register
    keymap = wm.keyconfigs.addon.keymaps.new(name="Node Editor", space_type="NODE_EDITOR")
AttributeError: 'NoneType' object has no attribute 'keymaps'
Traceback (most recent call last):
  File "/home/david/tmp/blenderscript/blender_script.py", line 9, in <module>
    ply_files = get_list_of_ply_files(ply_dir=argv[0])
NameError: name 'get_list_of_ply_files' is not defined

Blender quit
david@Desktop-3900x:~/tmp/blenderscript$ 

The first error may be the result of BlendLuxCore not expecting to run in background mode.

The second, get_list_of_ply_files() is defined after its usage (I don;t know if this is allowed in Python).

[gui-miotto]

Hi Dade,

Interesting here, using v2.3, I can run on background without problems. But I'll try to run it on foreground to see if something changes.

Regarding the function definition error, I'm sorry. I've made some last minutes changes on the code to make it more readable before posting here. But I ended up making this mistake. Just copy the "if __name__" block to the end of the file and it should work. Like this:

Code: Select all

import bpy
import os, sys, math
from glob import glob

def get_list_of_ply_files(ply_dir):
    query = os.path.join(ply_dir, '*.ply')
    ply_files = glob(query)
    return sorted(ply_files)


def delete_everything():
    bpy.ops.object.select_all(action='SELECT')
    bpy.ops.object.delete(use_global=False)
    for m in bpy.data.materials:
        bpy.data.materials.remove(m)
    for c in bpy.data.cameras:
        bpy.data.cameras.remove(c)
    for l in bpy.data.lights:
        bpy.data.lights.remove(l)


def setup_scene():
    # Luxcore engine options
    bpy.context.scene.render.engine = 'LUXCORE'
    bpy.context.scene.luxcore.config.device = 'OCL'
    bpy.context.scene.luxcore.denoiser.enabled = True
    bpy.context.scene.luxcore.opencl.use_native_cpu = False

    # Rendering stop criterium (halt condition)
    bpy.context.scene.luxcore.halt.enable = True
    bpy.context.scene.luxcore.halt.use_samples = True
    bpy.context.scene.luxcore.halt.samples = 1000

    # Output image properties
    x_res = 150
    bpy.context.scene.render.resolution_x = x_res
    bpy.context.scene.render.resolution_y = int(x_res * 5.5)
    bpy.context.scene.render.image_settings.color_mode = 'RGBA'

    # Turn-off world's natural light
    bpy.context.scene.world.luxcore.light = 'none'

    # Setup compositor
    bpy.context.scene.use_nodes = True
    render_layers = bpy.context.scene.node_tree.nodes['Render Layers']
    composite = bpy.context.scene.node_tree.nodes['Composite']
    links = bpy.context.scene.node_tree.links
    links.remove(links[0])
    links.new(render_layers.outputs['Alpha'], composite.inputs['Alpha'])
    links.new(render_layers.outputs['DENOISED'], composite.inputs['Image'])


def create_light():
    bpy.ops.object.light_add(
        type='AREA',
        location=(-0.01, 0, 0),
        rotation=(0, -math.pi/2., 0.),
        )
    backlight = bpy.context.active_object
    backlight.name = 'Backlight'
    backlight.data.luxcore.is_laser = True


def create_camera():
    bpy.ops.object.camera_add(
        location=(.02, 0., 2.75E-3),
        rotation=(math.pi/2., 0., math.pi/2.),
        )
    camera = bpy.context.active_object
    camera.data.type = 'ORTHO'
    camera.data.ortho_scale = 0.0055
    camera.data.luxcore.use_clipping = False
    camera.data.luxcore.imagepipeline.transparent_film = True
    bpy.context.scene.camera = camera


def create_material():
    material = bpy.data.materials.new(name="water")
    material.luxcore.node_tree = bpy.data.node_groups.new(
        name="water_node_tree",
        type="luxcore_material_nodes",
        )
    material.luxcore.node_tree.fake_user = True
    nodes = material.luxcore.node_tree.nodes

    mat_output = nodes.new("LuxCoreNodeMatOutput")
    mat_output.location = (200, 0)

    mat_specs = nodes.new("LuxCoreNodeMatGlass")
    mat_specs.location = (0, 0)
    mat_specs.rough = True
    mat_specs.inputs["Roughness"].default_value = 0.02
    mat_specs.inputs["IOR"].default_value = 1.33
    mat_specs.inputs["Opacity"].default_value = 1.0

    material.luxcore.node_tree.links.new(
        mat_specs.outputs["Material"],
        mat_output.inputs["Material"])

    return material


def create_fluid(ply_file_path, material, angle, smooth=True):
    # Import ply geometry
    bpy.ops.import_mesh.ply(filepath=ply_file_path)
    fluid = bpy.context.active_object
    fluid.rotation_euler[2] = angle

    if smooth:
        # Corrective smooth modifier (tries to keep volume constant)
        fluid.modifiers.new(name="smoother", type='CORRECTIVE_SMOOTH')
        fluid.modifiers["smoother"].factor = 0.5
        fluid.modifiers["smoother"].iterations = 30
        fluid.modifiers["smoother"].smooth_type = 'LENGTH_WEIGHTED'
        fluid.modifiers["smoother"].use_only_smooth = True
        fluid.modifiers["smoother"].use_pin_boundary = True
        bpy.ops.object.modifier_apply(modifier="smoother")
        # Smooth shader
        bpy.ops.object.shade_smooth()

    # Apply water material
    fluid.data.materials.append(material)

    return fluid

def delete_fluid(fluid):
    bpy.ops.object.select_all(action='DESELECT')
    fluid.select_set(True)
    bpy.ops.object.delete()

def render_scene(file_name):
    bpy.context.scene.render.filepath = file_name
    bpy.ops.render.render(write_still=True)

if __name__ == "__main__":
    # Read command line arguments
    argv = sys.argv
    argv = argv[argv.index("--") + 1:]  # get all args after "--"
    ply_files = get_list_of_ply_files(ply_dir=argv[0])
    render_dir = argv[1]
    angle = float(argv[2])

    # Stuff that is common for all the images
    delete_everything()
    setup_scene()
    create_camera()
    create_light()
    material = create_material()

    for ply_file in ply_files:
        fluid = create_fluid(ply_file, material, angle)
        image_path = os.path.join(render_dir, fluid.name + '.png')
        render_scene(image_path)
        delete_fluid(fluid)