I was wondering about this (I'm assuming by render farm you refer to network rendering?). What's the vision for how network render will work? Will it still composite inside of Blender, or will it involve export to a standalone renderer like before? Network rendering was actually the very first reason I started using Lux, so I'm really excited to try out this feature when it's available!
Network rendering support
Network rendering support
Re: BlendLuxCore Development
I'm sure Dade will make a big post about it when it's finished, explaining everything.
Re: Network rendering support
The general plan
I'm working on the network rendering support. I'm currently developing 2 tools: pyluxcorenetconsole and pyluxcorenetnode. They are command line tools and a GUI version will follow. You lunch pyluxcorenetnode everywhere you want to run the renderings. pyluxcorenetconsole is like the old luxconsole, you pass as arguments the scenes to render.
They are going to be 2 rendering modes supported:
I have pyluxcorenetconsole and pyluxcorenetnode up and running on my local network. I expect to have full support for "Single image rendering" and "Local network connected" nodes in v2.0.
Major differences with Classic Lux Network rendering
I'm working on the network rendering support. I'm currently developing 2 tools: pyluxcorenetconsole and pyluxcorenetnode. They are command line tools and a GUI version will follow. You lunch pyluxcorenetnode everywhere you want to run the renderings. pyluxcorenetconsole is like the old luxconsole, you pass as arguments the scenes to render.
They are going to be 2 rendering modes supported:
- Single image rendering: all the nodes are used to render the same scene. It is mostly useful for rendering still complex high-res images.
- Multi image rendering: each nodes render a single image in parallel with the others. It is mostly useful for rendering animations.
- Local network connected nodes: usually the PCs on your studio/home network. The are automatically discovered thanks to network broadcast.
- Cloud nodes: usually AWS EC2 rented servers. The are manually added (i.e. pass to pyluxcorenetconsole their IP addresses).
- Cloud batch services: AWS Batch (https://aws.amazon.com/batch/?nc1=h_ls) service for "fire-and-forget" renderings. It is like using a render farm service, you pass the scenes to render, disconnect and get a notification when the rendering is done. AWS Batch uses AWS EC2 spot (https://aws.amazon.com/ec2/spot/?nc1=h_ls) instances, they have an high discount (up to a 90%) and they can be a very cost effective solution.
I have pyluxcorenetconsole and pyluxcorenetnode up and running on my local network. I expect to have full support for "Single image rendering" and "Local network connected" nodes in v2.0.
Major differences with Classic Lux Network rendering
- auto discover of all node connected to the local network. No need to use any parameters, just run pyluxcorenetconsole and pyluxcorenetnode. Add/remove any node you want on the fly.
- pyluxcorenetconsole handles all the nodes in parallel (not like old network rendering where you download one film at time, etc.). It already feels quite faster than the old rendering support.
- pyluxcorenetconsole keep everything on disk and doesn't directly perform any rendering task. If you want use the same PC where pyluxcorenetconsole run for the rendering, you just run a pyluxcorenetnode too. To run pyluxcorenetconsole alone you need very little ram and cpu (only network bandwidth to comunicate with other nodes).
- the support for cloud services to perform "hardware-less" renderings with only on demand cost (no upfront hardware costs). For instance, you can have only a laptop a still render an hour long animation if you are ready for paying the Amazon costs for its services.
Re: Network rendering support
Yay! Can't wait to try this out! The planned changes address some of the biggest annoyances with previous network rendering (namely connecting / disconnecting nodes, and needing to dedicate the source computer to the render job). GUI will be nice though simply as a way to monitor progress and adjust camera settings on the fly. I do an awful lot of that, so guess I will have to be more careful with my settings before pushing out a job for now.
How will integration with Blender work? I can get BlendLuxCore to write the scene file without rendering if I want, but would be nice to also have a checkbox for "run external renderer" similar to before. Of course makes no sense to have that checkbox when the external renderer isn't ready, so maybe you already have it there and it's just hidden?
Re: Network rendering support
Great plan!
Nice to hear this.
A good Blender integration will be really useful.
Nice to hear this.
A good Blender integration will be really useful.
Re: Network rendering support
It is really nice, now it can even resume a previously interrupted network rendering (i.e. if the MD5 of the scene is the same) so you can basically stop/start/kill pyluxcorenetconsole, and any copy of pyluxcorenetnode, any number of times you want and they will do their best to just continue the rendering. Without the need of any specific command, they are should auto-magically work.
I will try to include pyluxcorenetconsole and pyluxcorenetnode in v2.0alpha2.
I will try to include pyluxcorenetconsole and pyluxcorenetnode in v2.0alpha2.
-
- Supporting Users
- Posts: 141
- Joined: Tue Jan 09, 2018 6:48 pm
Re: Network rendering support
@Dade
Took a fast look at Linux alpha2 and found luxcoreui broken cause of misconfigured structure.
Either the embree/libs must reside by the binaries ( SET(CMAKE_INSTALL_RPATH "$ORIGIN")
or you must change this to SET(CMAKE_INSTALL_RPATH "../lib/$ORIGIN") to keep dirs as is.
( if it worked for you its because you have systemwide installs of those deps too )
The latter implies to also use a lib dir then in addon, to not have to build with 2 references.
I was not able to test networking either, cause of more misreferenced embree. Gave up at that point.
OFFTOPIC: i set forumconfig to not show my online status, but i can see it still. Bug ? ( in old forum this worked )
Jens
Took a fast look at Linux alpha2 and found luxcoreui broken cause of misconfigured structure.
Either the embree/libs must reside by the binaries ( SET(CMAKE_INSTALL_RPATH "$ORIGIN")
or you must change this to SET(CMAKE_INSTALL_RPATH "../lib/$ORIGIN") to keep dirs as is.
( if it worked for you its because you have systemwide installs of those deps too )
The latter implies to also use a lib dir then in addon, to not have to build with 2 references.
I was not able to test networking either, cause of more misreferenced embree. Gave up at that point.
OFFTOPIC: i set forumconfig to not show my online status, but i can see it still. Bug ? ( in old forum this worked )
Jens
Re: Network rendering support
I also could not get the network tools to work on Linux.
But I tested them on Windows and they worked great.
By the way I had an idea for the future network manager UI:
Maybe we could make it so you can drag and drop a .blend file onto it and it automatically does a FILESAVER export to .bcf and starts rendering.
But I tested them on Windows and they worked great.
By the way I had an idea for the future network manager UI:
Maybe we could make it so you can drag and drop a .blend file onto it and it automatically does a FILESAVER export to .bcf and starts rendering.
Re: Network rendering support
It has always been like that, you have only to set LD_LIBRARY_PATH to point to "<installation path>/lib". As alternative fast work around, you can just copy the 3 .so where you want to execute luxcore/pyluxcorenode/etc.jensverwiebe wrote: ↑Sun Jan 21, 2018 4:20 pm @Dade
Took a fast look at Linux alpha2 and found luxcoreui broken cause of misconfigured structure.
Either the embree/libs must reside by the binaries ( SET(CMAKE_INSTALL_RPATH "$ORIGIN")
or you must change this to SET(CMAKE_INSTALL_RPATH "../lib/$ORIGIN") to keep dirs as is.
Note: Windows includes the DLLs directly inside the pyluxcoretools so it doesn't require to take the DLLs around, I will do the same in the next Linux release (but luxcoreui will still need the 3 .so).
The visibility state is saved for about 5 mins, may be re-rty later to check if you are still visible ?jensverwiebe wrote: ↑Sun Jan 21, 2018 4:20 pm OFFTOPIC: i set forumconfig to not show my online status, but i can see it still. Bug ? ( in old forum this worked )