Network rendering support

Discussion related to the Engine functionality, implementations and API.
jensverwiebe
Supporting Users
Posts: 141
Joined: Tue Jan 09, 2018 6:48 pm

Re: Network rendering support

Post by jensverwiebe » Tue Jan 30, 2018 10:24 pm

Sharlybg wrote:
Tue Jan 30, 2018 10:18 pm
Just forget to ask and propose if possible :

__ i don't know the GUI tools behind blender and old luxrender UI but assume the licence here must bring what we need.
Blender has an completely own gui written with ogl draws ( ghost ) and i fear Qt along with apache might be incompatible.
I may be wrong with the latter.
IMGui might be it as we already use it, have experience with and i saw really complex programs done with it.

Jens

jensverwiebe
Supporting Users
Posts: 141
Joined: Tue Jan 09, 2018 6:48 pm

Re: Network rendering support

Post by jensverwiebe » Sun Feb 04, 2018 2:05 pm

Followup from the alpha3 announce discussion:

This is what i do atm to keep the package small and keep a good overlook ( wip ):

1: have a simplified dir:
screenshot-window-2018-02-04-145854.png
( luxcoreui has now an '$ORIGIN/lib' rpath ! , pyluxcorebla.py's are included in /lib )

2: Use one cmd script ( could be a binary as well ) to acces all cmd's ( todo: merger, done ):

Code: Select all

#!/bin/bash

sleep 0.2  # just to let the terminal appear and become active
wmctrl -r :ACTIVE: -e 5,0,0,860,440 # set left/top, experimental

function jumpto
{
    label=$1
    cmd=$(sed -n "/$label:/{:a;n;p;ba};" $0 | grep -v ':$')
    eval "$cmd"
    exit
}

start=${1:-"start"}
filename=${1:-"filename"}
jumpto $start

start:
pause=0
echo Choose task: [S]ingleConsole \ Net[C]onsole \ Net[N]ode \ [M]erge \( Leave empty to abort \)
read task

case "$task" in
         "") echo "Canceled by user ..."; sleep 1; exit
            ;;
        S|s) echo "Setup Console"
	action=console
            ;;
        C|c)  echo "Setup NetConsole"
	action=netconsole
            ;;
        N|n) echo "Setup NetNode"
	action=netnode
            ;;
        M|m) echo "Setup Merge" 
	action=merge
            ;;
        *) echo "Invalid choice ..."; sleep 1; jumpto $start
            ;;
esac

echo Set extra arguments here \(-h shows helpchart\) \( + return \)
read args
if [ $args == "" ]; then
	echo "No arguments"
else
	echo "Arguments: $args"
fi

if [ "$args" == "-h" ]; then
	pause=10
	echo "Holding help-page for $pause seconds"
elif [ "$task" == "m" ]; then
	pause=3
	echo "Holding result-page for $pause seconds"
fi

filename:
if [[ "$task" != "n" && "$args" != "-h" ]]; then
	echo "Drop or type path(s) to file(s)"
	read filepath
	if [ "$filepath" == "" ]
		then echo "Missing file(s), try again ..."; jumpto $filename
	fi
fi

echo Starting $action
gnome-terminal -e '/bin/bash -c "python3 ./lib/pyluxcore'$action'.py '"$args"' '"$filepath"'; sleep '$pause'" '

jumpto $start
Remark: the sleeps are needed to keep the helppage open a while.

This package is much smaller than the orig. alpha3 ( 90mb vs. 220mb uncompressed, 24mb vs. 140mb bz2 compressed )
and does not need any install. I have already some rework in mind, is on todo.

EDIT: did a few optimizations, added merger

Jens

User avatar
Sharlybg
Supporting Users
Posts: 885
Joined: Mon Dec 04, 2017 10:11 pm
Location: Ivory Coast

Re: Network rendering support

Post by Sharlybg » Thu Oct 04, 2018 2:45 pm

- I have done the first rendering with a AWS EC2 node in the mix. I was using my home i7 3930K+i7 860 and a remote Intel(R) Xeon(R) CPU E5-2666 v3 @ 2.90GHz (I think it has 18 cores/36 threads) with 60GB ram from AWS Frankfurt data center. The network rendering seems to works well even over internet: It was transferring a 1024x1024 film in less than 2 seconds.

The rendering speed of the AWS Xeon isn't very impressive (even if I like to compile LuxCore with a "make -j 36" :D ) however, the point here is that you can literally lunch any number of AWS EC2 instances you want/can pay.
Hi Dade ! do you think we can render Animation now with AWS EC2 and Luxcore now ?

And What the steps if it is possible ?
Support LuxCoreRender project with salts and bounties

Portfolio : https://www.behance.net/DRAVIA

User avatar
Dade
Developer
Posts: 1538
Joined: Mon Dec 04, 2017 8:36 pm

Re: Network rendering support

Post by Dade » Thu Oct 04, 2018 3:46 pm

Sharlybg wrote:
Thu Oct 04, 2018 2:45 pm
Hi Dade ! do you think we can render Animation now with AWS EC2 and Luxcore now ?

And What the steps if it is possible ?
It is possible but renting one or more high end machine on AWS is quite expansive (and complex). AWS is not intended for end-user usage and you need some good know-how to be able to handle VM creation, etc. on your own.

You can find the prices here: https://aws.amazon.com/ec2/pricing/on-demand/

The point would be to use Spot instances instead of normal one (there is a huge discount), however all the management is quite complex (i.e. someone would have to setup a render farm service and than sell the service so the end-user would see only a front-end and not the AWS back-end).
Support LuxCoreRender project with salts and bounties

User avatar
Sharlybg
Supporting Users
Posts: 885
Joined: Mon Dec 04, 2017 10:11 pm
Location: Ivory Coast

Re: Network rendering support

Post by Sharlybg » Thu Oct 04, 2018 4:13 pm

The point would be to use Spot instances instead of normal one (there is a huge discount), however all the management is quite complex (i.e. someone would have to setup a render farm service and than sell the service so the end-user would see only a front-end and not the AWS back-end).
Renderbot ? any possibility to add Luxcore 2.0 support ?

http://renderbot.net/
Support LuxCoreRender project with salts and bounties

Portfolio : https://www.behance.net/DRAVIA

User avatar
Dade
Developer
Posts: 1538
Joined: Mon Dec 04, 2017 8:36 pm

Re: Network rendering support

Post by Dade » Thu Oct 04, 2018 4:30 pm

Sharlybg wrote:
Thu Oct 04, 2018 4:13 pm
The point would be to use Spot instances instead of normal one (there is a huge discount), however all the management is quite complex (i.e. someone would have to setup a render farm service and than sell the service so the end-user would see only a front-end and not the AWS back-end).
Renderbot ? any possibility to add Luxcore 2.0 support ?

http://renderbot.net/
Yes, but has it been abandoned ?
Support LuxCoreRender project with salts and bounties

User avatar
Sharlybg
Supporting Users
Posts: 885
Joined: Mon Dec 04, 2017 10:11 pm
Location: Ivory Coast

Re: Network rendering support

Post by Sharlybg » Fri Oct 05, 2018 11:22 am

Yes, but has it been abandoned ?
No but it is only for cycles render.
The point would be to use Spot instances instead of normal one (there is a huge discount), however all the management is quite complex (i.e. someone would have to setup a render farm service and than sell the service so the end-user would see only a front-end and not the AWS back-end).
I really want to try following a steps by steps instructions from you or BYOB to be able to set VM for Luxcore.
Support LuxCoreRender project with salts and bounties

Portfolio : https://www.behance.net/DRAVIA

User avatar
Sharlybg
Supporting Users
Posts: 885
Joined: Mon Dec 04, 2017 10:11 pm
Location: Ivory Coast

Re: Network rendering support

Post by Sharlybg » Fri Oct 05, 2018 12:10 pm

I've created my account on AWS. They want me to select an AMI. Going to learn and familliarize a bit with it.
Support LuxCoreRender project with salts and bounties

Portfolio : https://www.behance.net/DRAVIA

User avatar
Dade
Developer
Posts: 1538
Joined: Mon Dec 04, 2017 8:36 pm

Re: Network rendering support

Post by Dade » Fri Oct 05, 2018 1:33 pm

Sharlybg wrote:
Fri Oct 05, 2018 12:10 pm
I've created my account on AWS. They want me to select an AMI. Going to learn and familliarize a bit with it.
You can use a standard Ubuntu 14.04LTS or 16, install standalone LuxCore and use the command line tools.
Support LuxCoreRender project with salts and bounties

User avatar
Sharlybg
Supporting Users
Posts: 885
Joined: Mon Dec 04, 2017 10:11 pm
Location: Ivory Coast

Re: Network rendering support

Post by Sharlybg » Fri Oct 05, 2018 1:47 pm

You can use a standard Ubuntu 14.04LTS or 16, install standalone LuxCore and use the command line tools.
i'm already connected to a Window server ( fear a bit about linux , not a great expert of it ). i had follow the instructions on the Website and manage to connect to the remote instance ( the free one t2 micro ). they suggest me to create a second administrator account for backup if error occur , wich i finally get done.

The remote windows desktop is open. now i need to install blender 2.79b and luxcore alpha 4 normally ?
Support LuxCoreRender project with salts and bounties

Portfolio : https://www.behance.net/DRAVIA

Post Reply