Network rendering support
Re: Network rendering support
Thanks for the summary. Then I'll look into wxPython.
Re: Network rendering support
fall on this on the web : https://www.josephrex.me/getting-starte ... in-python/Dade wrote: ↑Sun Jan 28, 2018 1:06 pmThe problem is the network rendering code is now written in Python so we would have C++ GUI calling Python network code (something still to explore) and than Python network code calling C++ pyluxcore, it looks like a recipe for troubles.
As far as I have seen at the moment:
- https://www.wxpython.org/ looks like a good option;
- I really like Qt but I have been able to understand the dual license mess of https://www.riverbankcomputing.com/soft ... /download5 and if we can use it;
- https://wiki.qt.io/PySide is another Qt, the license is ok but it looks like it is not actively developed anymore.
- Tk: I was using Tcl/Tk 30 years ago, the GUI looks prehistoric but Tk is included in Python, this is a major selling point.
Wx looks like the best option at the moment for developing all PyLuxCoreTools GUIs but I'm quite confused.
and this the conclusion of the author :
Conclusion:
Building GUI programs may be a challenge but as the world evolves, it is very necessary. You should also not directly use generated code from pyuic because you may want to make changes to the UI at any point. Each compilation with pyuic will make you lose your changes you’ve made on the generated code. I’ve chosen Qt as my best module for building GUI. You may choose something else that works for you but from what I hear, Tkinter has a lot of downsides so it’s not recommended.
Re: Network rendering support
Sure, PyQt looks good and I already know Qt from C++ programming, but if the licensing situation is unclear as Dade said we might not be able to use it.
PyQt is licensed under GPL. LuxCore API is using Apache license.
Maybe we just need to license the network manager GUI under GPL - is it a problem if GPL code calls an Apache library?
edit: I also found this, which seems to make the same point as I:
(Really annoying that you have to become a hobby lawyer if you are a developer)
PyQt is licensed under GPL. LuxCore API is using Apache license.
Maybe we just need to license the network manager GUI under GPL - is it a problem if GPL code calls an Apache library?
edit: I also found this, which seems to make the same point as I:
edit2: Something else: https://gist.github.com/takluyver/f7d95b0354a01bae2428http://hselab.org/pyqt5-getting-started.html wrote:Since hillmaker is Apache 2.0 licensed, should be no problem to use PyQt but then the combined package might have to be GPL licensed (not that that's a deal breaker).
(Really annoying that you have to become a hobby lawyer if you are a developer)
Re: Network rendering support
Some progress with network rendering:
- it is now possible to rendering nodes with OpenCL support and nodes without (i.e. mix PATHOCL and PATHCPU for instance).
- I have done the first rendering with a AWS EC2 node in the mix. I was using my home i7 3930K+i7 860 and a remote Intel(R) Xeon(R) CPU E5-2666 v3 @ 2.90GHz (I think it has 18 cores/36 threads) with 60GB ram from AWS Frankfurt data center. The network rendering seems to works well even over internet: It was transferring a 1024x1024 film in less than 2 seconds.
The rendering speed of the AWS Xeon isn't very impressive (even if I like to compile LuxCore with a "make -j 36" ) however, the point here is that you can literally lunch any number of AWS EC2 instances you want/can pay.
- it is now possible to rendering nodes with OpenCL support and nodes without (i.e. mix PATHOCL and PATHCPU for instance).
- I have done the first rendering with a AWS EC2 node in the mix. I was using my home i7 3930K+i7 860 and a remote Intel(R) Xeon(R) CPU E5-2666 v3 @ 2.90GHz (I think it has 18 cores/36 threads) with 60GB ram from AWS Frankfurt data center. The network rendering seems to works well even over internet: It was transferring a 1024x1024 film in less than 2 seconds.
The rendering speed of the AWS Xeon isn't very impressive (even if I like to compile LuxCore with a "make -j 36" ) however, the point here is that you can literally lunch any number of AWS EC2 instances you want/can pay.
Re: Network rendering support
__ Well done and great flexibility for a render engine in Alpha stage- it is now possible to rendering nodes with OpenCL support and nodes without (i.e. mix PATHOCL and PATHCPU for instance).
-
- Supporting Users
- Posts: 141
- Joined: Tue Jan 09, 2018 6:48 pm
Re: Network rendering support
@Dade
Latest pyluxcorenet is broken:
I added an default to fix that:
Another oddity: since a while i cannot get to render ocl in pyluxcorenetnode.py anymore, it looks like all is initiated fine as well as kernels compile, but
the stats are always showing no contributions happen. This is only on localhost, another machine node renders in all cases.
Same with an cpu scene works. Odd, any ideas ?
Btw: pyluxcoreconsole or luxcoreui blenderaddon all do ocl.
Jens
Latest pyluxcorenet is broken:
Code: Select all
yaddayadda
Traceback (most recent call last):
File "samples/pyluxcorenetconsole/pyluxcorenetconsole.py", line 38, in <module>
cmd.main(sys.argv)
File "./lib/pyluxcoretools.zip/pyluxcoretools/pyluxcorenetconsole/cmd.py", line 148, in main
File "./lib/pyluxcoretools.zip/pyluxcoretools/pyluxcorenetconsole/cmd.py", line 95, in Exec
self.cmdqueue = []
TypeError: 'NoneType' object is not iterable
yaddayadda
Code: Select all
parser.add_argument("-n", "--nodes", metavar="IPADDRESS", nargs="+",
default="",
help="rendering nodes ip addresses")
Another oddity: since a while i cannot get to render ocl in pyluxcorenetnode.py anymore, it looks like all is initiated fine as well as kernels compile, but
the stats are always showing no contributions happen. This is only on localhost, another machine node renders in all cases.
Code: Select all
...
[MainThread][2018-01-28 19:05:19,524] [LuxCore][1468.316] [PathOCLBaseRenderThread::2] Kernels compilation time: 8ms
[MainThread][2018-01-28 19:05:19,551] Received command: GET_STATS
[MainThread][2018-01-28 19:05:19,576] [Elapsed time: 1sec][Samples 0][Avg. samples/sec 0.00M on 0.0K tris]
[MainThread][2018-01-28 19:05:29,577] Received command: GET_STATS
[MainThread][2018-01-28 19:05:29,631] [Elapsed time: 11sec][Samples 0][Avg. samples/sec 0.00M on 0.0K tris]
...
Btw: pyluxcoreconsole or luxcoreui blenderaddon all do ocl.
Jens
Re: Network rendering support
I should have fixed both problems.
The second was probably happening because I was setting opencl.cpu.workgroup.size/opencl.gpu.workgroup.size to 0 to use the default driver value but Mac OS must have some problem with that.
The second was probably happening because I was setting opencl.cpu.workgroup.size/opencl.gpu.workgroup.size to 0 to use the default driver value but Mac OS must have some problem with that.
-
- Supporting Users
- Posts: 141
- Joined: Tue Jan 09, 2018 6:48 pm
Re: Network rendering support
I check in a minute. Btw: i am mainly on linux, here the node refusing ocl happend with a localhost setup only.
More in a few minutes.
#1 aka non iterable arg is fixed in cmd.py
Jens
More in a few minutes.
#1 aka non iterable arg is fixed in cmd.py
Jens
Last edited by jensverwiebe on Sun Jan 28, 2018 8:14 pm, edited 1 time in total.
-
- Supporting Users
- Posts: 141
- Joined: Tue Jan 09, 2018 6:48 pm
Re: Network rendering support
#2 is also working again. ( not sure i understand why your change applies for my situation on linux net + node locally )
That said, i had no probs with an osx node found atm. using only 2 gpu ocl.
Same on Linux: i disable normally the intel.icd to not use cpu for ocl additional to my 3 gpu, its ofter even slower else.
Atm. obvious probs fixed, thx Dade.
Jens
Going now to recompile osx too and check later cpu ocl ( i normally don't use it along with way faster gpu's )[MainThread][2018-01-28 21:05:01,538] [LuxCore][19.120] [PathOCLBaseRenderThread::2] Kernels compilation time: 8ms
[MainThread][2018-01-28 21:05:01,565] Received command: GET_STATS
[MainThread][2018-01-28 21:05:01,591] [Elapsed time: 1sec][Samples 0][Avg. samples/sec 0.00M on 0.0K tris]
[MainThread][2018-01-28 21:05:11,591] Received command: GET_STATS
[MainThread][2018-01-28 21:05:11,615] [Elapsed time: 11sec][Samples 330][Avg. samples/sec 62.15M on 0.0K tris]
[MainThread][2018-01-28 21:05:21,615] Received command: GET_STATS
[MainThread][2018-01-28 21:05:21,660] [Elapsed time: 21sec][Samples 657][Avg. samples/sec 64.67M on 0.0K tris]
That said, i had no probs with an osx node found atm. using only 2 gpu ocl.
Same on Linux: i disable normally the intel.icd to not use cpu for ocl additional to my 3 gpu, its ofter even slower else.
Atm. obvious probs fixed, thx Dade.
Jens
Last edited by jensverwiebe on Sun Jan 28, 2018 8:14 pm, edited 1 time in total.
Re: Network rendering support
You can use a command line option like "-D opencl.devices.select 010" to have a specific setting in every rendering node. You can use any number of custom properties and they will be used to override the settings coming from the network.