reading flm files

Discussion related to the LuxCore functionality, implementations and API.
daros
Posts: 280
Joined: Thu Dec 12, 2019 3:25 pm
Location: inside human language
Contact:

Re: reading flm files

Post by daros »

Hi Dade, please help us with this. It's one of the very last implementations we need to complete in order to have Luxcore fully working in FELIX. Thanks!
User avatar
Dade
Developer
Developer
Posts: 5672
Joined: Mon Dec 04, 2017 8:36 pm
Location: Italy

Re: reading flm files

Post by Dade »

daros wrote: Tue Mar 23, 2021 8:06 am Hi Dade, please help us with this. It's one of the very last implementations we need to complete in order to have Luxcore fully working in FELIX. Thanks!
I just implemented film text portable format (https://github.com/LuxCoreRender/LuxCor ... 541a6e90db) ... to discover that Boost text serialization doesn't even support serialization of float INF and NaN values. This stuff is just rubbish, they are out of their mind including in a Boost official release stuff that doesn't even fully support a native C type like float.

Anyway, in my opinion, the best solution for FELIX is to do everything server side (as far as I know, this is the core philosophy of the software).

You current solution, even with a working portable file format, requires:

1) to download locally a film file (a very large file);
2) process locally the light groups and run locally the image pipeline (potentially long and memory consuming operations).

Instead you could:

1) transmit the light groups scale parameters to the server (i.e. few bytes).
2) process light groups and image pipe line on the server (the film file is already there and there are no portable issues).
3) receive a JPG file of the end result (may be even scaled down for a faster response; followed by an high resolution result when the parameter are consolidate).

This is solution is likely to be faster, more efficient in many cases: think to a laptop with a 4K rendering, it is likely to be slow or run out of memory in the first case while it will work flawless in the second case.
Support LuxCoreRender project with salts and bounties
daros
Posts: 280
Joined: Thu Dec 12, 2019 3:25 pm
Location: inside human language
Contact:

Re: reading flm files

Post by daros »

Hi Dade, we are going to use the flm file for different purposes as the luxcore user is using it normally.
The luxcore user is using the flm file to make lighting variations on a final high-res image in order to save rendering time.
We are going to use flm file to allow our user to equalize complex lighting situations in real time and get a scene with correct lighting.

imagine a scene like this one.

view92021.jpg
It contains 50 different light sources types, about 3000 light sources, 24.000.000 polygons of unique objects, 1.2 gpixel textures. No GPU can deliver you a real-time feedback on such a scene. You will always have to wait minutes for each lighting variation in order to have the feedback you need.
In FELIX, in order to allow our user to set-up the lighting of such a scene quickly what do we do? We produce a low-res-flm (from 20 to 150MB), on this low-res flm the user can play with light intensities in real-time even from a laptop. Wen the user has completed the light equalization on the flm we copy the new settings to the 3D scene replacing the orginal emitter values (we already did this part of the job for maxwell). Now the user can render the high-res image of a 100% coherent scene.
In this way we reduce the lighting set-up times from a half day to 15 minutes.
Here youy see it in action on a laptop.
https://youtu.be/cC-IBS-u-WY

So that's why we need it going in the user side.
Since we cannot do it with the FLM because the linux/windows issue we need to do it with exr.
Bartek will continue the talk with you. Thanks!
Last edited by daros on Wed Mar 24, 2021 9:05 am, edited 1 time in total.
User avatar
TAO
Developer
Developer
Posts: 850
Joined: Sun Mar 24, 2019 4:49 pm
Location: France
Contact:

Re: reading flm files

Post by TAO »

What i see in the youtube link was about the light group and can be simply done in luxCore.
Last edited by TAO on Wed Mar 24, 2021 11:59 am, edited 2 times in total.
User avatar
Dade
Developer
Developer
Posts: 5672
Joined: Mon Dec 04, 2017 8:36 pm
Location: Italy

Re: reading flm files

Post by Dade »

daros wrote: Wed Mar 24, 2021 8:42 am So that's why we need it going in the user side.
Uh, no, there is no need to be user side, it is explained in my previous post. You can run the image pipeline with a change of light group scales server-side.
Support LuxCoreRender project with salts and bounties
daros
Posts: 280
Joined: Thu Dec 12, 2019 3:25 pm
Location: inside human language
Contact:

Re: reading flm files

Post by daros »

playing with the sliders on the server would be much slower as it is currently on the user side.
bartek_zgo
Posts: 116
Joined: Mon Oct 26, 2020 11:42 am

Re: reading flm files

Post by bartek_zgo »

Dade, we want to do this on client side as it is rear time. Sending parameters to server and the result back to user will take time, while there is no need to complicate the task that we want to do.
Just forget about flm and compatibility issues. We know that we can not send it from linux to windows. We want to work with exr files. But we have one problem with exr files that I can not understand. If I tell luxcore to write one exr for each light group using pipelines it is working. I've tested and it is ok. But for some reason if I do it without pipelines using only flm file it is not working. So I have flowing workflow. I have two computer. Each one renders exactly the same scene with different seed. Than we have to merge two flm files. The merge works perfectly for main image, but if I save film->SaveOutput(fileName, OUTPUT_RADIANCE_GROUP,... the images looks not correct. So as I said before. Saving RADIANCE_GROUP using pipelines and using flm produce 2 different images. That is my problem.
User avatar
Dade
Developer
Developer
Posts: 5672
Joined: Mon Dec 04, 2017 8:36 pm
Location: Italy

Re: reading flm files

Post by Dade »

bartek_zgo wrote: Wed Mar 24, 2021 9:16 am The merge works perfectly for main image, but if I save film->SaveOutput(fileName, OUTPUT_RADIANCE_GROUP,... the images looks not correct. So as I said before. Saving RADIANCE_GROUP using pipelines and using flm produce 2 different images. That is my problem.
What tool (or API call) are you using to merge the films ? I'm testing your scenario here and it is working fine. I did 2 renderings with different seeds than I added the following lines to "python3 lib/pyluxcoretools.zip merge" (https://github.com/LuxCoreRender/LuxCor ... rge/cmd.py) before "logger.info("Done.")":

Code: Select all

	baseFilm.SaveOutput("0.exr", pyluxcore.FilmOutputType.RADIANCE_GROUP, pyluxcore.Properties().SetFromString("id = 0"))
	baseFilm.SaveOutput("1.exr", pyluxcore.FilmOutputType.RADIANCE_GROUP, pyluxcore.Properties().SetFromString("id = 1"))
to save 2 light groups. Than I have run the command:

Code: Select all

python3 lib/pyluxcoretools.zip merge film0.flm film1.flm
to merge the 2 films. The RADIANCE_GROUP outputs (i.e. "0.exr" and "1.exr") look correct.
Support LuxCoreRender project with salts and bounties
bartek_zgo
Posts: 116
Joined: Mon Oct 26, 2020 11:42 am

Re: reading flm files

Post by bartek_zgo »

I'm doing exactly the same thing. Have you seen my images in previous post?
viewtopic.php?p=28987#p28987
He is saving something but it looks strange. Are In your case images identical when you take them from pipeline and when you take them after merge?
User avatar
Dade
Developer
Developer
Posts: 5672
Joined: Mon Dec 04, 2017 8:36 pm
Location: Italy

Re: reading flm files

Post by Dade »

bartek_zgo wrote: Wed Mar 24, 2021 10:32 am I'm doing exactly the same thing. Have you seen my images in previous post?
viewtopic.php?p=28987#p28987
He is saving something but it looks strange. Are In your case images identical when you take them from pipeline and when you take them after merge?
It is actually the image you call correct to be probably wrong because it includes a lot of "INF" (infinite floating point values). All the white pixels are INF.

Anyway, I need a procedure to reproduce your problem here (a scene + standard tools like LuxCoreUI and pyluxcoretools merge, etc.) otherwise I can not see the problem.
Support LuxCoreRender project with salts and bounties
Post Reply