PhotonGI cache

Discussion related to the Engine functionality, implementations and API.
User avatar
Dade
Developer
Developer
Posts: 3144
Joined: Mon Dec 04, 2017 8:36 pm
Location: Italy

Re: PhotonGI cache

Post by Dade » Thu Mar 28, 2019 1:40 pm

epilectrolytics wrote:
Thu Mar 28, 2019 11:44 am
In many cases interior rendering is done in only one room, would it be possible here to disable visibility/metropolis distribution and render a cache for the complete room that then would work with any possible camera position/animation inside?
Yes, if you switch from the PhotonGI METROPOLIS sampler to RANDOM but METROPOLIS+"animated camera" solution should be still more efficient, especially with HDR light coming trough windows, etc.
Support LuxCoreRender project with salts and bounties

User avatar
Dade
Developer
Developer
Posts: 3144
Joined: Mon Dec 04, 2017 8:36 pm
Location: Italy

Re: PhotonGI cache

Post by Dade » Thu Mar 28, 2019 1:45 pm

lacilaci wrote:
Thu Mar 28, 2019 1:06 pm
AFAIK vray has an "walkthrough" option, so it either recalculates cache per frame or incrementally adds new cache entries
From VRay manual (https://docs.chaosgroup.com/display/VRA ... e+Settings):
Mode – Determines the rendering mode of the light cache:

Single frame – Computes a new light cache for each frame of an animation.

Fly-through – Computes a light cache for an entire fly-through animation, assuming that the camera position/orientation is the only thing that changes.
The movement of the camera in the active time segment only is taken in consideration. Note that it may be better to set Scale to World for fly-through animations. The light cache for the entire animated sequence is computed only at the first rendered frame and is reused without changes for subsequent frames.

From file – The light cache is loaded from a file. The light cache file does not include the prefiltering of the light cache; prefiltering is performed after the light cache is loaded, so that you can adjust it without the need to recompute the light cache.

Progressive path tracing – The light cache algorithm is used to sample the final image progressively. For a discussion of this mode, see the Progressive Path Tracing With V-Ray tutorial.
They don't seem to have the incremental solution (see the statement in bold). "Fly-through" is the solution I was talking about, where you compute everything on the first frame.
Support LuxCoreRender project with salts and bounties

User avatar
B.Y.O.B.
Developer
Developer
Posts: 2962
Joined: Mon Dec 04, 2017 10:08 pm
Location: Germany
Contact:

Re: PhotonGI cache

Post by B.Y.O.B. » Thu Mar 28, 2019 1:48 pm

Dade wrote:
Thu Mar 28, 2019 1:38 pm
B.Y.O.B. wrote:
Thu Mar 28, 2019 11:24 am
Basically the same as I already do for motion blur.
Does this answer your question?
But can Blender switch camera, camera type, FOV, etc. during an animation ? All that kind of camera changes would not be covered by the motion blur trick :?:
Yes, you can do all you mentioned and more in Blender. Almost any parameter can be animated.
So yes, a list of transformation matrices is not enough.
Support LuxCoreRender project with salts and bounties

User avatar
Dade
Developer
Developer
Posts: 3144
Joined: Mon Dec 04, 2017 8:36 pm
Location: Italy

Re: PhotonGI cache

Post by Dade » Thu Mar 28, 2019 2:36 pm

B.Y.O.B. wrote:
Thu Mar 28, 2019 1:48 pm
Yes, you can do all you mentioned and more in Blender. Almost any parameter can be animated.
So yes, a list of transformation matrices is not enough.
The question is if it worth it. I mean can easily add the support for "Fly-trough" VRay-like animation mode where you move around only a single camera. Or add the support for a solution with less restrictions.

However I have also realized that the "Fly-trough" mode allow us to render the animation without exporting every single frame (!!!). You can export the scene once and tell me what delta time to render (from t0 to t1). It is sincerely a quite cool idea and a very fast way to render most ArchViz animations (export and pre-processing is done only once for all frames !).

I think we must implement the "Fly-trough" mode (and than, may be, add more general mode if needed).
Support LuxCoreRender project with salts and bounties

User avatar
B.Y.O.B.
Developer
Developer
Posts: 2962
Joined: Mon Dec 04, 2017 10:08 pm
Location: Germany
Contact:

Re: PhotonGI cache

Post by B.Y.O.B. » Thu Mar 28, 2019 3:13 pm

Maybe we should investigate how often plain fly-throughs are even used. I can imagine that in a professional production they are often too limited, in most cases e.g. for archviz you want some curtains moving in the wind, trees in the wind, falling leaves etc.
Dade wrote:
Thu Mar 28, 2019 2:36 pm
However I have also realized that the "Fly-trough" mode allow us to render the animation without exporting every single frame (!!!). You can export the scene once and tell me what delta time to render (from t0 to t1). It is sincerely a quite cool idea and a very fast way to render most ArchViz animations (export and pre-processing is done only once for all frames !).
The holy grail for me would be to have this in the Blender addon. Export the scene once, then only do updates on changed stuff.
I'm pretty sure it is possible, I can already make the RenderEngine instance persistent throughout the animation with a property.
But I will wait after I did the port to Blender 2.8 before I try my hand at it, because I think the new depsgraph will make it much easier to detect updated datablocks in a reliable way.
By the way, even Cycles does not have this feature yet - it's complicated.
Support LuxCoreRender project with salts and bounties

User avatar
epilectrolytics
Donor
Donor
Posts: 561
Joined: Thu Oct 04, 2018 6:06 am

Re: PhotonGI cache

Post by epilectrolytics » Thu Mar 28, 2019 5:39 pm

Dade wrote:
Thu Mar 28, 2019 1:40 pm
epilectrolytics wrote:
Thu Mar 28, 2019 11:44 am
would it be possible here to disable visibility/metropolis distribution
Yes, if you switch from the PhotonGI METROPOLIS sampler to RANDOM
Ok, now this is not exposed in Blender, same as the new persistent cache (yet).
I could do this in LuxCoreUI but not with animation.
So I'd like to ask for photonGI sample method to be exposed in Blender.
MBPro 15" 16GB i7-4850HQ GT750M, MacOS 10.13.6 & Win10Pro PC 16GB Ryzen 2700X, 2 x RTX 2070

User avatar
Dade
Developer
Developer
Posts: 3144
Joined: Mon Dec 04, 2017 8:36 pm
Location: Italy

Re: PhotonGI cache

Post by Dade » Thu Mar 28, 2019 8:31 pm

B.Y.O.B. wrote:
Thu Mar 28, 2019 3:13 pm
Maybe we should investigate how often plain fly-throughs are even used. I can imagine that in a professional production they are often too limited, in most cases e.g. for archviz you want some curtains moving in the wind, trees in the wind, falling leaves etc.
In that cases, PhotonGI cache will have to be built from scratch ... so, basically, they don't matter (in the context of persistent PGI cache).
Support LuxCoreRender project with salts and bounties

User avatar
Dade
Developer
Developer
Posts: 3144
Joined: Mon Dec 04, 2017 8:36 pm
Location: Italy

Re: PhotonGI cache

Post by Dade » Fri Mar 29, 2019 5:02 pm

I have added the new option to build PhotonGI cache over an arbitrary animation time length (instead of the default camera shutter open/close time):

Code: Select all

path.photongi.photon.time.start = 0.0
path.photongi.photon.time.end = 1.0
path.photongi.persistent.file = cornell.pgi
The above properties will sample the time between [0.0, 1.0] and use an persistent file cache named "cornell.pgi".

The .scn file will look like:

Code: Select all

scene.camera.lookat = -2.78 -8. 2.73 -2.78 2. 2.73
scene.camera.fieldofview = 39.1463
scene.camera.shutteropen = 0.5
scene.camera.shutterclose = 0.5
scene.camera.motion.0.time = 0.0
scene.camera.motion.0.transformation = 1.0 0.0 0.0 0.0  0.0 1.0 0.0 0.0  0.0 0.0 1.0 0.0  -6.0 0.0 0.0 1.0
scene.camera.motion.1.time = 1.0
scene.camera.motion.1.transformation = 1.0 0.0 0.0 0.0  0.0 1.0 0.0 0.0  0.0 0.0 1.0 0.0  6.0 0.0 0.0 1.0
Defining camera animation from -6 X to 6 X.

I have added a new command line option to pyluxcoretools console to be able to overwrite camera shutter open/close time from command line (i.e. "-t <shutter open> <shutter close>").

And use this script to render a first test animation:

Code: Select all

#!/usr/bin/python
# -*- coding: utf-8 -*-

import os
import sys

# To avoid .pyc files
sys.dont_write_bytecode = True

# For the develop environment and Linux workaround for https://github.com/LuxCoreRender/LuxCore/issues/80
sys.path.append(".")
sys.path.append("./lib")
sys.path.append("./lib/pyluxcoretools.zip")

import pyluxcoretools.pyluxcoreconsole.cmd as consoleCmd

if __name__ == '__main__':
	imageWidth = "512"
	imageHeight = "512"
	haltSpp = "256"

	framesCount =  5 * 30
	frameTime = 1.0 / framesCount

	for frame in range(framesCount):
		frameFileName = "frame{:03d}.png".format(frame)
		print("Rendering frame: {}".format(frameFileName))

		shutterOpen = frame * frameTime
		shutterClose = shutterOpen + frameTime

		cmdArgv = ["cornell-anum.py", "scenes/cornell/cornell-anim.cfg", \
				   "-D", "renderengine.type", "PATHOCL", \
				   "-D", "sampler.type", "SOBOL", \
				   "-w", imageWidth, \
				   "-e", imageHeight, \
				   "-D", "batch.haltspp", haltSpp, \
				   "-t", str(shutterOpen), str(shutterClose)]

		consoleCmd.main(cmdArgv)

		os.rename("denoised.png", frameFileName)
The resulting animation is: https://drive.google.com/file/d/1fFW4Uu ... sp=sharing

Each frame is rendered using the same cache (and the same scene files) making the pre-processing time 0 and the rendering time is less than 1 second given the simplicity of the scene.

P.S. I'm investigating the strange color outside left and right wall.
Support LuxCoreRender project with salts and bounties

User avatar
B.Y.O.B.
Developer
Developer
Posts: 2962
Joined: Mon Dec 04, 2017 10:08 pm
Location: Germany
Contact:

Re: PhotonGI cache

Post by B.Y.O.B. » Fri Mar 29, 2019 5:30 pm

Nice progress!
Dade wrote:
Fri Mar 29, 2019 5:02 pm
Each frame is rendered using the same cache (and the same scene files) making the pre-processing time 0
Does this mean that meshes, imagemaps etc. are already cached? Or is it just 0 because of the simplicity of the scene, after the cache building is removed?
Support LuxCoreRender project with salts and bounties

User avatar
Dade
Developer
Developer
Posts: 3144
Joined: Mon Dec 04, 2017 8:36 pm
Location: Italy

Re: PhotonGI cache

Post by Dade » Fri Mar 29, 2019 6:49 pm

B.Y.O.B. wrote:
Fri Mar 29, 2019 5:30 pm
Nice progress!
Dade wrote:
Fri Mar 29, 2019 5:02 pm
Each frame is rendered using the same cache (and the same scene files) making the pre-processing time 0
Does this mean that meshes, imagemaps etc. are already cached? Or is it just 0 because of the simplicity of the scene, after the cache building is removed?
It reuses exactly the same scene and then set the "frame" to render using camera shutter open/close (https://github.com/LuxCoreRender/LuxCor ... md.py#L143):

Code: Select all

	# Overwrite camera shutter open/close
	if (args.camera_shutter):
		cameraProps = config.GetScene().ToProperties().GetAllProperties("scene.camera")
		cameraProps.Set(pyluxcore.Property("scene.camera.shutteropen", [args.camera_shutter[0]]))
		cameraProps.Set(pyluxcore.Property("scene.camera.shutterclose", [args.camera_shutter[1]]))
		config.GetScene().Parse(cameraProps)
The original scene is in the usual .cfg/.scn format but it could be in any valid format including the single file serialized format.
Support LuxCoreRender project with salts and bounties

Post Reply