LuxCoreRender Forums

I have manually deployed the build, now it's available.

In general, when the build succeeds but the deployment fails, it is possible to (re)deploy only by accessing the release pipeline, without running the whole build again:


I'll cancel the running build.

Hi,

Attached test scene crashes on first cache update whenever Caustic Cache is enabled. I've monkeyed with parameters, tried all sampler combinations with no luck and I decided to report.

I've compiled latest (e63626411c55ff621e041e541a05a712faf4ef9b) exe from source with VS 2017 Build Tools using x64 Native Prompt but this problem persists (at least for me) since the introduction of cache update (SPPM) logic.

Thanks..

Chawl wrote: Fri Jun 21, 2019 3:08 pm Attached test scene crashes on first cache update whenever Caustic Cache is enabled.

I should have fixed the problem. You were in a quite specific case where the caustic cache is empty (it is what many "PhotonGI metropolis sampler is unable to find a useful light path" are going to mean).

Aha thanks, I think this classic test scene is very hard for a whatver sampler with an infinitesimally small point light indeed.

I forgot to add, I'm on an old AMD Phenom II X6 1075T which supports almost no modern extension sets, if it is important.

Edit: I confirm 2d3ce31 commit fixed crashing problem.

Dade wrote: Fri Jun 21, 2019 9:40 am I should have fixed this problem.

Thanks Dade and acasta69 for fixing things!
Indeed persistent indirect and periodic caustic cache now work together!

But an old bug reemerged again: When indirect and caustic cache are written for the first time, the caustic cache is not filled properly.
It can easily be tested and seen in debug mode in the concrete4 scene.
With periodic update the subsequent cache builds have the expected normal size because no indirect is build along.
But the bad first caustic one stored in persistent is always thrown in the mix at first messing up things.
. It is even worse with the Corona test file (see above).
When I render the animation, 400 samples are finished in 45s without a single cache update taking place, though I specified 7 step samples (should be more, if they would work).
Update: It is possible I had periodic update unchecked when persistent cache was written but checked later, maybe that's not allowed?

In order to make periodic cache update work it is important that each update is equally sampled, preferably with the specified step sample amount.
In my testing this works kind of in LuxCoreUI, but Blender always takes a minute or more before the first update no matter the step size, and when the render is already finished, just bad luck
No idea where this higher latency comes from.

We have here two different algorithms with their own pace of converging that need to be properly synchronized.
In CPU only that has to be done alternatingly which already works.
With OpneCL there is a huge speed disparity between GPU + cache accelerated path tracing and slow CPU based photon mapping.
I'd suggest that hybrid path tracing gets abandoned here and instead caustic cache updating is done continuously on CPU and at the same time path tracing on the GPU for the specified step number and then waiting until the next cache update is ready (or vice versa if the sampling takes longer in case of a weak GPU).

The idea to permutate a cache during sampling is pure genius and I think it has high potential not only for caustic but indirect too in order to reduce flickering and make object animation finally possible with cache acceleration!

epilectrolytics wrote: Fri Jun 21, 2019 6:47 pm It is even worse with the Corona test file (see above).
When I render the animation, 400 samples are finished in 45s without a single cache update taking place, though I specified 7 step samples (should be more, if they would work).
Update: It is possible I had periodic update unchecked when persistent cache was written but checked later, maybe that's not allowed?

Got the Corona scene now rendering with update, apparently caustic cache works with the update settings from when the persistent cache was written and not at the time of rendering.
If so, this caused a lot of confusion for me

I'm wondering what's going on during cache update:
The visibility particles are not updated?
Then the cache updates (new photon cast) are camera/view independent?
Then the cache updates could be used frame independently for all frames of a camera animation without the need to recalculate each time?
Like when rendering with persistent cache, one indirect and, say 50 small differently seeded caustic caches could be rendered and stored on HD, then fed into the rendering of a frame sequentially?
Live cache update only needed for single/still frame rendering?
No more worries with CPU/GPU sync while animating?

Great speed advantage

Update: The Corona animation (16 frames ATM) starts to show caustic flickering, looks like periodic cache is not animation proof without very high amount of updates for solid photon convergence

epilectrolytics wrote: Fri Jun 21, 2019 6:47 pm I'd suggest that hybrid path tracing gets abandoned here and instead caustic cache updating is done continuously on CPU and at the same time path tracing on the GPU for the specified step number and then waiting until the next cache update is ready (or vice versa if the sampling takes longer in case of a weak GPU).

The idea to permutate a cache during sampling is pure genius and I think it has high potential not only for caustic but indirect too in order to reduce flickering and make object animation finally possible with cache acceleration!

What exactly are the advanteges of hybrid path tracing? An average CPU is much slower than an average GPU. Is not CPU path a side product of the feasibilty study of implementing new features? Because writing software for CPU is easier.
Would it not be more performant to use CPU for tricky recursive algorithms? I mean if rendering could be divided in two parts. One part could be computed with straight forward calculations on a GPU. The other part could analyse results of first part and decide what parameters have to be improved for the first part and then feed the first part with the improvements. If rendering was a recursive approximation procedure to solve equotions this approach could speed up e.g. converging. For example, GPU casts rays in all directions depending on lamp type. A few of these beams cut camera sensor plane before limited number of bounces. Directions and positions of cast and any bounce of the successful beams get stored. The CPU creates new lamps based on successful angle and on material properties at bounce location. The new lamps have a limited number of directions, small variances of the former successful direction. Depending on total path length the angle variances could be very small and high precision floating numbers may be a topic. Another approach could be. For each lamp a list is created with possible random directions. GPU traces the path of any of these directions. CPU creates then a new list. This time random directions around a successful direction get created with small variances. Unsuccessful directions get an avoided cone of directions. Unused random directions of first pass are created and the to be avoided directions are cancelled. Principally every bounce creates a new lamp. All lamps can be treated the same way.

FarbigeWelt wrote: Sat Jun 22, 2019 9:51 am What exactly are the advanteges of hybrid path tracing?

On old hardware like my MacBook with weak graphics hybrid rendering helps to get the most out of it (I use it with Cycles).
On newer PCs with good graphics it is of less use, my PC is like ten percent faster in PathOCL GPU+CPU but sucks 33% more power (300W vs 400W)!

Therefore I never use hybrid rendering and came to similar conclusions like you:
Do the pathtracing on the most efficient GPUs and have the CPU doing all supportive tasks like denoising, cache building etc, but in parallel if memory permits.

But organising/scheduling tasks for parallel execution may be considerable effort for the developer so it's reasonable to do it only if a big enough user base screams for it loud enough
(inhaling deeply, holding breath, waiting for v2.4 feature poll...)

epilectrolytics wrote: Sat Jun 22, 2019 8:39 am Update: The Corona animation (16 frames ATM) starts to show caustic flickering, looks like periodic cache is not animation proof without very high amount of updates for solid photon convergence

And here is a short clip, first 50 frames with big caustic cache settings (10m), 350 samples = 4min/frame, scond part with smaller cache (0.5m), 500 samples = 3min/frame. Period step was 11 or 15.

Lookup radius was 1cm which is too big, blurry caustics produce a halo around the glass shadow, but sharper caustics would have needed even more updates to converge.
While the direct caustics in the glass shadow are somewhat stable the reflections in the glossy table material show very strong flickering.
Also there are occasional firefly photons, variance clamping does not prevent those.

So when you pause the video it looks somewhat ok, but for flicker free animation much more updates would be needed, maybe more than image samples which is out of question.

Judging from this first attempt, periodic caustic cache is ready for stills rendering, but not animation.
A reusable persistent cache update sequence if possible would eliminate any flickering.

List of issues remaining after the crashes have been dealt with:
- incomplete caustic cache when combined with indirect cache
- firefly photons
- artefacts of certain SDS-paths (wallpaper example)
- stripe pattern artefacts
- also see list by kintuX

Optimisation potential:
- simultaneous cache updating on CPU with path tracing on GPU for more speed and efficiency
- persistent caustic cache sequence for speeding up animations and avoiding flicker
- indirect cache updates to deal with ghost reflections in glossy surfaces and reduce flickering for object animation



Caustic cache is difficult to deal with because everything can be seen directly in the image.
There are possibly more issues to pop up the more users adopt this.
It would take too much time to deal with all this before release; my suggestion is to move on to next version (given all crash issues are dealt with) and continue PGI improving with lower priority while taking on the material system in v2.3.

Quoting another thread viewtopic.php?f=5&t=1185&start=40

Dade wrote: Mon Jul 01, 2019 12:49 pm

happyboy wrote: Mon Jul 01, 2019 12:05 pm

Dade wrote: Sun Jun 30, 2019 9:00 am Note: PhotonGI doesn't support radiance groups at the moment.

ah... Is supporting radiance groups in PhotonGI officially planed? How difficult is it? No good way to do it? Or it can be done, but there may be some penalty/restrictions? Or nothing really difficult, but needs time to implement/test?

It more a matter of memory usage: you need a separate cache for each light group (it is a bit expansive). I also wonder if it is worth doing because rendering time with the cache are really low. Light groups were development many years ago when rendering time was measured more in days than minutes.

After some further investigations we believe supporting radiance groups in PhotonGI will help us greatly, but I also understand that LuxCore 2.2 is almost feature-freezed and v2.3 material system overhaul is following. So I'm thinking about developing this feature (supporting radiance groups in PhotonGI) by my own. I've just skimmed over related codes but haven't even started build & debugging LuxCore yet. @Dade, please tell me is it suitable for a community contribution? How hard is it (diffulcty of both developing and getting merged back to LuxCore)?

My current plan is to have a separate cache for each USED radiance group (so no performance/memory drop if you only use one group), so I don't have to modify the internals of pgic codes, only need to change photon tracing thread and codes that fetch info from the cache.

Thanks!