Metropolis SDS problem?

Use this forum for general user support and related questions.
Forum rules
Please upload a testscene that allows developers to reproduce the problem, and attach some images.
epilectrolytics
Donor
Donor
Posts: 790
Joined: Thu Oct 04, 2018 6:06 am

Metropolis SDS problem?

Post by epilectrolytics »

Mainly a Cycles user I resorted to LuxRender BiDir whenever I needed better caustics.
Now with the Mac version of LuxCore it's very exciting to see the progress the developers have achieved, BiDir is now much faster and ready for HD animations on my MacBook!

Due to the yet not stable Mac version I have been experimenting with simple scenes like this:
glasscube.zip
(310.66 KiB) Downloaded 282 times
glasscube-.jpg
It seems that certain reflections that were rendered in Luxrender are disregarded in Luxcore: The reflections (specular) of the caustic pattern (diffuse) in the glass body that caused the pattern (specular). In my understanding these are those notorious SDS paths about which Dade said somewhere that BiDir cannot even render them by design.

Maybe it is something different but I made a comparison with the same file in LuxRender vs LuxCore and here's the result:
metroSDS.jpg
While caustic patterns are rendered much faster in LuxCore (clean after 1000 samples while in LuxRender still noisy after 5000) said reflections start as fireflies (see first image) and converge very slowly and get close to invisibility with further ray depth (in the top part of the second image the reflections on the backside cannot be found).

This is not a BiDir thing but presumably comes from an improved metropolis algorithm; in both renderers path vs bidir show the same result:
While the old metropolis catches those reflections immediately (see bottom part of the second image) the new one disregards them in path and bidir so that they only show in unguided sampling as fireflies.

Now with SPPM and BiDirVCM gone there is apparently no renderer available to render reflections on glass correctly except for very slow path+ sobol?
User avatar
Dade
Developer
Developer
Posts: 5672
Joined: Mon Dec 04, 2017 8:36 pm
Location: Italy

Re: Metropolis SDS problem?

Post by Dade »

epilectrolytics wrote: Tue Oct 09, 2018 2:02 pm In my understanding these are those notorious SDS paths about which Dade said somewhere that BiDir cannot even render them by design.
Correct, if you use a point light, you will see there is not reflected caustic at all, both in LuxCore and in old Lux.

But you seem to use a sun light source so it can be directly intersected by eye paths (i.e. there is a very bright and small/tiny/micro circle in the sky dome). The result is that you are rendering the reflected caustic, more or less, like a plain path tracer (i.e. very very hard, nearly impossible).

You can increase the sun disc relative size so it become easier to find for eye paths. You can also decrease (a lot) the Metropolis probability of a large mutations so caustics and hard paths will receive more samples.

LuxCore and old Lux BiDir are quite different and use different ways to compute path weights but the number of connection (as far as I remember) is the same so the end result should be similar.

There some little difference in Metropolis too, old Lux uses a QMC random number generator while LuxCore uses a normal pseudo-random number generator.

LuxCore and old Lux may have also different default values for many parameters too.
Support LuxCoreRender project with salts and bounties
User avatar
B.Y.O.B.
Developer
Developer
Posts: 4146
Joined: Mon Dec 04, 2017 10:08 pm
Location: Germany
Contact:

Re: Metropolis SDS problem?

Post by B.Y.O.B. »

epilectrolytics wrote: Tue Oct 09, 2018 2:02 pm Now with SPPM and BiDirVCM gone
LuxCore has a BidirVM engine, but it is unfinished/experimental, so it is not exposed in the Blender addon.
The easiest way to do a quick test with it is to replace "BIDIRCPU" with "BIDIRVMCPU" in this line:
https://github.com/LuxCoreRender/BlendL ... ig.py#L125
(assuming you use v2.1alpha4)
You might also want to set some options of this engine. See https://wiki.luxcorerender.org/LuxCore_ ... IRVMCPU.22

Then you can render with BidirVM if you select the Bidir engine in the addon.
Obviously this is just a quick hack, adding another option for the BidirVM engine would be a lot more work.

Be careful with the BidirVM engine, it is not left out without reason. It can use insane amounts of RAM very quickly and has some other quirks.
Fox
Posts: 437
Joined: Sat Mar 31, 2018 11:17 am

Re: Metropolis SDS problem?

Post by Fox »

The BiDirVMcpu sort of works, but the caustics ain't that sharp.
RGB_IMAGEPIPELINE_0.png
epilectrolytics
Donor
Donor
Posts: 790
Joined: Thu Oct 04, 2018 6:06 am

Re: Metropolis SDS problem?

Post by epilectrolytics »

Dade wrote: Tue Oct 09, 2018 2:26 pmBut you seem to use a sun light source so it can be directly intersected by eye paths
Thanks for that info resolving the mystery about SDS paths in BiDir!
So actually in BiDir not everything is rendered bidirectionally but in case of extended light sources unidirectional (eye path) rendering happens too and makes also certain SDS paths viable.

Now still the question remains why those special paths render so much worse in the new version?
I've spend two evenings now rendering this scene on my old system (LuxRender 1.6 experimental LuxCore API) and the new one (2.1 alpha4) and found out that the old version treats those unidirectional SDS paths like the bidirectional caustic paths: strong caustics resolve quickly, weak caustics slowly and adjusting the sampler affects both kinds similarly.
In the new version all bidirectional caustics resolve quickly no matter the strength or the settings which is a huge advancement.
But the unidirectional SDS paths which were formerly part of the game have now somehow gotten out of the focus of the algorithms (i suspect metropolis) and are left behind.
By setting large mutation probability to 1% I can force some effect on them but they never catch up with the rest.
Also they are somehow weaker than before, with higher ray depth there is a lot of internal reflecting going on in a glass body but those multi bounced reflections don't ever materialize.

So currently this means if the speed advantage of the new engine is to be utilised certain reflections have to be discarded by setting a clamp value that removes the fireflies they cause.

In many cases this won't be a problem, accidentally I've created a special case here with my simple setup.
B.Y.O.B. wrote: Tue Oct 09, 2018 2:40 pmLuxCore has a BidirVM engine, but it is unfinished/experimental, so it is not exposed in the Blender addon.
That's great, it was my favourite render engine!
I got it working in BlendCore with the hack you suggested but where to adjust the additional settings when they don't show up in the add-on?
Is there another config file where the default values can be changed (would need to close/reopen blender)?

The image fox posted before shows that the initial radius needs to be reduced.

Maybe there is a middle way between exposed and unexposed by having a custom properties panel in the render settings where this stuff could be activated. I'm still new here and unexperienced but maybe this is worth a request for the next alpha version feature list?
User avatar
B.Y.O.B.
Developer
Developer
Posts: 4146
Joined: Mon Dec 04, 2017 10:08 pm
Location: Germany
Contact:

Re: Metropolis SDS problem?

Post by B.Y.O.B. »

epilectrolytics wrote: Wed Oct 10, 2018 8:20 am Is there another config file where the default values can be changed (would need to close/reopen blender)?

The image fox posted before shows that the initial radius needs to be reduced.
You can just add more lines to the definitions dictionary:

Code: Select all

luxcore_engine = "BIDIRVMCPU"
definitions["light.maxdepth"] = config.bidir_light_maxdepth
definitions["path.maxdepth"] = config.bidir_path_maxdepth

# new stuff
definitions["bidirvm.startradius.scale"] = 0.03
# etc.
Drawback: Every time you want to change a value, you will have to restart Blender.

If you want to have the option exposed in the UI, you have to do the following: See the Blender API manual on how to do that exactly, or take a look around in the files to see how the other properties are created.
E.g. for the startradius you want a FloatProperty(name="startradius", default=0.03).
Fox
Posts: 437
Joined: Sat Mar 31, 2018 11:17 am

Re: Metropolis SDS problem?

Post by Fox »

I used start radius 0.001 for this cube model of 2 m in size.
So, something smaller should be used.
Also the hdr lighting looks way better.
User avatar
B.Y.O.B.
Developer
Developer
Posts: 4146
Joined: Mon Dec 04, 2017 10:08 pm
Location: Germany
Contact:

Re: Metropolis SDS problem?

Post by B.Y.O.B. »

By the way, you don't have to hack BlendLuxCore to get access to BidirVM.
You can export the scene to LuxCore format and use luxcoreui.
epilectrolytics
Donor
Donor
Posts: 790
Joined: Thu Oct 04, 2018 6:06 am

status report

Post by epilectrolytics »

@B.Y.O.B.: Thanks, it works!
I couldn't get luxcore.ui to work yet on my Mac, I think Dade has fixed the issue of .cfg files not getting opened but I have to wait till @robbrown makes a new release.

Code: Select all

else:
                # config.engine == BIDIR
                luxcore_engine = "BIDIRVMCPU"
                definitions["light.maxdepth"] = config.bidir_light_maxdepth
                definitions["path.maxdepth"] = config.bidir_path_maxdepth

                # sppm settings
                definitions["bidirvm.lightpath.count"] = 131072
                definitions["bidirvm.startradius.scale"] = 0.0005
                definitions["bidirvm.alpha"] = 0.75
Now I understand why this is not official :D
Very difficult to tweak and some issues:

- BiDirVM + Metropolis doesn't get energy conservation right: The picture Fox posted clearly shows it. Over time the caustic pattern gets brighter as if the photon power is continuously aggregated instead of averaged out. With the random sampler lighting is right.
- clamp output does not work.
- when the initial radius is to big, its consecutive decreasing should converge towards pixel size, instead I have seen it getting blurrier sometimes.
- very ugly display (b/w stripe pattern) full size with random sampler.
- this is such a wonderful toy to play with :mrgreen:

Sorry but I have nothing to show yet, will post tomorrow.
User avatar
B.Y.O.B.
Developer
Developer
Posts: 4146
Joined: Mon Dec 04, 2017 10:08 pm
Location: Germany
Contact:

Re: Metropolis SDS problem?

Post by B.Y.O.B. »

epilectrolytics wrote: Wed Oct 10, 2018 3:43 pm I couldn't get luxcore.ui to work yet on my Mac, I think Dade has fixed the issue of .cfg files not getting opened but I have to wait till @robbrown makes a new release.
This bug does not happen with scenes exported from BlendLuxCore, it only happened with the scenes in the LuxCore repo.
Post Reply