PhotonGI cache

Discussion related to the LuxCore functionality, implementations and API.
User avatar
Dade
Developer
Posts: 4723
Joined: Mon Dec 04, 2017 8:36 pm
Location: Italy

Re: PhotonGI cache

Post by Dade » Sun Aug 30, 2020 6:54 pm

Sharlybg wrote:
Sun Aug 30, 2020 6:16 pm
Cache computing ?
No, cache lookup.
Sharlybg wrote:
Sun Aug 30, 2020 6:16 pm
Yesterday i was reading about it ( hyperion render corona render and Mitsuba implementation).
There is a reason why they are all CPU renders.
Support LuxCoreRender project with salts and bounties

User avatar
Sharlybg
Donor
Posts: 2375
Joined: Mon Dec 04, 2017 10:11 pm
Location: Ivory Coast

Re: PhotonGI cache

Post by Sharlybg » Sun Aug 30, 2020 8:14 pm

There is a reason why they are all CPU renders.
CPU doesn't automatically mean Slow (It is not easy to beat corona render on comparable hardware even for Luxcore and Vray wich are GPU capable).
In the other side it is Clear that this method fix the cases below plus is faster in complexe render scenario :
Animation with moving object
Animation with animated light ( color / position)
Still or animation with mostly reflective surface
It is a lot of scenario cases where we miss the speed bump
Provided by PGI.
Luxcore is fast really fast a real pleasure. For the same reason it is frustrating to loose all that speed in a lot of scenario cases. And when you force to stay on speed you have to deal with flickering.
Any solution to that :?: :?

Edit

Btw if the comunication between GPU and CPU during the learning part is what make this enefficient for GPU due to overhead why this part isn't done directly on the GPU.
Isn't GPU master of machine learning algorithm ?
Support LuxCoreRender project with salts and bounties

Portfolio : https://www.behance.net/DRAVIA

User avatar
Sharlybg
Donor
Posts: 2375
Joined: Mon Dec 04, 2017 10:11 pm
Location: Ivory Coast

Re: PhotonGI cache

Post by Sharlybg » Mon Aug 31, 2020 8:51 am

DADE i think there is a solution :D

https://youtu.be/36Y-hm3QNyI (Btw the method show by the presentator seem different to vorba one he say it pre train for five minute off screen. Don't know if it change something about GPU implementation. Anyway path tracing + next even estimation seem to be the best candidate for GPUs)

In this video online learning method is compared to 3 differents other method. between them there path tracing next event exploration It produce amazing speed and can work on the GPUs as show in this second video :

https://youtu.be/bGU_JVyvzLA

https://youtu.be/I5AiBF_BkbY

I my shooting myself in eyes or there something good here :roll:

Edit : a very nice paper showing this Path tracing next event estimation ( aka shadow ray) used for volume rendering ( it is an area where we are not that good).
https://cs.dartmouth.edu/~wjarosz/publi ... slides.pdf
The idea of next‐event estimation(NEE) or shadow rays is, to reuse the same path sample to sample light transport for different bounces. This is done by drawing explicit connections from every path vertex to the light source. This technique is very popular due to the performance improvements it brings.
Next event estimation plus Cos MIS https://www.maurocomi.com/monteCarloPathTracer.html

Image
Support LuxCoreRender project with salts and bounties

Portfolio : https://www.behance.net/DRAVIA

User avatar
Sharlybg
Donor
Posts: 2375
Joined: Mon Dec 04, 2017 10:11 pm
Location: Ivory Coast

Re: PhotonGI cache

Post by Sharlybg » Mon Aug 31, 2020 12:11 pm

https://docs.blender.org/manual/en/late ... noise.html
Cycles uses path tracing with next event estimation, which is not good at rendering all types of light effects, like caustics, but has the advantage of being able to render more detailed and larger scenes compared to some other rendering algorithms. This is because we do not need to store, for example, a photon map in memory, and because we can keep rays relatively coherent to use an on-demand image cache, compared to e.g. bidirectional path tracing.
It seem a bit common as even cycles already use it. probably it is already implemented inside Luxcore and i'm loosing time searching arround it :roll:
(I hope No so that there is a big room for speed up).

It is already implemented in corona since 2013 or so : https://forum.corona-renderer.com/index.php?topic=358.0

Do we already do Next event exploration ? :?:
Support LuxCoreRender project with salts and bounties

Portfolio : https://www.behance.net/DRAVIA

User avatar
Dade
Developer
Posts: 4723
Joined: Mon Dec 04, 2017 8:36 pm
Location: Italy

Re: PhotonGI cache

Post by Dade » Mon Aug 31, 2020 12:28 pm

Sharlybg wrote:
Mon Aug 31, 2020 12:11 pm
Do we already do Next event exploration ? :?:
It is about 10 years we do.
Support LuxCoreRender project with salts and bounties

User avatar
Sharlybg
Donor
Posts: 2375
Joined: Mon Dec 04, 2017 10:11 pm
Location: Ivory Coast

Re: PhotonGI cache

Post by Sharlybg » Mon Aug 31, 2020 12:54 pm

Dade wrote:
Mon Aug 31, 2020 12:28 pm
Sharlybg wrote:
Mon Aug 31, 2020 12:11 pm
Do we already do Next event exploration ? :?:
It is about 10 years we do.
So thank you very much for stopping this hemorrhage Doctor :mrgreen: :lol:
Support LuxCoreRender project with salts and bounties

Portfolio : https://www.behance.net/DRAVIA

User avatar
Sharlybg
Donor
Posts: 2375
Joined: Mon Dec 04, 2017 10:11 pm
Location: Ivory Coast

Re: PhotonGI cache

Post by Sharlybg » Wed Sep 02, 2020 12:10 pm

I'm back again :mrgreen: . There are been some nice improvement recently in the GPU compute area and memory bandwidth.

PCIe 4.0
And more recently nvidia RTX IO.
Image

As for pratical path guiding technic the data transfert from CPU memory to GPU memory plus frequent communication was the bottleneck nullifying all sort of speed up I am wondering if theses 2 news improvement change something.
Support LuxCoreRender project with salts and bounties

Portfolio : https://www.behance.net/DRAVIA

User avatar
FarbigeWelt
Donor
Posts: 1003
Joined: Sun Jul 01, 2018 12:07 pm
Location: Switzerland
Contact:

Re: PhotonGI cache

Post by FarbigeWelt » Thu Sep 03, 2020 8:17 am

Sharlybg wrote:
Wed Sep 02, 2020 12:10 pm
I'm back again :mrgreen: . There are been some nice improvement recently in the GPU compute area and memory bandwidth.

PCIe 4.0
And more recently nvidia RTX IO.
Image

As for pratical path guiding technic the data transfert from CPU memory to GPU memory plus frequent communication was the bottleneck nullifying all sort of speed up I am wondering if theses 2 news improvement change something.
:?: :?: :?:

AMD Radeon 5700 XT
GPU Memory Memory Speed (Effective)
14 Gbps
Max Memory Size
8 GB
Memory Type (GPU)
GDDR6
Memory Interface
256-bit
Max. Memory Bandwidth
448 GB/s (<- 14 Gbit/s / 8 bit * 1 byte * 256)

On nvidia‘s graphic transfer rate from RAM to VRAM is 14 GB/s. This is 32 times less than VRAM to GPU cache.
Light and Word designing Creator - www.farbigewelt.ch - aka quantenkristall || #luxcorerender
Windows 10 Pro 64 || 2x16 Cores, AMD Ryzen 3950X @3.5GHz, 64 GB RAM, DDR4 @3.2 GHz
2x openCL, AMD Radeon RX 5700 XT, 8 GB VRAM || Gfp = SFFT Gflops

User avatar
lacilaci
Donor
Posts: 1940
Joined: Fri May 04, 2018 5:16 am

Re: PhotonGI cache

Post by lacilaci » Thu Sep 03, 2020 10:46 am

FarbigeWelt wrote:
Thu Sep 03, 2020 8:17 am
Sharlybg wrote:
Wed Sep 02, 2020 12:10 pm
I'm back again :mrgreen: . There are been some nice improvement recently in the GPU compute area and memory bandwidth.

PCIe 4.0
And more recently nvidia RTX IO.
Image

As for pratical path guiding technic the data transfert from CPU memory to GPU memory plus frequent communication was the bottleneck nullifying all sort of speed up I am wondering if theses 2 news improvement change something.
:?: :?: :?:

AMD Radeon 5700 XT
GPU Memory Memory Speed (Effective)
14 Gbps
Max Memory Size
8 GB
Memory Type (GPU)
GDDR6
Memory Interface
256-bit
Max. Memory Bandwidth
448 GB/s (<- 14 Gbit/s / 8 bit * 1 byte * 256)

On nvidia‘s graphic transfer rate from RAM to VRAM is 14 GB/s. This is 32 times less than VRAM to GPU cache.
I'm no expert, but my understanding was that this is about uncompressing textures for example straight in gpu memory so that they don't have to do that in system ram and then load to gpu vram.

All in all this looks to me very promising for hybrid rendering/game engines rather than how overall data is handled by gpu

Anyways, cpu rendering is dead, and in a year offline rendering is going to be dead

User avatar
Dade
Developer
Posts: 4723
Joined: Mon Dec 04, 2017 8:36 pm
Location: Italy

Re: PhotonGI cache

Post by Dade » Thu Sep 03, 2020 10:55 am

FarbigeWelt wrote:
Thu Sep 03, 2020 8:17 am
On nvidia‘s graphic transfer rate from RAM to VRAM is 14 GB/s. This is 32 times less than VRAM to GPU cache.
The slide is about CPU/SSD/etc. (any device attached to the PCIe bus) to GPU ram transfer rate. It can have a huge impact on some application.

In our case, it could highly increase the speed of "out of core" rendering.
Support LuxCoreRender project with salts and bounties

Post Reply