Real-time Neural Radiance Caching for Path Tracing

General computer graphic news and papers.
Post Reply
User avatar
Sharlybg
Donor
Donor
Posts: 3101
Joined: Mon Dec 04, 2017 10:11 pm
Location: Ivory Coast

Real-time Neural Radiance Caching for Path Tracing

Post by Sharlybg »

Don't For you but this is just like black magic. We were searching for an Online learning algo capable of running on GPU with very good Perf.
And look like Researcher at Nvidia just make it happen. The good point about this is that it is fully dynamic. Meaning that you can move Camera Geometry light source without any headaches 8-) . Performance wize it is based on 3ms training wich extremelly low. It doesn't fear glossy rays.

We introduce a real-time neural radiance caching technique for path-traced global illumination. Our system is designed to handle fully dynamic scenes, and makes no assumptions about the lighting, geometry, and materials. The data-driven nature of our approach sidesteps many difficulties of caching algorithms, such as locating, interpolating, and updating cache points. Since pretraining neural networks to handle novel, dynamic scenes is a formidable generalization challenge, we do away with pretraining and instead achieve generalization via adaptation, i.e. we opt for training the radiance cache while rendering. We employ self-training—reminiscent of radiosity algorithms—to provide low-noise training targets and simulate infinite-bounce transport by merely iterating few-bounce training updates. The updates and cache queries incur a mild overhead—about 2.6ms on full HD resolution—thanks to a streaming implementation of the neural network that fully exploits modern hardware. We demonstrate significant noise reduction at the cost of little induced bias, and report state-of-the-art, real-time performance on a number of challenging scenarios.

Image

VIDEO
https://d1qx31qr3h6wln.cloudfront.net/p ... -video.mp4

PAPER
https://d1qx31qr3h6wln.cloudfront.net/p ... aper_4.pdf
Support LuxCoreRender project with salts and bounties

Portfolio : https://www.behance.net/DRAVIA
User avatar
TAO
Developer
Developer
Posts: 851
Joined: Sun Mar 24, 2019 4:49 pm
Location: France
Contact:

Re: Real-time Neural Radiance Caching for Path Tracing

Post by TAO »

Very impressive, I've seen a few other teams working on neural rendering techniques. All features of the computer will be Ai and it going so fast.
The next part is just my thought that I want to share and not reflect on the content here in any way. For the Neural techniques, in some points I like them and other times, honestly, don't. because in the new world there is no need for talent. I've seen an app that will create portraits or landscapes for you or even paint for you, considering this is just the beginning of the Neural age and very first attempt, in very soon future there will be much more advanced technique that requires almost no talent and experience at all. Good for people that don't know anything and do not want to learn and bad for people that whole life searching and working so hard.
But in the end, there is no scape of it and we should go with the flow.
User avatar
Sharlybg
Donor
Donor
Posts: 3101
Joined: Mon Dec 04, 2017 10:11 pm
Location: Ivory Coast

Re: Real-time Neural Radiance Caching for Path Tracing

Post by Sharlybg »

TAO wrote: Thu Jul 01, 2021 11:37 pm Very impressive, I've seen a few other teams working on neural rendering techniques. All features of the computer will be Ai and it going so fast.
The next part is just my thought that I want to share and not reflect on the content here in any way. For the Neural techniques, in some points I like them and other times, honestly, don't. because in the new world there is no need for talent. I've seen an app that will create portraits or landscapes for you or even paint for you, considering this is just the beginning of the Neural age and very first attempt, in very soon future there will be much more advanced technique that requires almost no talent and experience at all. Good for people that don't know anything and do not want to learn and bad for people that whole life searching and working so hard.
But in the end, there is no scape of it and we should go with the flow.
I understand your point. I still think real artist will find a way to stay relevent.For example a 1 million time faster AI based Luxcore will only make animation of any kind possible and will open door for gaming on Luxcore engine nothing else. On the painting and AI style transfert side it is more complicated but if AI can create any landscape by simple color painting then it mean that you need smaller team and time to create a movie for example and AI is not going to create 10 Oscars like movies very soon.
Sure some job will be lost but a lot of possibility will be open. And for me the AI growth is still to slow. For example OPTIX denoiser still produce wobbly output even at higher samples rate and this is after 4 Years of development. I still can't Do a movie like """The third and the seventh""" on a single GPU for a reasonnable amount of time this movie is from 2009. There is a lot of issue to be addressed before AI become a big threat to human Talent.
AI can't still not compress my 100 gigabytes scene so that i can render it on a mid range GPU. Even OOC rendering require huge amount of system memory. There still no AI to produce a production quality output of a Car based on photo. Everything AI do look like starting point for artist. There is always the polished part missing. AI is there but still i can't render mid range interior scene at 5 second / frame. Ai don't help me render caustic without cache. AI style transfert is there but still we do have to work very hard to produce photoreal CGI.
Seriously i'm not impressed by AI. I clearly see how it is growing but it will be a much longer time before it goes from Help boost to Artist Killer.
Last edited by Sharlybg on Fri Jul 02, 2021 10:33 am, edited 1 time in total.
Support LuxCoreRender project with salts and bounties

Portfolio : https://www.behance.net/DRAVIA
epilectrolytics
Donor
Donor
Posts: 790
Joined: Thu Oct 04, 2018 6:06 am

Re: Real-time Neural Radiance Caching for Path Tracing

Post by epilectrolytics »

Indeed, now that LuxCore has adopted CUDA/Optix and utilizes RTX-cores it is time to think about how to make use of all those tensor cores as well.
Metropolis sampling is not working well on GPUs, a tensor powered neural sampling would be very welcome.
Post Reply