Page 1 of 2

Opencl clenqueuwritebuffer bug

Posted: Fri May 08, 2020 12:31 pm
by Sharlybg
just a couple of time now that i get a bug while animating with 2.4 Cuda build. Each 48 frames i get an error "clenqueuwritebuffer". And render stop.
look like the gpu go out of memory. but this is no matter the scene size. as it happen now with a small size scene.

will try to reproduce it with a simpler scene for debugging.
Capture d'écran 2020-05-08 12.25.05.png

Re: Opencl clenqueuwritebuffer bug

Posted: Fri May 08, 2020 1:35 pm
by Dade
It is likely to be some kind of memory leak with CUDA (it would be useful to understand if it happens only with CUDA or also with OpenCL).

Re: Opencl clenqueuwritebuffer bug

Posted: Wed May 13, 2020 9:58 pm
by Theo_Gottwald
I also get this error, but when this happens, the computer needs a reboot.
After a reboot here it works - if the GPU Memory is large enough.
Here its like that, that even a restart of Blender will not fix the problem.
Only a restart of the computer will fix the problem.

Re: Opencl clenqueuwritebuffer bug

Posted: Wed May 13, 2020 10:18 pm
by Dade
Theo_Gottwald wrote:
Wed May 13, 2020 9:58 pm
I also get this error, but when this happens, the computer needs a reboot.
After a reboot here it works - if the GPU Memory is large enough.
Here its like that, that even a restart of Blender will not fix the problem.
Only a restart of the computer will fix the problem.
This really sounds more like a driver bug than an application bug.

Re: Opencl clenqueuwritebuffer bug

Posted: Sun May 31, 2020 5:35 am
by Theo_Gottwald
I can agree to that, because also Cycles sometimes just quits rendering.
I have 2x GTX 1080 currently. But seeing the rendertime I am working on something with RTX :-)
Just nor sure if i should take it for that price :-)
https://www.alibaba.com/product-detail/ ... 723bQTMzZ8
To night i had an animation rendering. Just at frame 37 it stopped rendering with this error.
If you want to try, download it here:


This one:
42 MB Scene.
This one is also for download.
42 MB Scene.

Re: Opencl clenqueuwritebuffer bug

Posted: Mon Jun 01, 2020 8:48 pm
by Sharlybg
I still have this error plus blender crash from time to time each 48 frames render.

Btw 150§ for a RTX 2080 TI is pretty strange. would like to see how this end up :roll:

Re: Opencl clenqueuwritebuffer bug

Posted: Tue Jun 02, 2020 9:56 am
by Martini
I don't know if it's the same bug, but I also experienced an out-of-memory error rendering an animation on my own scene. (Apologies, I did not capture the exact error message)

I tried rendering again, and Blender crashed. It seems the memory became freed at that point. I loaded Blender up and ran a few frames again with Process Explorer open, and noticed this:
memory usage climbing between frames
memory usage climbing between frames
A couple seconds before each frame starts rendering, there is a little step up in the VRAM used, and it does not go down after the rendering finishes. The increase in VRAM seems to occur right around the instant this is printed in the console:

Code: Select all

[LuxRays][1016.266] Total BVH memory usage: 1125128Kbytes
[LuxRays][1016.375] Adding DataSet accelerator: EMBREE
[LuxRays][1016.375] Total vertex count: 71893735
[LuxRays][1016.375] Total triangle count: 24151310
[LuxRays][1018.250] EmbreeAccel build time: 1875ms
[LuxRays][1018.797] [Device GeForce RTX 2080 Ti OpenCLIntersect] BVH mesh vertices buffer size: 842504Kbytes
[LuxRays][1019.859] [Device GeForce RTX 2080 Ti OpenCLIntersect] BVH nodes buffer size: 1125128Kbytes
[LuxRays][1020.266] [BVHKernel] Compiler options: -D LUXRAYS_OPENCL_KERNEL -D PARAM_RAY_EPSILON_MIN=1e-05f -D PARAM_RAY_EPSILON_MAX=0.1f -D LUXRAYS_OPENCL_DEVICE -D LUXRAYS_OS_WINDOWS -cl-fast-relaxed-math -cl-mad-enable
[LuxRays][1020.266] [BVHKernel] Compiling kernels
[LuxRays][1020.266] [BVHKernel] Program cached
[LuxRays][1020.797] [Device GeForce RTX 2080 Ti OpenCLIntersect] BVH mesh vertices buffer size: 842504Kbytes
[LuxRays][1021.875] [Device GeForce RTX 2080 Ti OpenCLIntersect] BVH nodes buffer size: 1125128Kbytes
[LuxRays][1022.266] [BVHKernel] Compiler options: -D LUXRAYS_OPENCL_KERNEL -D PARAM_RAY_EPSILON_MIN=1e-05f -D PARAM_RAY_EPSILON_MAX=0.1f -D LUXRAYS_OPENCL_DEVICE -D LUXRAYS_OS_WINDOWS -cl-fast-relaxed-math -cl-mad-enable
[LuxRays][1022.266] [BVHKernel] Compiling kernels
[LuxRays][1022.266] [BVHKernel] Program cached
[LuxCore][1022.344] [PathOCLRenderEngine] OpenCL task count: 524288
[LuxCore][1022.344] [PathOCLBaseRenderEngine] OpenCL max. page memory size: 2883584Kbytes
Is it weird that it does the BVH and compilation twice? I do have 2 RTX 2080 Ti so I suppose that's related. But I wonder if only 1 of them is being freed after the render?

I'll try some more tests (1 GPU only... CPU only...) and see what happens...

I'm using Blender 2.83 beta with latest daily BlendLuxCore (20b0093) on Windows 10. nVIDIA GeForce Game Ready Driver 441.66. Exact hardware in my signature.

Edit: CPU-only gives no issues. Running on a single GPU gives the same problem as shown above (except that it only does the BVH and compilation once), and when the memory amount reaches the max, Blender crashes and closes instantly.

Edit 2: I get the same result in Blender 2.82a. Just before each successive frame, the memory usage seems to go up around 155mb (between approx. 154mb and 167mb; sample size: about 12 frames)

Edit 3: Increase seems relatively constant at about 155mb, even when varying the render resolution and rendering a completely different .blend file

Edit 4: Finally figured out why CUDA wasn't working. nvrtc64_101_0.dll is missing from the BlendLuxCore-latest-win64.zip! Managed to find a copy from a previous CUDA-specific build. Now CUDA is working, and the memory gets freed after a render. So maybe it's just an OpenCL specific problem right now?

Edit 5: Ha, the DLL is now back in the new daily build. :lol:

Edit 6: Decided to test it - switched from CUDA back to OpenCL and rendered a few frames. The difference is quite noticeable:
CUDA (left) vs OpenCL (right)
CUDA (left) vs OpenCL (right)

Re: Opencl clenqueuwritebuffer bug

Posted: Tue Jun 02, 2020 9:11 pm
by Sharlybg
This issue is a real pain. Yesterday night i had to wake up more than 10 Times to relaunch a 1000 frames test animation.
Any solution will be welcome.

Re: Opencl clenqueuwritebuffer bug

Posted: Wed Jun 03, 2020 2:58 am
by Martini
Sharlybg wrote:
Tue Jun 02, 2020 9:11 pm
This issue is a real pain. Yesterday i had to wake more than 10 Times to relaunch a 1000 frames test animation.
Any solution will be welcome.
Ouch. :cry:

If there is a possibility to use CUDA instead of OpenCL, I might suggest that?

Otherwise, try rendering in small batches from the CLI.

Code: Select all

\path\to\blender.exe -b "\path\to\your\scene.blend" -s 1 -e 40 -a
\path\to\blender.exe -b "\path\to\your\scene.blend" -s 41 -e 80 -a
\path\to\blender.exe -b "\path\to\your\scene.blend" -s 81 -e 120 -a
etc. for however many frames you have :lol:

You can paste those all into a batch file (e.g. render.cmd) and it should basically run unattended.

Re: Opencl clenqueuwritebuffer bug

Posted: Wed Jun 03, 2020 12:13 pm
by Sharlybg
If there is a possibility to use CUDA instead of OpenCL, I might suggest that?
Had the same problem with CUDA before so i didn't test it. But with the new unified package CUDA work just fine :D

You litterally save me thanks.