Metal support

[Ahr0n]

Just already spoke with Dade about bringing support to Metal for LuxCore. An anticipated enhancment for all Mac users which should give significant improvement in speed right now and even better results in near future as Apple declared to work more on hybrid usage of CPU and GPU on their devices.

This thread is to try making the first version of Metal implementation to Lux. We need people who just want to help bringing this implementation to life. Simply - if you have a Mac computer and you like coding and testing or you don't have a Mac but you can give some advice - you can try contributing into this development. I'd like to warmly welcome everybody who just want to try : - )

Dade - the first step is done. What's next?

Some useful links:
https://developer.apple.com/metal/sample-code/ - Metal documentation and sample code
https://developer.apple.com/videos/play/wwdc2019/611/ - Bringing OpenGL Apps to Metal

[Dade]

Dade - the first step is done. What's next?

Ok, first an introduction to the hardware abstraction layer in LuxCoreRender than a couple of questions.

LuxCoreRender internal hardware abstraction layer: HardwareDevice

LuxCore has the generic concept of "HardwareDevice" (https://github.com/LuxCoreRender/LuxCor ... ice.h#L113) and it will able to work with anything implementing this interface. It includes few basic operations like:

- allocating/freeing memory on a device;
- transferring memory from/to a device;
- compiling a kernel;
- executing a kernel.

LuxCore use an "HardwareDevice" to run the image pipeline. While it is not going to be particularly fancy to run image pipeline on Metal, it is the first step and milestone we can reach (like I did with CUDA). Once this works, 90% of work is likely to be done.

You can find some additional detail here: viewtopic.php?f=5&t=1974

LuxCoreRender internal hardware abstraction layer: HardwareIntersectionDevice

HardwareIntersectionDevice (https://github.com/LuxCoreRender/LuxCor ... ondevice.h) is an extended version of the HardwareDevice and has only one additional feature:

- tracing an array of rays in a scene.

LuxCore uses an "HardwareIntersectionDevice" to run all GPU render engines. An implementation of HardwareIntersectionDevice will allow to run all GPU rendering engines on Metal.

Some question

To have an idea of the amount of work required, I have a couple of questions:

- OpenCL C was trivial to compile with CUDA (i.e. at the end of the day is just C with a couple of additional functions). What kind of kernel language does Metal use ?

- OpenCL is able to compile kernels at runtime. CUDA is able too by using the NVRTC library (https://docs.nvidia.com/cuda/nvrtc/index.html). How does Metal work ? Does it compile GPU kernels at runtime or only statically ?

[Ahr0n]

- What kind of kernel language does Metal use ?

- According to Apple documentation Metal uses C++14 as a programming language (https://developer.apple.com/metal/Metal ... cation.pdf)

- OpenCL is able to compile kernels at runtime. CUDA is able too by using the NVRTC library (https://docs.nvidia.com/cuda/nvrtc/index.html). How does Metal work ? Does it compile GPU kernels at runtime or only statically ?

The equivalent of NVRTC library is probably MTLLibrary (https://developer.apple.com/documentati ... mtllibrary). As it says:

A MTLLibrary object contains Metal shading language source code that is compiled during the app build process or at runtime from a text string

. So it looks like it is almost identical to CUDA.

[Dade]

- According to Apple documentation Metal uses C++14 as a programming language (https://developer.apple.com/metal/Metal ... cation.pdf)

This can be handy to solve some portability issue, for instance, OpenCL C has a float3 native type, it is emulated in CUDA with a just a class: https://github.com/LuxCoreRender/LuxCor ... ce_math.cl

The same can be done in Metal.

The equivalent of NVRTC library is probably MTLLibrary (https://developer.apple.com/documentati ... mtllibrary). As it says:

A MTLLibrary object contains Metal shading language source code that is compiled during the app build process or at runtime from a text string

. So it looks like it is almost identical to CUDA.

Good again.

I don't see any major obstacle. I created a "metal_imagepipeline" branch where this work can be committed.

The very first steps are:

1) created an empty MetalHardwareDevice class like OpenCLDevice (https://github.com/LuxCoreRender/LuxCor ... cldevice.h) and CUDADevice (https://github.com/LuxCoreRender/LuxCor ... dadevice.h)

2) implement MetalDeviceDescription class (like OpenCLDeviceDescription and CUDADeviceDescription)

3) add the MetalDeviceDescription initialization to LuxRays Context (https://github.com/LuxCoreRender/LuxCor ... t.cpp#L120)

So the Metal devices will start to be listed by LuxCore as available devices.

Any question ?