Compiling rules: when should it happen?

[marcatore]

Today I had a bit of free time for testing.

One question: compiling kernels should happen everytime I enable a new render feature? For example I create a new material type I've never created before or I had a sky texture and so on.
I'm asking this because I've launched some F12 render and sometimes I had to wait the compiling kernel time before see something but I didn't understood why.

[Dade]

Today I had a bit of free time for testing.

One question: compiling kernels should happen everytime I enable a new render feature? For example I create a new material type I've never created before or I had a sky texture and so on.
I'm asking this because I've launched some F12 render and sometimes I had to wait the compiling kernel time before see something but I didn't understood why.

Yes but it is even a bit worse: not only when you use a new feature but also when you change a parameter of some feature.

For instance , this LuxCore log line with the kernel symbol definitions:

[LuxCore][1.250] [PathOCLBaseRenderThread::0] Defined symbols: -D LUXRAYS_OPENCL_KERNEL -D SLG_OPENCL_KERNEL -D RENDER_ENGINE_PATHOCL -D PARAM_RAY_EPSILON_MIN=1.000000e-05f -D PARAM_RAY_EPSILON_MAX=1.000000e-01f -D PARAM_LIGHT_WORLD_RADIUS_SCALE=1.050000e+00f -D PARAM_ACCEL_BVH -D PARAM_FILM_RADIANCE_GROUP_0 -D PARAM_FILM_RADIANCE_GROUP_COUNT=1 -D PARAM_FILM_CHANNELS_HAS_CONVERGENCE -D PARAM_ENABLE_TEX_CONST_FLOAT -D PARAM_ENABLE_TEX_CONST_FLOAT3 -D PARAM_ENABLE_TEX_IMAGEMAP -D PARAM_ENABLE_TEX_NORMALMAP -D PARAM_ENABLE_TEX_FRESNELCOLOR -D PARAM_ENABLE_TEX_FRESNELCONST -D PARAM_ENABLE_MAT_MATTE -D PARAM_ENABLE_MAT_MIRROR -D PARAM_CAMERA_TYPE=0 -D PARAM_HAS_SKYLIGHT2 -D PARAM_HAS_ENVLIGHTS -D PARAM_MAX_PATH_DEPTH=3 -D PARAM_MAX_PATH_DEPTH_DIFFUSE=3 -D PARAM_MAX_PATH_DEPTH_GLOSSY=3 -D PARAM_MAX_PATH_DEPTH_SPECULAR=3 -D PARAM_RR_DEPTH=5 -D PARAM_RR_CAP=5.000000e-01f -D PARAM_SQRT_VARIANCE_CLAMP_MAX_VALUE=8.000000e+00f -D PARAM_IMAGE_FILTER_TYPE=5 -D PARAM_IMAGE_FILTER_WIDTH_X=2.000000e+00f -D PARAM_IMAGE_FILTER_WIDTH_Y=2.000000e+00f -D PARAM_IMAGE_FILTER_PIXEL_WIDTH_X=1 -D PARAM_IMAGE_FILTER_PIXEL_WIDTH_Y=1 -D PARAM_SAMPLER_TYPE=2 -D PARAM_SAMPLER_SOBOL_STARTOFFSET=32

There is stuff like "PARAM_ENABLE_MAT_MATTE" so a compilation will be triggered when you use a Matte material for the first time but there is also "PARAM_MAX_PATH_DEPTH=3" so any time you change the path tracing max. depth, a new kernel compilation will be required.

Any change in the above line will trigger a compilation (i.e. you have to factor all possibile combination of all possible parameters !).

Over the time I have greatly reduced the number of parameters, in particular, I'm trying to remove all parameters like PARAM_MAX_PATH_DEPTH or PARAM_RAY_EPSILON_MAX, etc. (the one that can have an infinite number of values).

[marcatore]

ok. understood.

I don't know how much I can ask but , in my PC, every recompiling needs me more than 15 minutes. So if it's a correct recompiling time, it's a bit difficult to use.

If my recompiling time is correct I hope that something will be done to reduce this.

[jensverwiebe]

ok. understood.

I don't know how much I can ask but , in my PC, every recompiling needs me more than 15 minutes. So if it's a correct recompiling time, it's a bit difficult to use.

If my recompiling time is correct I hope that something will be done to reduce this.

Aaaah damn... still those long compile times with Nvidia ?
Try to avoid procedurals atm..
There shoulld be a solution, for i have way shorter kernel compile times with other renderers ( ocl ).
Perhaps again some voodoo trick ?
@Dade: the way slower kernel compile on NVidia jumped in with cuda/ocl 8 drivers if i remeber right. ( linux >345.xx driver ).
Perhaps that hints to some change in NV api.

Jens

[B.Y.O.B.]

There's the option to have some features always enabled, this is code from LuxBlend:

Code: Select all

enabled_features = (
    'MATTE ROUGHMATTE MATTETRANSLUCENT ROUGHMATTETRANSLUCENT'
    'GLOSSY2 GLOSSYTRANSLUCENT'
    'GLASS ARCHGLASS ROUGHGLASS'
    'MIRROR METAL2'
    'HOMOGENEOUS_VOL CLEAR_VOL'
    'IMAGEMAPS_BYTE_FORMA TIMAGEMAPS_HALF_FORMA TIMAGEMAPS_1xCHANNELS IMAGEMAPS_3xCHANNELS'
    'HAS_BUMPMAPS'
)
self.properties.Set(pyluxcore.Property('opencl.code.alwaysenabled', enabled_features))

I did not have the time to add this to the new addon yet.
In LuxBlend it was only used for viewport renders, but I guess it would be good to also have it as an option for final renders (in cases where they are not so final, e.g. test renders).

[marcatore]

My VGA is a Quadro K5200 and the CPU is an intel Xeon E5 v3 @3.5Ghz with 16GB.

At the moment I have a very few basic material like matte, one glass, one metal.
Area lights with IES file, a Sun light + Sky background.

About the viewport preview I have to admit that it's quick to start. Just 1 or 2 seconds. the problem is only for the F12 render

[Dade]

I don't know how much I can ask but , in my PC, every recompiling needs me more than 15 minutes. So if it's a correct recompiling time, it's a bit difficult to use.

If my recompiling time is correct I hope that something will be done to reduce this.

As Jens wrote, this is mostly an NVIDIA problem because their compiler is insanely slow. As general workaround, just use the CPU for editing/pre-view and the GPU only for final render.

The 2 most "deadly" factors for NVIDIA compiler (and in general) are:

- procedural textures;

- recursive materials and textures (i.e. textures referencing other textures like scale, mix, etc.)

Anyway, I will continue in the effort to reduce the number of compilation.

P.S. B.Y.O.B. post is also another aspect to factor.

[Dade]

At the moment I have a very few basic material like matte, one glass, one metal.
Area lights with IES file, a Sun light + Sky background.

Mmm, 15 minutes for that kind of scene sound a bit too much even for NVIDIA compiler

[B.Y.O.B.]

About the viewport preview I have to admit that it's quick to start. Just 1 or 2 seconds. the problem is only for the F12 render

Currently the viewport is CPU-only (no OpenCL involved).
With RTPATHOCL you would moan a lot

[marcatore]

Currently the viewport is CPU-only (no OpenCL involved).
With RTPATHOCL you would moan a lot

Eheheh...There is always a good reason for everything