Compiling rules: when should it happen?

[jensverwiebe]

I someone has a testscene with abnormal compiletime here, it would be nice to test.
Actually i use NV opencl 1.2 cuda 9.0.282 drivers.( maxwell + pascal cpu's )
We could then judge if its old drivers or a still present problem.

Jens

[kintuX]

I someone has a testscene with abnormal compiletime here, it would be nice to test.
Actually i use NV opencl 1.2 cuda 9.0.282 drivers.( maxwell + pascal cpu's )
We could then judge if its old drivers or a still present problem.

Jens

I have a scene that fits the description...
It works OK with 2.78 & 2.79, but crashes like crazy on 2.79a RC (every Undo & Lux OCL compile takes ~20min). Even purging orphan data crashes Blender.
Still analyzing what the cause could be... taking waaay too much time Guess i should also report it to BF.

Should i organize it in orderly fashion or just send it to you?
It's a mess... driving me crazy

[jensverwiebe]

I someone has a testscene with abnormal compiletime here, it would be nice to test.
Actually i use NV opencl 1.2 cuda 9.0.282 drivers.( maxwell + pascal cpu's )
We could then judge if its old drivers or a still present problem.

Jens

I have a scene that fits the description...
It works OK with 2.78 & 2.79, but crashes like crazy on 2.79a RC (every Undo & Lux OCL compile takes ~20min). Even purging orphan data crashes Blender.
Still analyzing what the cause could be... taking waaay too much time Guess i should also report it to BF.

Should i organize it in orderly fashion or just send it to you?
It's a mess... driving me crazy

Just post it here in a streamlined manner showing that bug(s).
So all devs can give attention if they want to.
Btw: 'am a blender dev as well so can check both sides. ( on Linux )
Probably can commit a fix for 2.79a, still time for showstopper fixes.

Jens

[kintuX]

I someone has a testscene with abnormal compiletime here, it would be nice to test.
Actually i use NV opencl 1.2 cuda 9.0.282 drivers.( maxwell + pascal cpu's )
We could then judge if its old drivers or a still present problem.

Jens

I have a scene that fits the description...
It works OK with 2.78 & 2.79, but crashes like crazy on 2.79a RC (every Undo & Lux OCL compile takes ~20min). Even purging orphan data crashes Blender.
Still analyzing what the cause could be... taking waaay too much time Guess i should also report it to BF.

Should i organize it in orderly fashion or just send it to you?
It's a mess... driving me crazy

Just post it here in a streamlined manner showing that bug(s).
So all devs can give attention if they want to.
Btw: 'am a blender dev as well so can check both sides.
Probably can commit a fix for 2.79a, still time for showstopper fixes.

Jens

OK.
Tried to do compilation time comparison:
- v2.78for Blender 2.78c & LuxCoreSDK 1.7: 34693ms
- v2.79 same scene, nodes redone for 2.79a RC LuxCoreAlpha3: previously took ~20min, now gives an error:

Code: Select all

Error opening image file : C:\Users\py\AppData\Local\Temp\tmpcdc0x23o (error = OpenImageIO could not find a format reader for "C:\Users\py\AppData\Local\Temp\tmpcdc0x23o". Is it a file format that OpenImageIO doesn't know about?
)

Can't post in the open. Sent you PM - link with both versions. Textures (.DDS) packed in.

[jensverwiebe]

First sight:
Hmm, never seen someone using .dds files in blender or lux.
Must first investigate what this is even

EDIT: removed text, dds works ootb normally, but this scene is bugged somewhat.

Testing further .... i skip dds for now and convert those to a supported image format and watch the compiletimes ....
Using for now: mogrify -format png *.dds

Uhm

Code: Select all

Unpacking image "m6gt_cockpit.png" to temp file "/tmp/tmpqulul6pd"
Couldn't save picture.
WARNING: Material "Interior": Error: Image 'm6gt_cockpit.png' could not be saved to '/tmp/tmpqulul6pd'

Wtf ....

EDIT6: made simple testscene myself and tested with .dds texture == works. Problems lays elsewhere.

Code: Select all

[SDL][5.934] Reading texture map: /media/Workdata3/test_and_bugs/luxcore_tests/buggedMcL/textures/head.dds

It has something todo with blender packaging, seems the luxcore2 exporter has probs with packed images somewhat ....
This blend drives me crazy atm. Odd things happening. Perhaps it got somehow corrupted ?

I should have started from the other end with testing, sigh ...

Jens

[Dade]

Tried to do compilation time comparison:
- v2.78for Blender 2.78c & LuxCoreSDK 1.7: 34693ms
- v2.79 same scene, nodes redone for 2.79a RC LuxCoreAlpha3: previously took ~20min, now gives an error:

Is that 34secs Vs. 20mins ?

[jensverwiebe]

I went another route for now:
The 2.78 file is done with luxcore 1.7 and works fine also in blender 2.79.
Kernelcompiletimes where around 12ms per gpu (final).Preview was even faster.
I made sure kernelcaches where deleted before each run. ( verified times from log )

Getting the 2.79 file to run is a hazzle. Is it really done for luxcore2 from scratch ?
Its completely odd, all tex use .dds but it only works on the person and helmet.

Would need more time to sort this out.

But i would not expect longer kernel compiletimes for this scene on luxcore2 initially.

Will report if i got it to run.

Odd, i started over and now a simple unpack of the textures gave me correct render.
Measuring now kernelcompiletimes.
EDIT: final kernels needed around 6000ms each ( 3 gpu == 18 seconds ), so nothing to worry.

Jens

[jensverwiebe]

Final analysis:

The 2.78 car ( luxcore ) loads fine in 2.79(a) too and renders out of the box.
Its obvious that kernel compile is not abnormal long, export takes longer

The 2.79 car ( luxcore2 ) only renders when the textures get unpacked. Why it once failed
after is unclear, but in a second run from scratch it worked ootb too.
Kernel compile was also in an acceptable range ( 6 secs per gpu )

Luxcore2 compiletimes seem longer, could be other measuring just ? Dunno yet.

The problem seems luxcore exporter does not read blender packed images right now.
Thats an exporter miss atm. Not a biggie guess just development status.

Sidenote: pyluxcore.so should have been renamed pyluxcore2.so for better side by side testing
and further classic lux usage. Atm. one has the enable either one for proper loading. Not a biggie.
Classic users can remove pyluxcore.so from old luxblend for this purpose.

@kintuX: i erased your files from my hd now for your safety.
My hint would be to check if you can get newer NVidia drivers, 34 secs for compile is a bit long for your gpu.
20 minutes must have come from corrupted data, whatever this was.
To overcome the textureloading issue, just use the blender unpack function to get discrete image files for now.

Jens

[kintuX]

Tried to do compilation time comparison:
- v2.78for Blender 2.78c & LuxCoreSDK 1.7: 34693ms
- v2.79 same scene, nodes redone for 2.79a RC LuxCoreAlpha3: previously took ~20min, now gives an error:

Is that 34secs Vs. 20mins ?

Yup, that's right (2GPUs - so around the same times as jens got)

@ jensverwiebe
basically i just redid the shading/nodes for 2.79 with Luxcore2beta, since old ones don't get through
had noticed problems with packed textures before, but didn't pay any further attention, here i had textures unpacked (packed it for the sake of posting) had also similar problem with relative texture paths when they were called from python (on Windows)
+ the strangest thing is that something inside this scene bothers Blender 2.79a RC even without Lux (any action then Undo & crash)
...
will get through the full posts tomorrow, refreshed - am starting to lose vision & thought flow...

[marcatore]

I'm preparing the scene to share with you and testing before release it I've redone a render test with OCL and now the recompiling is fast ( I was not prepared to measure it cause I was expecting minutes but actually it was really fast cause after some seconds, probably less then 5, it starts).

The only thing I did it was to delete some objects I will not allowed to share and I recreate the node tree of 2 materials.
About the packed textures I'm not sure if there are.

I'll check with the other scene.