Page 2 of 3

Re: How to speedup instances operations?

Posted: Fri Feb 25, 2022 6:28 pm
by TAO
Hi "Dade", if this is a correct approach please give me access to the repo so I can update it and also add more features. Thank you. if I need to improve it I will be more than happy to give me a few suggestions.

Re: How to speedup instances operations?

Posted: Fri Feb 25, 2022 9:11 pm
by Dade
You can fill a "Pull Request", anyone can: https://docs.github.com/en/pull-request ... ll-request

Re: How to speedup instances operations?

Posted: Fri Feb 25, 2022 10:13 pm
by daros
Wow! That's great! This was the last issue which forced our users to reinitialize the real time session. Thanks a lot!

Re: How to speedup instances operations?

Posted: Sat Feb 26, 2022 7:47 am
by TAO
Dade wrote: Fri Feb 25, 2022 9:11 pm You can fill a "Pull Request", anyone can: https://docs.github.com/en/pull-request ... ll-request
Thank you dade, a will create a pull request, just one question. should I work on new code and test it with the master branch always? and do you have any suggestions about the code?

Re: How to speedup instances operations?

Posted: Sat Feb 26, 2022 11:04 am
by Dade
TAO wrote: Sat Feb 26, 2022 7:47 am
Dade wrote: Fri Feb 25, 2022 9:11 pm You can fill a "Pull Request", anyone can: https://docs.github.com/en/pull-request ... ll-request
Thank you dade, a will create a pull request, just one question. should I work on new code and test it with the master branch always? and do you have any suggestions about the code?
Yes, the master branch is always the "next" main version, maintenance/minor versions, if they existing, are on a separate branches.

Re: How to speedup instances operations?

Posted: Tue Mar 01, 2022 3:00 pm
by bartek_zgo
Thanks TAO for your code.
Unfortunately I'm not able to compile. I cloned repository and started from 0. Start command firt_run and than build-64-sse2 LuxCore-sdk 5
Everything was perfect. No compile errors.
So I made changes in scene.cpp and scene.h. Run again build-64-sse2 LuxCore 5. No compile errors.
So I made changes in luxcore.h. And I got following error:

Code: Select all

/mnt/nodeHDD1/LuxCore/LuxCore_latest/LinuxCompile/LuxCore-sdk/src/luxcore/luxcoreimpl.cpp: In constructor ‘luxcore::detail::RenderConfigImpl::RenderConfigImpl(const luxrays::Properties&, luxcore::detail::SceneImpl*)’:
/mnt/nodeHDD1/LuxCore/LuxCore_latest/LinuxCompile/LuxCore-sdk/src/luxcore/luxcoreimpl.cpp:1225:44: error: invalid new-expression of abstract class type ‘luxcore::detail::SceneImpl’
   scene = new SceneImpl(renderConfig->scene);
                                            ^
In file included from /mnt/nodeHDD1/LuxCore/LuxCore_latest/LinuxCompile/LuxCore-sdk/src/luxcore/luxcoreimpl.cpp:39:
/mnt/nodeHDD1/LuxCore/LuxCore_latest/LinuxCompile/LuxCore-sdk/include/luxcore/luxcoreimpl.h:145:7: note:   because the following virtual functions are pure within ‘luxcore::detail::SceneImpl’:
 class SceneImpl : public Scene {
       ^~~~~~~~~
In file included from /mnt/nodeHDD1/LuxCore/LuxCore_latest/LinuxCompile/LuxCore-sdk/src/luxcore/luxcoreimpl.cpp:38:
/mnt/nodeHDD1/LuxCore/LuxCore_latest/LinuxCompile/LuxCore-sdk/include/luxcore/luxcore.h:1094:15: note:  ‘virtual void luxcore::Scene::DeleteObjects(std::vector<std::__cxx11::basic_string<char> >&)’
  virtual void DeleteObjects(std::vector<std::string> &objNames) = 0;
               ^~~~~~~~~~~~~
/mnt/nodeHDD1/LuxCore/LuxCore_latest/LinuxCompile/LuxCore-sdk/include/luxcore/luxcore.h:1101:15: note:  ‘virtual void luxcore::Scene::DeleteObjectsInstance(const string&, unsigned int, unsigned int)’
  virtual void DeleteObjectsInstance(const std::string &prefixName, const unsigned int count, const unsigned int start) = 0;
               ^~~~~~~~~~~~~~~~~~~~~


Re: How to speedup instances operations?

Posted: Tue Mar 01, 2022 3:55 pm
by TAO
you need to add it in luxcoreimpl .
in luxcoreimpl.h around the line 206 add delete function deceleration, it should look like this:

Code: Select all

	void DeleteObject(const std::string &objName);
	void DeleteObjects(std::vector<std::string> &objNames);
	void DeleteObjectsInstance(const std::string &prefixName, const unsigned int count, const unsigned int start);
	void DeleteLight(const std::string &lightName);
	void DeleteLights(std::vector<std::string> &lightNames);
in luxcoreimpl.cpp around the line 1060 add functions as before you can add scenePropertiesCache.Clear(); in it too. it should look like this:

Code: Select all

void SceneImpl::DeleteObject(const string &objName) {
	API_BEGIN("{}", ToArgString(objName));

	// Invalidate the scene properties cache
	scenePropertiesCache.Clear();

	scene->DeleteObject(objName);

	API_END();
}

void SceneImpl::DeleteObjects(std::vector<std::string> &objNames) {

	// Invalidate the scene properties cache
	scenePropertiesCache.Clear();

	BOOST_FOREACH(const string  &objName, objNames) {
		scene->DeleteObject(objName);
	}
}

void SceneImpl::DeleteObjectsInstance(const std::string &prefixName, const unsigned int count, const unsigned int start) {

	// Invalidate the scene properties cache
	scenePropertiesCache.Clear();

	// Deleting the instances from the start amount till the counting number of instances
	for (unsigned int i = 0; i < start + count; i++)
	{
		string objectName = prefixName + std::to_string(i);
		scene->DeleteObject(objectName);
	}
}

void SceneImpl::DeleteLight(const string &lightName) {
	API_BEGIN("{}", ToArgString(lightName));

	// Invalidate the scene properties cache
	scenePropertiesCache.Clear();

	scene->DeleteLight(lightName);

	API_END();
}

void SceneImpl::DeleteLights(std::vector<std::string> &lightNames) {

	// Invalidate the scene properties cache
	scenePropertiesCache.Clear();

	// Separate the objects and send them to delete
	BOOST_FOREACH(const string  &lightName, lightNames) {
		scene->DeleteLight(lightName);
	}
}
avoid repeating the existing functions. I add the deleting multiple lights too if you do not need it to remove it or use the next message to add it to the source.
it should compile with no issue.

Re: How to speedup instances operations?

Posted: Tue Mar 01, 2022 4:34 pm
by TAO
in the last scene, i added deleting multiple lights too, so you need to remove it or added it as blew to other parts of the code.
in scene.h line 161 after DeleteLight deceleration add:

Code: Select all

	void DeleteLights(std::vector<std::string> &lightNames);
in luxcore.h line 1103 after DeleteLight deceleration add:

Code: Select all

	/*!
	 * \brief Deletes multiple lights from the scene.
	 *
	 * \param lightNames is the vector array name of the object to delete. Note: to delete
	 * area lights, use DeleteObject().
	 */
	virtual void DeleteLights(std::vector<std::string> &lightNames) = 0;
and finally in scene.cpp line 526 after DeleteLight deceleration add:

Code: Select all

void Scene::DeleteLights(std::vector<std::string> &lightNames) {
	// Separate the objects and send them to delete
	BOOST_FOREACH(const string  &lightName, lightNames) {
		DeleteLight(lightName);
	}
}

Re: How to speedup instances operations?

Posted: Wed Mar 02, 2022 9:14 am
by bartek_zgo
Hi. I have compiled the code. It is working but there is no time improvement.
I have analyzed the code and in my opinion the problem is in NamedObjectVector::DeleteObj(const std::string &name);
This code is dramatically slow. You should add DeleteObj(objectNamesArray) or something like that.
Take a look at this code. First you search index by name: GetIndex(name); (just this is not a happy solution when repeated many times).
But after this, you rebuild some index array! So it is done, each time you remove an instance. So if I remove 1000 instances, you rebuild the index array 1000 times!
First quite easy fix would be to pass a vector of instances names to remove, and rebuild indexes only once after removing all instances!
This is an easy way, that should speed up. But to do it well, it should be done in more complex way, that I'm not sure is it possible or not, because I don't know the code very well.
We want to remove some range of instances based on index. So I think GetIndex(name) could be called only once. As I suppose next instance should just have next index. I hope that if for example instance "petal_74" has index 138, "petal_75" has index 139. I hope so, but I don't know the data structure.

Re: How to speedup instances operations?

Posted: Wed Mar 02, 2022 9:53 am
by TAO
bartek_zgo wrote: Wed Mar 02, 2022 9:14 am Hi. I have compiled the code. It is working but there is no time improvement.
I have analyzed the code and in my opinion the problem is in NamedObjectVector::DeleteObj(const std::string &name);
This code is dramatically slow. You should add DeleteObj(objectNamesArray) or something like that.
Take a look at this code. First you search index by name: GetIndex(name); (just this is not a happy solution when repeated many times).
But after this, you rebuild some index array! So it is done, each time you remove an instance. So if I remove 1000 instances, you rebuild the index array 1000 times!
First quite easy fix would be to pass a vector of instances names to remove, and rebuild indexes only once after removing all instances!
This is an easy way, that should speed up. But to do it well, it should be done in more complex way, that I'm not sure is it possible or not, because I don't know the code very well.
We want to remove some range of instances based on index. So I think GetIndex(name) could be called only once. As I suppose next instance should just have next index. I hope that if for example instance "petal_74" has index 138, "petal_75" has index 139. I hope so, but I don't know the data structure.
i will check the code to see if there is any way to make the delete function faster as well.
in mean while the function corresponding to delete objects is in namedobjectvectore.cpp

Code: Select all

void NamedObjectVector::DeleteObj(const std::string &name) {
	const u_int index = GetIndex(name);
	objs.erase(objs.begin() + index);

	// I have to rebuild the indices from scratch
	name2index.clear();
	index2obj.clear();

	for (u_int i = 0; i < objs.size(); ++i) {
		NamedObject *obj = objs[i];

		name2index.insert(Name2IndexType::value_type(obj->GetName(), i));
		index2obj.insert(Index2ObjType::value_type(i, obj));	
	}
}
if you have enough time, as you mention in last post you can do what you have in mind in it.