QQmlEngine garbage collector

Rugaru

Hello everyone!
Faced an automatic garbage collection issue when using QQmlEngine.
The task is to use the built-in scripting engine to dynamically create your objects inherited from QObject with the ability to control their life by the built-in garbage collector.
I wrote a small example for the test.
The test script generates a large number of objects with large arrays of ints in a separate function and I expect that when the function exits, these objects will be removed by the garbage collector.
Or, they will be deleted after a while, because there are no links to them and when they are created, they are set to the ownership to QQmlEngine::JavaScriptOwnership.
When using the old QtScript module, the garbage collector worked fine, but when switching to QML, similar problems began.
Qt 5.15.2 compiller MSVC 2019 x64

Script handler:

H file:
class CScriptHandle : public QObject
{
	Q_OBJECT

private:
	class CScriptEngine *m_scriptEngine = nullptr;

public:
	CScriptHandle() : QObject(nullptr) {}

	QQmlEngine *m_jsEngine = nullptr;

	void RunEngine(const QString &functionName, const QString &scriptBody);

	template<class T>
	inline QJSValue AddNewScriptObject(T *obj)
	{
		obj->setParent(m_jsEngine);
		m_jsEngine->setObjectOwnership(obj, QQmlEngine::JavaScriptOwnership);
		return m_jsEngine->newQObject(obj);
	}

	inline QJSValue AddNewScriptArray()
	{
		return m_jsEngine->newArray();
	}

	inline QJSValue NullScriptObject()
	{
		return m_jsEngine->newQObject(nullptr);
	}
};



CPP file:
void CScriptHandle::RunEngine(const QString &functionName, const QString &scriptBody)
{
	m_jsEngine = new QQmlEngine();

	m_jsEngine->installExtensions(QJSEngine::GarbageCollectionExtension, m_jsEngine->globalObject());

	QJSValue metaObjectFindItemProperty = m_jsEngine->newQMetaObject(&CScriptObject::staticMetaObject);
	m_jsEngine->globalObject().setProperty("MyObj", metaObjectFindItemProperty);

	m_scriptEngine = new CScriptEngine(this);
	m_jsEngine->globalObject().setProperty("MyCore", AddNewScriptObject(m_scriptEngine));

	QJSValue result = m_jsEngine->evaluate(scriptBody);

	if (result.isError())
	{
		qDebug() << "Evaluate script error! Line:" << result.property("lineNumber").toString() << "\n" << result.toString();
	}
	else
	{
		result = m_jsEngine->globalObject().property(functionName).call();

		if (result.isError())
		{
			qDebug() << "Running script error! Line:" << result.property("lineNumber").toString() << "\n" << result.toString();
		}
	}

	m_jsEngine->collectGarbage();

	delete m_jsEngine;
	m_jsEngine = nullptr;
}

Global script class, named MyCore:

H file:
class CScriptEngine : public QObject
{
	Q_OBJECT

private:
	CScriptHandle *m_scriptHandle = nullptr;

public:
	CScriptEngine(CScriptHandle *scriptHandle);

	Q_INVOKABLE void SleepMs(int delay);

	Q_INVOKABLE void Debug(const QString &text);

	Q_INVOKABLE QJSValue GenerateObjectWithArray();
};



CPP file:
int g_scriptObjectsCount = 0;

CScriptObject::CScriptObject()
: QObject(nullptr)
{
	g_scriptObjectsCount++;
	qDebug() << "new CScriptObject, total:" << g_scriptObjectsCount;
}

CScriptObject::~CScriptObject()
{
	g_scriptObjectsCount--;
	qDebug() << "delete CScriptObject, total:" << g_scriptObjectsCount;
}

CScriptEngine::CScriptEngine(CScriptHandle *scriptHandle)
: QObject(nullptr), m_scriptHandle(scriptHandle)
{
}

void CScriptEngine::SleepMs(int delay)
{
	QApplication::processEvents();
	thread()->msleep(delay);
	QApplication::processEvents();
	m_scriptHandle->m_jsEngine->collectGarbage();
	QApplication::processEvents();
}

void CScriptEngine::Debug(const QString &text)
{
	qDebug() << text;
}

QJSValue CScriptEngine::GenerateObjectWithArray()
{
	return m_scriptHandle->AddNewScriptObject(new CScriptObject());
}

Text class with big int array (to track memory allocation/cleanup with the built-in garbage collector):

class CScriptObject : public QObject
{
	Q_OBJECT

public:
	Q_INVOKABLE CScriptObject();
	Q_INVOKABLE CScriptObject(const CScriptObject &so) :QObject(nullptr) {};
	virtual ~CScriptObject();

	int m_array[50000];
};
Q_DECLARE_METATYPE(CScriptObject);

The script runs in a separate thread:

QtConcurrent::run([this]()
{
	m_scriptHandle = new CScriptHandle();
	m_scriptHandle->RunEngine("main", ui->pte_Script->toPlainText());
	delete m_scriptHandle;
	m_scriptHandle = nullptr;
});

Script to run - main:

function main()
{
   while (true)
   {
      MyCore.Debug("test");
      CreateSOA();
      MyCore.SleepMs(1000);
   }
}

function CreateSOA()
{
   for (var i = 0; i < 10000; i++)
   {
      MyCore.GenerateObjectWithArray();
   }
}

KroMignon

@Rugaru said in QQmlEngine garbage collector:

while (true)
{
MyCore.Debug("test");
CreateSOA();
MyCore.SleepMs(1000);
}

This will look the event loop the the used thread, so no signal/slot will be handled.
Never forget that Qt is an asynchronous framework. All events are stored in the event loop and handled on next iteration.

Your forever loop will breaking this.

Rugaru

@KroMignon Thanks for your reply!
So if I add message handling to the while loop QApplication::processEvents(); - can this solve the problem?
I'll check this option in the evening.

KroMignon

@Rugaru said in QQmlEngine garbage collector:

So if I add message handling to the while loop QApplication::processEvents(); - can this solve the problem?

For me, the real clean solution would be to use a timer, so you won't have any active wait.
This will also reduce the CPU usage!

auto tmr = new QTimer(this);

connect(tmr, &QTimer::timeout, [this]() {
      MyCore.Debug("test");
      CreateSOA();
});

tmr->setInterval(1000);
tmr->setSingleShort(false);
tmr->start();

Rugaru

@KroMignon said in QQmlEngine garbage collector:

@Rugaru said in QQmlEngine garbage collector:

So if I add message handling to the while loop QApplication::processEvents(); - can this solve the problem?

For me, the real clean solution would be to use a timer, so you won't have any active wait.
This will also reduce the CPU usage!
auto tmr = new QTimer(this);

connect(tmr, &QTimer::timeout, [this]() {
      MyCore.Debug("test");
      CreateSOA();
});

tmr->setInterval(1000);
tmr->setSingleShort(false);
tmr->start();

This is about C++ code.
But my task is to make the garbage collector work correctly when using JavaScript.
In a real project, API functions are implemented that the user can call from his own scripts on JavaScript.
This test project and script shows the essence of the problem without unnecessary code sections.

KroMignon

@Rugaru said in QQmlEngine garbage collector:

This is about C++ code.
But my task is to make the garbage collector work correctly when using JavaScript.
In a real project, API functions are implemented that the user can call from his own scripts on JavaScript.
This test project and script shows the essence of the problem without unnecessary code sections.

Sorry, had misunderstood the code.

So same with JavaScript could be (not tested sorry):

var myVar = setInterval(myTimer, 1000);

function myTimer() {
      MyCore.Debug("test");
      CreateSOA();
}

But always remember, if you want to use Qt, you have to ensure you are not breaking event loop. And QML is running in main thread. So a forever loop is not possible in combination with Qt/QML.

Rugaru

@KroMignon Understood, thanks!
I will try this option after work.

Rugaru

@KroMignon said in QQmlEngine garbage collector:

@Rugaru said in QQmlEngine garbage collector:

This is about C++ code.
But my task is to make the garbage collector work correctly when using JavaScript.
In a real project, API functions are implemented that the user can call from his own scripts on JavaScript.
This test project and script shows the essence of the problem without unnecessary code sections.

Sorry, had misunderstood the code.

So same with JavaScript could be (not tested sorry):
var myVar = setInterval(myTimer, 1000);

function myTimer() {
      MyCore.Debug("test");
      CreateSOA();
}
But always remember, if you want to use Qt, you have to ensure you are not breaking event loop. And QML is running in main thread. So a forever loop is not possible in combination with Qt/QML.

The timer option did not work, because I do not use QML, but only its JavaScript interpreter, as soon as I run the main function and start the timer in it, the script ends its work.
I corrected the code a bit in the first post, added the latest changes to it.
For the test, I tried to transfer execution to the main thread, before and after SleepMs - call the processEvents, set the garbage collector flags for QQmlEngine, but this did not fix the situation.
In my application, these scripts are executed exclusively in a parallel thread, nothing from QML (except the JS interpreter) is used.
For each script, a separate thread is launched, the specified function is executed, and after execution the thread ends.
Are there any other options to make the garbage collector work without changing the JS code?

function main()
{
   while (true)
   {
      MyCore.Debug("test");
      CreateSOA();
      MyCore.SleepMs(1000);
   }
}

function CreateSOA()
{
   for (var i = 0; i < 10000; i++)
   {
      var obj = MyCore.GenerateObjectWithArray();
   }
   // all obj must be deleted here, but it doesn't happen
}

KroMignon

@Rugaru said in QQmlEngine garbage collector:

// all obj must be deleted here, but it doesn't happen

This will never happen!
Garbarge Collector is called asynchronously, when JavaScript Engine have time for it.

You can call it directly if you want with gc()

See https://doc.qt.io/qt-5/qtquick-performance.html for mode details.

Rugaru

@KroMignon said in QQmlEngine garbage collector:

You can call it directly if you want with gc()

I tried adding this to the code, but the garbage collector still wouldn't start

function CreateSOA()
{
   for (var i = 0; i < 10000; i++)
   {
      var obj = MyCore.GenerateObjectWithArray();
   }
   gc();
   // all obj must be deleted here, but it doesn't happen
}

In addition, I added to the SleepMs function forcing a call to the garbage collector from C++ code with processing the event queue, but that also gave nothing

void CScriptEngine::SleepMs(int delay)
{
	QApplication::processEvents();
	thread()->msleep(delay);
	QApplication::processEvents();
	m_scriptHandle->m_jsEngine->collectGarbage();
	QApplication::processEvents();
}

In the documentation:

While garbage collection will be automatically triggered by the JavaScript engine when the amount of available free memory is low

But this does not happen either, when the memory is full, a crash occurs

KroMignon

@Rugaru said in QQmlEngine garbage collector:

But this does not happen either, when the memory is full, a crash occurs

By the way, I don't really understand why you are using QQmlEngine for this and no QtScriptEngine?
If I understanding what you want to do, you are not using any QML feature and don't want to do it.

I guess QtScriptEngine better fits you needs (cf. https://doc.qt.io/qt-5/qtscript-index.html)

Rugaru

@KroMignon said in QQmlEngine garbage collector:

I guess QtScriptEngine better fits you needs (cf. https://doc.qt.io/qt-5/qtscript-index.html)

Right now I'm using QtScriptEngine, the garbage collector works perfectly in it, but it implements support for ECMA-262 (2011, if I'm not mistaken) and lacks most of the convenient new features of the language
I want to switch to the QML engine (because QtScriptEngine is not supported for a long time, and in Qt6 it is not at all) just for this, but it has problems with the garbage collector, which I still can't solve normally.

KroMignon

@Rugaru said in QQmlEngine garbage collector:

I want to switch to the QML engine (because QtScriptEngine is not supported for a long time, and in Qt6 it is not at all) just for this, but it has problems with the garbage collector, which I still can't solve normally.

Sorry, I can't help you more on this issue.
Perhaps you should submit a ticket at QT BugReport ==> https://bugreports.qt.io

It maybe possible that, with Qt6 there have been optimizations done on QQmlEngine for better scripting support.
I am waitting for Qt 6.2 before starting to work with Qt6, so I can not give you more information.

Rugaru

Thank you for your help!
Yes, I will write a bug report and keep looking for options.

Rugaru

Possible solution to the problem: https://bugreports.qt.io/browse/QTBUG-94566
Worked for me, but it's a workaround