QML big memory consumption?
-
Hello,
Today I would like to rise a question/discussion about the huge memory consumption of my Qt quick 1.1/ QML application. Near the end of development, memory is a big concern for me. The application I made is entended to be used on embedded devices.
My original application where i see the concern exist of 15 to 20 qml pages of (800x480pixels) where i try to keep all opend pages in memory for future use. In that application I see a problem in temporary used components not freeing the memory again and the constant increase of memory while the application is running.
With the small code atached to this thread i want to corelate 3 questions to get a clear vision on how i can save memory or why things are happening. The sample app consist of a main page with a loader and a timer, after startup the main page is viewed and another page is loaded after 5 seconds, after 5 seconds the screen disapears again by unloading the page by putting "" as the source. after another 15 seconds i load an other page with the same components as the first extra page and as a result i see the memory increasing again.
- The sample app consumes 4 to 5 mb of memory only for 3 rectangles / 1 Loaders and 2 textBoxes each inside the rectangle. Is this normal behavior on all platforms? I only have experience for now on WinCe and win32.
- When the sample application is running We can see a constant increase of memory even on a destruction of a page the memory increases a little bit. What partially concerns me is the small memory increase after each page unload but also when I let that application run I see the memory increasing ( in the graph at the end you see an increase of memory without taking any action). A call to gc() in the timer does not make any difference, it sometimes makes it worse.
!http://picpaste.com/pics/memoryTesterDemo-Q9tsmT94.1390279278.png(memory tester picture)! - When a certain page is unloaded the memory is not released anymore by the program, this would be ok if the next component wich is the same would not produce more memory? Does anyone know what happens with the memory after destruction of a page, do we need to perform an extra action, function call? This is nicely seen in the graph by the 2 climbing pieces.
- when working with pictures the situation gets only worse. Each first picture use, increases the memory with one Mb when the orignal file is only one kb. does anyone has suggestions on how to handle memory on pictures except for the source size tag in the image component? Use of png, jpg, other?
the main qml file is like :
@
import QtQuick 1.1Rectangle {
width: 800
height: 480
color: "green"Loader
{
anchors.centerIn: parent
width: parent.width
height: parent.height
id: _loader
source: ""
}Timer:
{
running: true
interval: 5000
repeat: true
property int cycleCount : 0onTriggered:{
console.log("timer triggerd")
if(cycleCount == 1){
console.debug("loading page 1")
_loader.source = "testPage1.qml"
}
if(cycleCount == 3){
console.debug("unloading page 1")
_loader.source = ""
}
if(cycleCount == 6){
console.debug("loading page 2")
_loader.source = "testPage2.qml"
}
if(cycleCount == 8){
console.debug("unloading page 2")
_loader.source = ""
}
if(cycleCount == 10){
running = false
}
cycleCount +=1
}
}
}
@The testPage1.qml contains:
@
import QtQuick 1.1Rectangle {
anchors.centerIn: parent
anchors.fill: parent
color: "purple"
Text
{
anchors.centerIn: parent
font.pixelSize: 16
text: "hallo this is a test for testPage1"
font.family: "Arial"
font.bold: false
Component.onDestruction: console.log("text on page 1 will be destructed")
Component.onCompleted: console.log("text on page 1 is created")
}
Component.onCompleted: console.log("page 1 is created")
Component.onDestruction: console.log("page 1 is destructed")
}
@The testPage2.qml contains:
@
import QtQuick 1.1Rectangle {
anchors.centerIn: parent
anchors.fill: parent
color: "orange"
Text
{
anchors.centerIn: parent
font.pixelSize: 16
text: "hallo this is a test for testPage2"
font.family: "Calibri"
font.bold: false
Component.onDestruction: console.log("text on page 2 will be destructed")
Component.onCompleted: console.log("text on page 2 is created")
}
Component.onCompleted: console.log("page 2 is created")
Component.onDestruction: console.log("page 2 is destructed")
}
@The memory was watched in a thread with following content in the run
@
private:
SIZE_T m_previousPhysicalMem;#include "windows.h"
#include "psapi.h"
#pragma comment(linker, "/DEFAULTLIB:psapi.lib")static int numProcessors;
static HANDLE self;qDebug() << "virtualMemUsed" << ";" << "totalVirtualMem" << ";" << "physMemUsedByMe" << ";" << "physMemUsed" << ";" << "totalPhysMem";
while(true)
{
MEMORYSTATUSEX memInfo;
memInfo.dwLength = sizeof(MEMORYSTATUSEX);
GlobalMemoryStatusEx(&memInfo);
DWORDLONG totalVirtualMem = memInfo.ullTotalPageFile;
DWORDLONG virtualMemUsed = memInfo.ullTotalPageFile - memInfo.ullAvailPageFile;
DWORDLONG totalPhysMem = memInfo.ullTotalPhys;
DWORDLONG physMemUsed = memInfo.ullTotalPhys - memInfo.ullAvailPhys;
PROCESS_MEMORY_COUNTERS pmc;
GetProcessMemoryInfo(GetCurrentProcess(), &pmc, sizeof(pmc));
SIZE_T physMemUsedByMe = pmc.WorkingSetSize;qDebug() << virtualMemUsed << ";" << totalVirtualMem << ";" << physMemUsedByMe << ";" << physMemUsed << ";" << totalPhysMem;
msleep(1000);
}
@I really hope people can give me tips and tricks and insides into qml to save cleanup and destruct memory created by the declarative engine. Even optimalizations in the qt build arguments are worth trying.
-
Are you saying the program runs out of memory?
Notice that Qt caches components, aka QML documents. So unloading is not that big an advantage. I don't think its been suggested in any Qt documentation as a way to stop using memory. What made you do it?
And what does the OS think you are using in memory-count?
-
Hay,
bq. Are you saying the program runs out of memory?
Yes my developed application eventually runs out of memory on the embedded platforms. We have 140Mb of ram in total to run our application. If I monitor the system memory of my embedded device I see the memory increasing until 110 Mb with all possible components and screens loaded until now, this is concerning since not everything is developed yet.
bq. What made you do it? (to unload the screens)
some components we use are only used at startup level like progressbar and animated startup. So it was a hope unloading would free the used memory of that. It is interesting to know that it caches the components. Do you know if this is usual that we use that amount of memory with the qt quick components? Do you perhaps know what components allocated that much memory?
bq. And what does the OS think you are using in memory-count?
the memory count that the system indicates is relatively the same as the monitor inside the application. On the embedded device system monitor is the only thing we have but it is also the only application running in user space.
-
Today I experimented a bit with the ClearComponent cache after my startup procedure, but unfortunatly i dont see many difference in memory allocation. Some kb are freed against the many Mb. When i launch the sample qml application and follow the trace to qmalloc i see that a huge allocation takes place of the qfont engine. Do yo know how we can save memory on that part?
Here is a trace of the call with alloc over 1Mb.
QtCored4.dll!qMalloc(unsigned int size=112202) Line 57 C++
QtCored4.dll!QByteArray::resize(int size=112182) Line 1415 + 0xc bytes C++
QtGuid4.dll!QFontEngine::getSfntTable(unsigned int tag=1801810542) Line 728 C++
QtGuid4.dll!QFontEngine::loadKerningPairs(QFixed scalingFactor={...}) Line 806 + 0x11 bytes C++
QtGuid4.dll!QFontEngineWin::getCMap() Line 218 C++
QtGuid4.dll!QFontEngineWin::QFontEngineWin(const QString & name="Open Sans Extrabold", HFONT__ * _hfont=0x580a26ee, bool stockFont=false, tagLOGFONTW lf={...}) Line 344 C++
QtGuid4.dll!loadEngine(int script=0, const QFontDef & request={...}, HDC__ * fontHdc=0x00000000, int dpi=96, bool rawMode=false, const QtFontDesc * desc=0x004ce62c, const QStringList & family_list=[4]("Open Sans Extrabold","MS Shell Dlg 2","MS Sans Serif","")) Line 915 + 0x47 bytes C++
QtGuid4.dll!loadWin(const QFontPrivate * d=0x0203a348, int script=0, const QFontDef & req={...}) Line 1072 + 0x2d bytes C++
QtGuid4.dll!QFontDatabase::load(const QFontPrivate * d=0x0203a348, int script=0) Line 1119 + 0x11 bytes C++
QtGuid4.dll!QFontPrivate::engineForScript(int script=0) Line 305 + 0xd bytes C++
QtGuid4.dll!QTextEngine::fontEngine(const QScriptItem & si={...}, QFixed * ascent=0x0696ca34, QFixed * descent=0x0696ca30, QFixed * leading=0x0696ca38) Line 1936 + 0x13 bytes C++
QtGuid4.dll!QTextEngine::shapeTextWithHarfbuzz(int item=0) Line 1195 + 0x30 bytes C++
QtGuid4.dll!QTextEngine::shapeText(int item=0) Line 938 C++
QtGuid4.dll!QTextEngine::shape(int item=0) Line 1452 C++
QtGuid4.dll!QTextLine::layout_helper(int maxGlyphs=2147483647) Line 1753 C++
QtGuid4.dll!QTextLine::setNumColumns(int numColumns=2147483647) Line 1542 C++
QtGuid4.dll!QTextLayout::createLine() Line 817 C++
-
I'd definitely be interested in seeing what is causing most of the allocations. I assume that most of it is not in the font engine: although that will surely cause a large chunk to be allocated on application startup, it's unlikely to increase in an unbounded fashion.
With images, what is most likely happening is the image is being cached in the QDeclarativePixmapCache and not being released. We added API to scrub the cache in QtQuick2, not sure if it was backported to QtQuick1 or not.
In your simple rectangle + text example, the Loader element constructs a new component every time the source is reset and set again. If this component is parented to the engine, but never manually deleted by the loader during source reset, then it will stay alive forever. I'd be surprised if such a bug existed in Loader, but who knows - it's worth checking.
What does valgrind say about usage? When you exit the application normally, does valgrind report any leaks? If not, then I'd suggest that most of the memory is being used by JSC's heap (QtQuick 1.x, yeah?) - JSC heap gets torn down correctly on application exit, so valgrind won't report a leak, even though the JSC gc doesn't do much at runtime :-/
Anyway, more information is always good. With more detailed traces and analysis some solution might become apparent, but right now it's difficult to tell exactly what might be going on.
Cheers,
Chris. -
When I've reported long time ago a bug, that the simplest Qt application (only with main.cpp generated by QtCreator and one QML file with only one rectangle) produces 3000 lines of Valgrind outtput. I've received response that this is not the simplest Qt application...
Are you guys really willing to help?
Please notice that maresb has attached the source code! -
Hi kappa,
What do you want help with? What bug did you encounter?
Note that maresb's code is written for Qt Quick 1. Qt Quick 2 has a brand new engine that has much better performance than Qt Quick 1.
-
I know that when Qt Quick 2 came out, it was already far more efficient than Qt Quick 1, resource-wise. I also know that Qt 5.2 introduced further massive memory improvements to Qt Quick 2.
However, I don't know if porting is enough to resolve your issues. I also don't know if it's possible to fine-tune your app enough that it works with Qt Quick 1, because you haven't shown us your code.
(If I were to guess though, I'd say that porting to Qt 5.2.1 or later is your best bet)
-
I don't know if there are any performance comparison charts for Qt/QML apps (maybe one will create some, who knows), but I have currently two Qt Quick 2 apps and they usually use 40-60 Mb memory and not more, they are text, image and network heavy and use multiple screens and stuff, so not the simplest app every I would say.
