QML application constantly increases in memory usage

axel_s

Hi there,

I have some problems to understand why my (simplified) logger application increases in memory consumption. I used heaptrack to measure allocations and to track down the source of memory consumption. Closing the application seems to free memory just before closing, so that heaptrack does not show this as a leak.

The project is quite simple and based on QML with an own class derived by QAbstractTableModel providing the messages to a TableView.
In this test setup, test messages are pushed to the model in a specific frequency. If the maximum number of messages is reached, old messages are deleted.

Furthermore, I compiled the application using Visual Studio. Here the memory profiler gave a little bit more details about the allocations (see. screenshots attached).
According to the memory consumption report, for each entry a QObjectPrivate::Connection and QtPrivate::QSlotObject is created. It seems to stay alive even after deleting the element.
I have no idea, if this is as expected and why this function doesn't free connections if rows are deleted.

Can anyone help me with this? I'd appreciate any suggestions! Thanks!

Screenshots of Memory Profiler:

In the following the most important source files (sorry for the long post, I was not allowed to upload zip files):

main.qml:

import QtQuick 2.0
import QtQuick.Controls 1.4
import QtQuick.Controls.Styles 1.4
import QtQuick.Layouts 1.2

TabView {
    id: view
    objectName: "tabView"
    anchors.fill: parent
    Layout.fillWidth: true
    Layout.fillHeight: true
    Layout.margins: 5

    tabPosition: 1
    clip: true
    style: TabViewStyle {
        tab: Item {}
    }
    Tab {
        title: "Debugger"
        Item {
            anchors.fill: parent
            anchors.margins: 20
            property string headerColour: "lightsteelblue"

            TableView {
                id: messageView
                model: logger.messageModel

                property int defaultHeight: parent.height - 10
                height:                   defaultHeight
                anchors.horizontalCenter: parent.horizontalCenter
                width:                    parent.width

                itemDelegate: Text {
                    anchors.left: !!parent ? parent.left : undefined
                    anchors.leftMargin: 4
                    text:  styleData.value
                }

                TableViewColumn{title: qsTr("ID")       ; role: "identifier"   ; width:  30}
                TableViewColumn{title: qsTr("Message")  ; role: "message"      ; width: 780} //  530
            }
        }
    }
}

Message.cpp:

#include "Message.h"

#include <QCoreApplication>

int Message::nextID = 0;
int Message::maxID  = 1e5;

Message::Message()
    : id(++Message::nextID)
    , message()
{
    if (nextID >= maxID) nextID = 0;
}

QString Message::print() const
{
    return QString()
         + message + "\n"
         + QString("---") + "\n";
}

MessageDataModel.cpp:

#include "MessageDataModel.h"

#include <QtCore/QSet>

#include <cmath>
#include <set>
#include <iostream>
#include <thread>
using std::cout;
using std::cerr;
using std::endl;
using std::vector;
using std::string;

MessageDataModel::MessageDataModel(QObject *parent)
    : QAbstractTableModel(parent)
    , displayInOrder(true)
{}

int MessageDataModel::rowCount(const QModelIndex &parent) const
{
    return messages.size();
}

int MessageDataModel::columnCount(const QModelIndex &parent) const
{
    return Message::NumAttributes;
}

QHash<int, QByteArray> MessageDataModel::roleNames() const
{
    return {
        {IDRole             , "identifier"    },
        {MessageRole        , "message"       },
    };
}

QVariant MessageDataModel::data(const QModelIndex &index, int role) const
{
    if (index.row() < 0 || messages.size() <= index.row())
        return QVariant();

    const Message& msg = message(index.row());

    switch (role) {
    case IDRole             : return QString::number(msg.id);
    case MessageRole        : return msg.message;
    }
    return QVariant();
}

//=========== end of methods required by AbstractTableView ===============

void MessageDataModel::pushRow(std::unique_ptr<Message> newMessage)
{
    // reduce model (if necessary) before inserting elements
    enforceLimit();

    // insert new messages
    int insertStart = displayInOrder ? messages.size() : 0;
    beginInsertRows(QModelIndex(), insertStart, insertStart);
    messages.push_back(std::move(newMessage));
    endInsertRows();
}

// ========= private =========
void MessageDataModel::enforceLimit()
{
    int remaining = messageLimit;
    int currentSize = messages.size();
    if (currentSize > remaining) {
        remaining = int(remaining * (1.0 - deletePercentage/100.0)); // don't delete just enough, delete a little more
        int startIndex = displayInOrder ? 0 : remaining;
        beginRemoveRows(QModelIndex(), startIndex, startIndex+currentSize-remaining-1);
        messages.erase(messages.begin(),                          // first <N> messages are deleted (the oldest messages)
                       messages.end() - remaining);               //    where N = currentSize - remaining
        endRemoveRows();
    }
}

Test Logger (Logger.cpp):

#include "Logger.h"

#include <QFile>
#include <QFileInfo>
#include <QDebug>

#include <string>


// NON-STATIC
Logger::Logger(QObject* parent)
    : QObject(parent)
    , _model()
{

    connect(&_timer, &QTimer::timeout, this, &Logger::addDebugMessages);
    _timer.start(50);
}

Logger::~Logger()
{
}

MessageDataModel* Logger::messageModel()
{
    return &_model;
}

void Logger::addDebugMessages()
{
    std::unique_ptr<Message> msg (new Message());
    msg->message   = QString("TestString").append(QString::number(msg->id));
    _model.pushRow(std::move(msg));

}

SGaist

Hi and welcome to devnet,

Your call to erase only removes the pointer, it doesn't delete what's pointed to so you first have to delete the messages objects and then call erase on the vector.

axel_s

Hi SGaist, thanks for your reply.
I am pretty sure the Message objects are deleted because the elements we delete from the messages vector is a unique_ptr. I just tested this by adding a destructor to the Message class, each message gets destroyed actually..

axel_s

After doing some further testing I noticed, that memory consumption gets stable, if I use a ListView instead of TableView. I only exchanged the TableView { ... } with this:

ListView {
        id: messageView
        model: logger.messageModel

        property int defaultHeight: parent.height - 10
        height:                   defaultHeight
        anchors.horizontalCenter: parent.horizontalCenter
        width:                    parent.width

        delegate: Text {
            anchors.left: !!parent ? parent.left : undefined
            anchors.leftMargin: 4
            text:  message
        }
    }

May this have something to do with the way I use TableView?

SGaist

There might be some things here about performance.

axel_s

Thanks, I had a look at this page already without any further insights.
Even if there is the trade-off between memory and performance, I'd never expect some kind of caching for objects which have been removed from the model. An implementation of a logger for long-term applications would lead to huge memory consumption..

SGaist

Which version of Qt are you using by the way ?

Do you have the same results when running in release mode ?

axel_s

Sorry, I forgot to mention I am using Qt 5.9.1 with same results in release mode.

SGaist

Then you may have unearthed something. Can you test that with the 5.10 branch ? Maybe the dev to see if it is still happening ?