QML ListView and QAbstractListModel and prefetch part of the data from 10000 records

jeremy_k

@Frenk21 said in QML ListView and QAbstractListModel and prefetch part of the data from 10000 records:

@jeremy_k

For the moment each DataLoader has a object Message that passes it to the Reader. DataLoader slot is connected to the Message signal. And when the Reader finish with reading it emits the Message.signal. And this is working fine. I am not sure if this is a good solution as I dont know if its ok that one class emit signal from the other class.

Without code, it's hard to visualize. The description involves more steps than I imagined, but I don't know the details of the backend. Working is a good sign. Unfortunately it isn't proof of correctness in the way that not working is proof of incorrectness.

I have a quick proof of concept implementation of the pattern which might be of use. I'll post it after this message.

I can also assume that when I scroll the items in the ListView, the ListView will not load all of them. It will probably try to load/create visible, but as soon they become hidden it will unload/delete them. For example if I scroll fast from 0 to the row 1000 the ListView would not load all the items.

That's a ListView delegate management detail. cacheBuffer and reuseItems influence the management, but you're still handing over control in exchange for convenience. 0 to 1000 could be interpreted as going from 0-9 to 991-1000 with nothing between, or it could be just enough time in each segment to allow the corresponding delegate to be constructed before destruction. Performance testing can help decide which is more likely and tune accordingly.

Is there any performance impact on using this approach of prefetching/caching. As now it seems ListView first creates delegates for the items, then update the values after I read them in compare to using just the Model and fetchMore()?

Using the correct range of values from the beginning is going to take less computation and power than starting with a placeholder which is updated later. That's a cost imposed by a fluid/responsive user interface. fetchMore doesn't sound like a reasonable solution given the memory constraints and model size.

jeremy_k

main.cpp:

#include <QGuiApplication>
#include <QQmlApplicationEngine>
#include <QNetworkAccessManager>
#include <QNetworkReply>
#include <QObject>
#include <QTemporaryFile>
#include <QChar>
#include <QLibraryInfo>

class ImageLoader : public QObject {
    Q_OBJECT
    Q_PROPERTY(QUrl url MEMBER m_url WRITE setUrl NOTIFY urlChanged)
    Q_PROPERTY(QUrl image READ image NOTIFY imageChanged)
    QUrl m_url;
    QUrl m_imageFile;
    static QUrl m_defaultImage;

signals:
    void urlChanged();
    void imageChanged();

private:
    void handleReply(QNetworkReply *reply)
    {
        if (reply->error() != QNetworkReply::NoError)
            return;
        QVariant contentTypeVariant = reply->header(QNetworkRequest::ContentTypeHeader);
        if (! contentTypeVariant.isValid())
            return;
        QStringList contentType = contentTypeVariant.toString().split(QChar('/'));
        if (contentType.length() != 2)
            return;
        if (contentType.at(0) != QString("image"))
            return;
        QByteArray data = reply->readAll();
        {
            QTemporaryFile file(QString("example_XXXXXX.%1").arg(contentType.at(1)));
            file.setAutoRemove(false);
            file.open();
            file.write(data);
            m_imageFile = QUrl::fromLocalFile(file.fileName());
        }
        emit imageChanged();
    }

    void setUrl(const QUrl &newUrl)
    {
        if (m_url == newUrl)
            return;
        m_url = newUrl;
        QQmlEngine *engine = qmlEngine(this);
        if (!engine)
            return;
        QNetworkAccessManager *manager = engine->networkAccessManager();
        QNetworkRequest request(m_url);
        request.setAttribute(QNetworkRequest::AutoDeleteReplyOnFinishAttribute, true);
        QNetworkReply *reply = manager->get(request);
        QObject::connect(this, &QObject::destroyed, reply, &QNetworkReply::abort);
        QObject::connect(reply, &QNetworkReply::finished, this, [reply, this](){ handleReply(reply); });
        emit urlChanged();
    }

    const QUrl &image() const { return m_imageFile; }

public:
    ImageLoader(QObject *parent = nullptr) : QObject(parent), m_imageFile(ImageLoader::m_defaultImage) { }

    ~ImageLoader() {
        if (m_imageFile != m_defaultImage)
            QFile::remove(m_imageFile.toLocalFile());
    }
};

QUrl ImageLoader::m_defaultImage =
        QUrl::fromLocalFile(QLibraryInfo::location(QLibraryInfo::Qml2ImportsPath) + "/QtQuick/Dialogs/images/question.png");

int main(int argc, char *argv[])
{
    QGuiApplication app(argc, argv);

    qmlRegisterType<ImageLoader>("Example", 1, 0, "ImageLoader");

    QQmlApplicationEngine engine;
    const QUrl url(QStringLiteral("qrc:/main.qml"));
    QObject::connect(&engine, &QQmlApplicationEngine::objectCreated, &app,
    [url] (QObject *obj, const QUrl &objUrl) {
        if (!obj && url == objUrl)
            QCoreApplication::exit(-1);
    }, Qt::QueuedConnection);
    engine.load(url);

    return app.exec();
}

#include "main.moc"

main.qml:

import QtQuick 2.15
import QtQuick.Window 2.15
import Example 1.0

Window {
    width: 640
    height: 480
    visible: true

    ListView {
        anchors.fill: parent
        model: [ /* A list of image file URLs, eg "https://doc.qt.io/style/qt-logo-documentation.svg" */ ]
        delegate: Column {
            ImageLoader {
                id: loader
                url: modelData
            }
            Text {
                text: loader.url
            }
            Image {
                width: 100
                height: 100
                fillMode: Image.PreserveAspectFit
                source: loader.image
            }
        }
    }
}

Frenk21

@jeremy_k

Hello and sorry for delay.

Thanks for the example, but as the low communication part is already implemented and does not use QNetwork... I had to add my own read queue and demultiplex the signals. The code is now working perfectly :-) I really appreciate all the support and advices.

There is one observation that I found out. Sometimes when I scroll one object with the same index is apparently created, destroyed and immediate after that created once more. Even if the object/item is always visible.

So this post can be marked as solved :-)

Will try to put some rough example, very similar to yours, but it does not use QtNetwork..

jeremy_k

@Frenk21 said in QML ListView and QAbstractListModel and prefetch part of the data from 10000 records:

There is one observation that I found out. Sometimes when I scroll one object with the same index is apparently created, destroyed and immediate after that created once more. Even if the object/item is always visible.

The view management sometimes exhibits unexpected behavior. reuseItems is particular source of surprises.

Will try to put some rough example, very similar to yours, but it does not use QtNetwork..

I wouldn't bother going into the communication details, unless it's a point of concern and deserves its own question. Good examples should be a useful illustration for a wide audience, with documented public APIs. Nobody needs Hello, World for its own sake.