[SOLVED] Asynchronous data loading for QFileSystemModel

[chabo]

I would like to display meta-data and thumbnail stored in file(own format). Because loading and parsing these data can take some time I would like to do it on separate thread. I implemented it using QtConcurrent::run and emitting dataChanged signal.

Code:

class MyModel : public QFileSystemModel
{
  Q_OBJECT
 
public:
  MyModel(QObject *parent = nullptr);
  virtual ~MyModel();
 
  QVariant data(const QModelIndex& index, int role) const override;
 
signals:
  void thumbnailLoaded(const QString &);

protected slots:
  void updateThumbnail(const QString &);

protected:
  void loadThumbnail(const QString & path, const QDateTime & time);
 
protected:
  mutable QMutex _mutex;
 
  using Thumbnails = QHash<QString, QDateTime>;
  mutable Thumbnails _thumbnails;
  mutable QPixmapCache _pixmapCache;
};
 
 MyModel::MyModel(QObject * parent)
  : QFileSystemModel(parent)
{
  // FIX
  connect(this, SIGNAL(thumbnailLoaded(const QString &)), this, SLOT(updateThumbnail(const QString &)), Qt::QueuedConnection);
}

MyModel::~MyModel()
{
  QMutexLocker lock(&_mutex);

  _thumbnails.clear();
}

QVariant MyModel ::data(const QModelIndex& index, int role) const
{
  if (role == Qt::DecorationRole)
  {
    QFileInfo info = fileInfo(index);
 
    if (info.isFile() && info.suffix() == "my")
    {
/*
      // Synchronous
      QPixmap pixmap;
 
      auto it = _thumbnails.find(info.filePath());
      if (it == _thumbnails.end() || it.value() < info.lastModified() || !_pixmapCache.find(info.filePath(), &pixmap))
      {
        if (auto screenshot = LoadScreenShotFromFile(info.filePath()))
        {
          pixmap.loadFromData(screenshot->GetData(), screenshot->GetSize());
 
          _thumbnails[info.filePath()] = info.lastModified();
          _pixmapCache.insert(info.filePath(), pixmap);
        }
      }
*/
      // Asynchronous
      QPixmap pixmap;
      bool load = false;
 
      {
        QMutexLocker lock(&_mutex);
 
        auto it = _thumbnails.find(info.filePath());
        if (it != _thumbnails.end())
        {
          if (it.value().isValid())
          {
            if (it.value() == info.lastModified())
            {
              load = !_pixmapCache.find(info.filePath(), &pixmap);
            }
            else
            {
              load = true;
            }
          }
        }
        else
        {
          _thumbnails[info.filePath()] = QDateTime();
 
          load = true;
        }
      }
     
      if (load)
      {
        QtConcurrent::run(const_cast<MyModel *>(this), &MyModel::loadThumbnail, info.filePath(), info.lastModified());
      }
 
      return pixmap;
    }
  }
 
  return QFileSystemModel::data(index, role);
}
 
 
void MyModel::loadThumbnail(const QString & path, const QDateTime & time)
{
  QPixmap pixmap;
  bool loaded = false;
 
  if (auto screenshot = LoadScreenShotFromFile(info.filePath()))
  {
      loaded = pixmap.loadFromData(screenshot->GetData(), screenshot->GetSize());
  }

  bool changed = false;
  {
    QMutexLocker lock(&_mutex);

    auto it = _thumbnails.find(path);
    if (it != _thumbnails.end() && it.value() <= time)
    {
      if (loaded)
      {
        *it = time;
        _pixmapCache.insert(path, pixmap);
      }
      else
      {
        _thumbnails.erase(it);
        _pixmapCache.remove(path);
      }

      changed = true;
    }
  }

  if (changed)
  {
    // FIX
    emit thumbnailLoaded(path);
  }
}

void MyModel::updateThumbnail(const QString & path)
{
  QModelIndex i = index(path);
  
  if (i.isValid())
  {
    emit dataChanged(i, i, QVector<int>{QFileSystemModel::FileIconRole});
  }
}

The problem is that sometime view displaying this model is corrupted, only small icons without file names or only file names are displayed.
Code should be thread safe. Signal from the thread pool is run in GUI thread. Only problem I see is const_cast.

Am I missing something? Thanks for any help.

SOLVED

[SGaist]

Hi and welcome to devnet,

QFileSystemModel is already thread so you might be creating interference currently. You should maybe start creating your thumbnails only after directoryLoaded has been emitting so you are sure that all the information of the folder you are accessing is already available to the model.

Hope it helps

[chabo]

@SGaist

Hi, thanks for answer, but I already tried it and it didn't help.

[SGaist]

Rather than using a mutex, why not return a "busy" image while you create the thumbnail ? Once it's created emitting data changed will trigger the update and you can return the thumbnail

[chabo]

@SGaist
Could you point me to some documentation about "busy" image please. I am new to Qt.
I thought returning empty QPixmap while thumbnail is loading is enough, but maybe this is causing the problem.
After thumbnail is loaded I am emitting dataChanged.

[SGaist]

By "busy image" I meant a non null pixmap containing e.g. a clock or whatever you see fit to make your user understand that some processing is occurring

[chabo]

Ok, I've tried to return some preloaded image while thumbnail is loading. After real thumbnail was loaded and dataChanged was emitted, element size in QListView keeps the same size as preloaded image. I've also tried to return QColor which is also acceptable as result by DecorationRole in data() function. In this case it looks same as when I was returning empty QPixmap. Elements was small.
So I think problem is somewhere else. It looks like QListView doesn't react properly when element size is changed by DecorationRole in model data() function. Or I am missing some function call.
One more observation, this happens only when widget with QListView is hidden and then it shows up. I have QListView on QTabWidget. Same implementation for view and model, but problem occurs on tab which is hidden and then showed up.
Window resize fixes the problem when resizeMode is equal QListView::Adjust

[SGaist]

Can you share the setup of your QListView ?

[chabo]

 model = new MyModel(this);
 model->setRootPath(QDir::currentPath());
 model->setReadOnly(false);
 model->setFilter(QDir::Files | QDir::Dirs | QDir::NoDot);
 model->setNameFilters(QStringList{ "*.my" });
 model->setNameFilterDisables(false);

 ui->listView->setViewMode(QListView::IconMode);
 ui->listView->setModel(model);
 ui->listView->setRootIndex(_model->index(QDir::currentPath()));

  connect(ui->horizontalSliderIconSize, &QSlider::valueChanged, [=](int value)
  {
    ui->listView->setIconSize(QSize(value, value));
  });
  ui->horizontalSliderIconSize->setValue(128);

One more observation, there is difference depends on what you return from data() function.
If I return QIcon instead of QPixmap so code in data looks like:

.
.
if (load)
{
  QtConcurrent::run(const_cast<MyModel*>(this), &MyModel::loadThumbnail, info.filePath(), info.lastModified());
  return QColor(Qt::gray);
}
else
{
  return QIcon(pixmap);
}

the problem is not so visible, but when you slide horizontalSliderIconSize a little bit you can see that decoration of elements changed

[chabo]

Problem was emitting dataChanged signal from different thread.
After using Qt::QueuedConnection everything works as intended.

[SGaist]

Sorry, I didn't saw my answer wasn't sent.

Indeed, dataChanged will trigger GUI update which is not possible from thread other than the GUI thread.