Problem loading big Tiff Files

euchkatzl

I have wired problems loading big tiff files on windows machines (32 bit).
Im working with Qt 5.6.1

Sometimes my Program crashes on executing the sql query loading the ByteArray.
Sometimes it crashes when loading the image via QImageReader.
Images are loaded in Background Threads. Foreach of this Threads a sql connection is opened because of the limitations of this connections.

On my OS X System (which is of course 64 Bit) there are no problems.
Can this be an memory allocating issue ?

Look at these two Screenshots :
https://www.dropbox.com/s/k9hqcy329uhktq9/crash.png?dl=0
https://www.dropbox.com/s/i8ccqdfgvz280tb/crash2.png?dl=0

Seems that QWebEngine has any problems !? Can anybody help me out or give my an advise what i could try.

Creating an 64 Bit Windows Application is no option because of lack of time.

mrjj

Hi
How big are those tiff?
A 32 bit process is limited to 2-3 GB
https://blogs.msdn.microsoft.com/tom/2008/04/10/chat-question-memory-limits-for-32-bit-and-64-bit-processes/

So it has to be huge TIFFs or app used lots of memory for other stuff , then
it is indeed possible to be related to memory.

Have you watched your apps mem use under windows?

euchkatzl

Memory Usage is between 700 and 800 MB in Task Manager.

mrjj

@euchkatzl
ok so it does seem like there is plenty of memory left.

Is the call stackalways the same when it crashes?

euchkatzl

Yes. Always one of these two mentioned in the Screenshots

mrjj

@euchkatzl
It does seems it crash inside Qt which is odd.

Lets see if some of the other mods have any
good ides from the call stack.

I assume testing with the tiff alone dont crash anything. only in full program?

euchkatzl

The Tiffs are about 70 MB per file.

It seems to crash in the Qt Module yes. That is why i posted this.

Loading the tiff in a standalone program don't crash.

euchkatzl

I even don't understand what is the connection between QWebEngineCore and QSqlQuery/ QImageReader.

Thats wired.

kshegunov

@euchkatzl
Hi,
How are you loading them, and into what? Are you using QPixmap? And what SQL plugin are you using?

Kind regards.

euchkatzl

The Images are loaded in a background Thread . Therefore QImage is used because QPixmaps can only be constructed in Gui Thread.

QImage ImageHelper::scaledImageFromImageData(const QByteArray &imageData, const QSize &size, const QRect &source, bool *oIsSvg, QSize *oSize) {
//    TraceLine t("scaleLoad");
    QTransform ot = orientationTransform(imageData);
    bool isSvg = isSvgImageData(imageData);
    if (oIsSvg) *oIsSvg = isSvg;

    QSize scaledSize = size;
    QRect clipRect = source;
    if (!ot.isIdentity()) {
        scaledSize = ot.mapRect(QRect(QPoint(), scaledSize)).size();
        if (!clipRect.isEmpty()) {
            clipRect = QRect();
//            QTransform t;
//            t.translate(size.width()/2.0,size.height()/2.0);
//            t *= ot;
//            t.translate(-size.width()/2.0,-size.height()/2.0);
//            clipRect = t.mapRect(clipRect);
        }
    }

    QByteArray data = imageData;
    QBuffer buff(&data);
    QImageReader reader(&buff);
    if (!reader.supportsOption(QImageIOHandler::Size)) {
        qDebug() << "ImageHelper::scaledImageFromImageData() can not determine image original size";
    }
    QSize orgSize = reader.size();
    if(oSize){
        *oSize = orgSize;
    }
    if (!clipRect.isEmpty()) {
        qreal dx = qreal(orgSize.width())/scaledSize.width();
        qreal dy = qreal(orgSize.height())/scaledSize.height();
        QRect clip = QTransform::fromScale(dx, dy).mapRect(QRectF(clipRect)).toAlignedRect();
        reader.setClipRect(clip);
        reader.setScaledSize(clipRect.size());
    } else {
        reader.setScaledSize(scaledSize);
    }
//    TraceLine *kk = new TraceLine("read");
    QImage img = reader.read();
    if(!img.isNull()){
    //    delete kk;
        if (!ot.isIdentity()) {
            img = img.transformed(ot, Qt::SmoothTransformation);
            QSize is = img.size();
            if (is != size) {
                qDebug() << "transformatino error";
            }
            if (!source.isEmpty()) {
                img = img.copy(source);
            }
        }
        QSize is = img.size();
        if (!source.isEmpty()) {
            Q_ASSERT(is == source.size());
        } else {
            Q_ASSERT(is == size);
        }
    }
    return img;
}

The Crash always occurs on the line :

QImage img = reader.read();

The ByteArray used in this function is loaded via the SqlLite Database Plugin.

I really think this is a memory allocating problem. But i don't understand why QWebEngine is in the Stack Trace

kshegunov

@euchkatzl
Well, nothing jumps out at me looking at the code.

I really think this is a memory allocating problem. But i don't understand why QWebEngine is in the Stack Trace

I'd be much more concerned why I don't see QImageReader::read() in the stack trace instead. According to the stack trace (stack trace #1) you're not in ImageHelper::scaledImageFromImageData at all, thus the above code tells us nothing for that case. I'm wary about race conditions (as you could've deduced from my questions), so can you run your code in a single thread and see if it crashes at any of the two places.

Kind regards.

euchkatzl

Thank you.
I've done this (run in a single Thread) and voila there is no crash anymore.

So i have to limit my Program loading only one tiff at a time on certain platforms (especially on Windows HighDPI because i have to load twice the size there).

We have also decided to port the Application to x64, Thank God.
There are no crashes anymore, even when loading much tiffs at a time.

Thank you all for you help. Topic is now solved for me.