Converting images to 64base is too slow.

MyNameIsQt

This program includes:

#include <QWebEngineView>
#include <QWebChannel>

By reading the saved file, getting the image binary data and converting it to base64, the image data is read through 'qtwebchannel' in 'js' and displayed on the canvas.

Multiple images are stored separately in one file, and the program reads the address value of each image and reads the data at that address. The program can read pictures in order through the ‘Next’ and ‘Previous’ buttons.
캡처4444.jpg

However, as you can see in the following function, reading the image is too slow. So I measured the time as shown in the following function.

// cpp
#include <QElapsedTimer>
QString Wemar::getImgObject()
{
    QElapsedTimer timer;

    timer.start();

    if (m_pBkgnd && m_pBkgnd->loadImageData(m_file)){
        qint64 loadTime = timer.elapsed(); // Time taken to loadImageData
        qDebug() << "Time to load image data: " << loadTime << "ms";

        timer.restart(); // reset timer

        const QImage& image = m_pBkgnd->getBkgndImg();
        qDebug() << "3 size" << image.width();

        if (image.isNull()) {
            qDebug() << "error: image is null";
            return QString(); 
        } else {
            qDebug() << "Image dimensions: " << image.width() << image.height();
        }

        qint64 imgCheckTime = timer.elapsed(); // Time taken to verify image
        qDebug() << "Time to check image: " << imgCheckTime << "ms";

        timer.restart();

        // Convert QImage to QByteArray
        QByteArray byteArray;
        QBuffer buffer(&byteArray);
        buffer.open(QIODevice::WriteOnly);

        qint64 bufferOpenTime = timer.elapsed(); // Time taken to open buffer
        qDebug() << "Time to open buffer: " << bufferOpenTime << "ms";

        timer.restart();
        image.save(&buffer, "WEBP");

        qint64 imgSaveTime = timer.elapsed(); // Time taken to save image
        qDebug() << "Time to save image as WEBP: " << imgSaveTime << "ms";

        timer.restart();

        // encoding to Base64 
        QString base64ImageData = QString::fromLatin1(byteArray.toBase64());

        qint64 base64EncodeTime = timer.elapsed(); // base64 encoded time
        
        qDebug() << "Time to encode Base64: " << base64EncodeTime << "ms";
        
        buffer.close(); /

        return base64ImageData; 
    } else {
        qDebug() << "error: m_pBkgnd is null";
        return QString(); 
    }
}

There seems to be no problem with the js part. The following is the part where data is read from js and passed to the drawCanvas function.

// js
document.addEventListener("DOMContentLoaded", function() {
       canvas = document.getElementById("myCanvas");
       gradientEffect = document.querySelector(".gradient-effect");

       if (typeof QWebChannel !== 'undefined') {
           new QWebChannel(qt.webChannelTransport, function(channel) {
               wpubManager = channel.objects.wpubManager;
               if (wpubManager) {
                   console.error("wpubManager is ready to use!");

                   var buttons = document.getElementsByClassName("ctnBtn");
                   for (var i = 0; i < buttons.length; i++) {
                       buttons[i].addEventListener("click", function() {
                           var action = this.getAttribute("data-action");
                           console.error(action);
                           wpubManager.eventClicked(action); 
                           handleImage();
                       });
                   }
               } else {
                   console.error("wpubManager is not defined. QML");
               }
           });
       } else {
           console.error("QWebChannel is not defined. Make sure qwebchannel.js is properly loaded.");
       }
   });

    async function handleImage() {
        var bkgndImageData = await wpubManager.getImgObject();
        if (bkgndImageData) {
        try {
            atob(bkgndImageData);
            console.error("Base64 is valid");
        } catch (e) {
            console.error("Base64 decode failed:", e);
        }
            console.error("image good");
            drawCanvas(bkgndImageData); // drawing image to canvas
        } else {
            console.error("Failed to get background image data.");
        }
    }

As you can see in the results, there is a clear difference in speed depending on the picture size. A slightly larger picture may take a few seconds, which is quite a long time.

**Result**:
Time to load image data:  4 ms
3 size 300
Image dimensions:  300 454
Time to check image:  0 ms
Time to open buffer:  0 ms
Time to save image as WEBP:  **137** ms
Time to encode Base64:  0 ms
js: Base64 is valid
js: image good
js: Image Data Length: 51328

Time to load image data:  26 ms
3 size 1227
Image dimensions:  1227 1600
Time to check image:  0 ms
Time to open buffer:  0 ms
Time to save image as WEBP:  **1290** ms
Time to encode Base64:  2 ms
js: Base64 is valid
js: image good
js: Image Data Length: 196080
 
Time to load image data:  3 ms
3 size 483
Image dimensions:  483 288
Time to check image:  0 ms
Time to open buffer:  0 ms
Time to save image as WEBP:  **111** ms
Time to encode Base64:  0 ms
js: Base64 is valid
js: image good
js: Image Data Length: 40472
 
Time to load image data:  42 ms
3 size 1649
Image dimensions:  1649 2048
Time to check image:  0 ms
Time to open buffer:  0 ms
Time to save image as WEBP:  **2077** ms
Time to encode Base64:  2 ms
js: Base64 is valid
js: image good
js: Image Data Length: 329128

Even at that size, 2-3 seconds is too much time for a program. Is there anything that can be improved in that function? Are there other ways to improve speed?

jsulm

@MyNameIsQt said in Converting images to 64base is too slow.:

2-3 seconds

Seconds?
You're printing milliseconds.

IgKh

Your results show that the bottleneck is not in converting from binary data to a Base-64 string, it is in the step before that - encoding the QImage pixel data as WebP data.

This isn't a big surprise - WebP is a format that is intended for websites to optimize their images ahead of time, so they are faster to decode by site visitors for any given quality. So this codec indeed requires more CPU resources to encode and compress relative to other formats, so naturally takes more time.

Since, if I understood correctly, you encode images just in time before they are displayed, and the encoded data remains in your process address space, there is no reason to use such an expensive codec. PNG or JPEG should work just as well, and you can also play with the quality argument to QImage::save to tune the compression level; the less compression the faster it is to encode, with the tradeoff that there is more data to copy around.

MyNameIsQt

@IgKh
The number of images contained in one file can be larger. So file size is important. I confirmed that when using webp, the size is reduced to about 2/3 compared to jpeg.

Thanks.

MyNameIsQt

@jsulm
When I press the program button, the actual time you feel is more than 2 to 3 seconds.

Christian Ehrlicher

Then cache the encoded images instead re-encoding them every time.
Also 2/3 sounds much but what's the actual size in bytes?

IgKh

@MyNameIsQt so if the source file is a media container of some sort, that already contains WebP encoded frames, why round trip through QImage? You are decoding and then encoding for nothing, and paying through the nose for it. Just take the byte buffer you used to construct the QImage and pass it as is to the web frame.

If the transcoding serves some purpose of doing image manipulation before display, that it was originally WebP doesn't mean it has to stay like that. The size benefit doesn't matter, it is all in RAM, and even a 100% quality PNG won't be significantly bigger than the fully expanded pixmap that is QImage.

MyNameIsQt

@Christian-Ehrlicher Now I have 100 random images in jpg and webp format saved in one file.
The results are as follows:
jpg: 30.633.851 bytes
webp: 25.845.760 bytes

File compression rate varies depending on what image save.

Christian Ehrlicher

So we really talk about 5MB more ram in contrast to slow loading speed. No need to discuss this further.

MyNameIsQt

@IgKh
I know what you mean and I tried it.
The following function is the original function.

QString WPubManager::getImgObject()
{
if (m_pBkgnd && m_pBkgnd->loadImageData(m_file)){
        const QImage& image = m_pBkgnd->getBkgndImg();
        qDebug() << "3 size" << image.width();

        if (image.isNull()) {
            qDebug() << "error: image is null";
            return QString(); 
        } else {
            qDebug() << "Image dimensions: " << image.width() << image.height();
        }

        QByteArray byteArray;
        QBuffer buffer(&byteArray);
        buffer.open(QIODevice::WriteOnly);

        image.save(&buffer, "WEBP");
        QString base64ImageData = QString::fromLatin1(byteArray.toBase64());
        qDebug() << "Base64 image data size: " << base64ImageData.size();
        buffer.close(); 

        return base64ImageData; 
    } else {
        qDebug() << "error: m_pBkgnd is null";
        return QString();
    }
}

The key here is tobase64: Images without this part processed will not be drawn on the canvas.

        QString base64ImageData = QString::fromLatin1(byteArray.toBase64());

Only after that part is processed can the next statement be processed in js.

// js
var img = new Image();
 img.src = "data:image/webp;base64," + imageData;

As you said, I sent several types of image data to js without using QImage.
But in any case, the toBase64() function must be processed. To do that, I don't know any other way than to go through QImage.

MyNameIsQt

@Christian-Ehrlicher
The saved image now contains many small thumbnails. For small thumbnails, there isn't much difference in compression rates between webp and jpg.

Additionally, the results shown now show that webp has a 25% more efficient compression rate. 5MB is not a big thing, but if 10,000 users upload files simultaneously to the web server (that I created), the amount of traffic or the physical size that occupies the server will not be negligible.

( * I had a pleasant fantasy of 10,000 people using the web I created at the same time. ha ha ha...)

IgKh

@MyNameIsQt I'm not questioning the need to pass the data to Javascript as Base-64 encoded text. But you can Base-64 encode any QByteArray...

Would you mind sharing how the QImage got created in your getBkgndImg() method? It should have been read from a byte array / binary buffer of some kind, and if that encodes a valid WebP image, directly Base-64 printing it should be sufficient.

To keep one step ahead, it is possible that the original image data is not actually WebP, or not valid enough in a way that Qt's image format plugin can handle but Chromium can't. You can double check by looking at the result of calling QImageReader::imageFormat on whatever you created the QImage from.

MyNameIsQt

@IgKh
getBkgndImg is just an inline function.

inline const QImage &getBkgndImg() const {
       if (m_pBkgndImage) {
           return *m_pBkgndImage;
       } else {            
           static const QImage emptyImage;
           return emptyImage;
       }
   }

The image data saved in the file is created using google's decodeWebp and encodeWebp functions. These work well. Checked in all browsers.

Base64 to Webp text
Webp
Thanks so much..

MyNameIsQt

@IgKh

// CBackground.h
   ...
   QByteArray m_decodedArray; 
    uint8_t* m_decodedData;
    size_t m_imageSize = 0;    
    ....
    inline const uchar* getDecodedData() const {
        return m_decodedData; // QByteArray 변환 없이 원시 데이터 포인터를 반환
    }

// CBackground.cpp
      bool CBackground::decodeWebp(QFile *pFile) 
      {      
          long imageSize = m_bkgndHeader.ImageSize;     
          long imageAddress = getImageAddress(); 
      
          if (!pFile->seek(imageAddress)) {      
              throw std::runtime_error("Failed to seek to image data");
                    return false;      
          }
      
          QByteArray imageData(imageSize, 0);      
          if (pFile->read(imageData.data(), imageSize) != imageSize) {      
              throw std::runtime_error("Failed to read image data");
                    return false;
                }      
          int width = 0, height = 0;      
          m_decodedData = WebPDecodeRGBA(reinterpret_cast<uchar*>(imageData.data()), imageSize, &width, &height);
      
          if (!m_decodedData) {      
              throw std::runtime_error("Failed to decode WebP image");      
              return false;      
          }
          m_imageSize = width * height * 4;

          retirm trie;
}

// wemar.cpp
QString Wemar::getImgObject()
{
       if (m_pBkgnd && m_pBkgnd->loadImageData(m_file)){       
            const uchar* decodedData = m_pBkgnd->getDecodedData();      
            QByteArray webpData(reinterpret_cast<const char*>(decodedData), m_pBkgnd->getImageSize());
     
               if (webpData.isEmpty()) {    
                qDebug() << "error: image data is empty";    
                return QString();   
            } else {    
                qDebug() << "Base64 before: " << webpData.size();                    
            }   
        QString base64ImageData = QString::fromLatin1(webpData.toBase64());
        qDebug() << "Base64 After: " << base64ImageData.size();

        return base64ImageData;            
        } else {    
            qDebug() << "error: m_pBkgnd is null";    
            return QString();     
        }
}

/// js
  new QWebChannel(qt.webChannelTransport, function(channel) {
               wpubManager = channel.objects.wpubManager;
            
    ....
 async function handleImage() {
        var bkgndImageData = await wpubManager.getImgObject(); // Promise 완료를 기다림
        if (bkgndImageData) {
            console.error("image good");
            drawCanvas(bkgndImageData);
        } else {
            console.error("Failed to get background image data.");
        }
    }
    ....
    // in drawCanvas function
        var img = new Image();
        img.src = "data:image/webp;base64," + imageData;

        img.onerror = function(){
            console.error("Image failed to load");
        };

        img.onload = function() {
            canvas.width = img.width;
            canvas.height = img.height;

            context.fillStyle = "#d4d4d4";
            context.fillRect(0, 0, canvas.width, canvas.height);

            context.drawImage(img, 0, 0, img.width, img.height);
        };

IgKh

@MyNameIsQt Thanks!

In this bit:

QByteArray imageData(imageSize, 0);      
if (pFile->read(imageData.data(), imageSize) != imageSize) {      
    throw std::runtime_error("Failed to read image data");
    return false;
}

Unless something funny is going on, it should sufficient to call QString::fromLatin1(imageData.toBase64()) after that point and send the result over to the web frame.

(P.S no need to return after throwing, the return is unreachable as exceptions propagate over an alternative control flow)

MyNameIsQt

@IgKh You didn't show me a direct solution to the problem, but you showed me the right approach to solving the problem. Using qimage as you mentioned is unnecessary. I need to focus on the issue of base64 encoding binary data. thank you