Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Get Qt Extensions
  • Unsolved
Collapse
Brand Logo
  1. Home
  2. Qt Development
  3. General and Desktop
  4. How to save image (about 36 mb) to db via QSqlQuery?
Forum Update on Monday, May 27th 2025

How to save image (about 36 mb) to db via QSqlQuery?

Scheduled Pinned Locked Moved Unsolved General and Desktop
9 Posts 4 Posters 791 Views
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • A Offline
    A Offline
    August Wiener
    wrote on 23 Aug 2019, 20:05 last edited by
    #1

    Hello there!
    I have a problem with QSqlQuery, when trying to save relatively large (~36 mb) image. When I call QSqlQuery::exec(), I get std::bad_alloc exeption. When I work with smaller images, everything is ok. Need to be sayed - this code executes in other instance of QThread in order to prevent GUI freezing, all code compiled in MinGW-32 compiler, there are 8 GB RAM on PC. And some times (very rarely) I've got this code running without this exception even with images this size (36 mb)

    here is the code:

    ......
    QByteArray byte_array_image, byte_array_icon;
    QBuffer buffer_image(&byte_array_image), buffer_icon(&byte_array_icon);
    
    buffer_image.open(QIODevice::WriteOnly);
    buffer_icon.open(QIODevice::WriteOnly);
    
    image->save(&buffer_image, format.toLocal8Bit().data());
    icon->save(&buffer_icon, "JPEG");
    
    query.prepare("INSERT INTO pictures (id, id_act, date_time, name, picture, icon) VALUES (:id, :id_act, :date_time, :name, :picture, :icon)");
    ......
    query.bindValue(":picture", byte_array_image);
    query.bindValue(":icon", byte_array_icon);
    byte_array_image.clear();
    byte_array_icon.clear();
    
    save_succeed = query.exec();
    
    if (!save_succeed)
        emit database_failed(QUERY_ERROR_HEADER, QUERY_ERROR_SAVE_IMAGE + query.lastError().text());
    
    emit query_finished();
    
    return save_succeed;
    

    As far as I can see, problem can be related with lack of heap space for QApplication to create instance of QString included all bytecode of image. In order to fix that, I ve tried to split this image up to ten parts, but it doesn't help.

    Here is ten-parts-splitted query code:

    ............
    QByteArray byte_array_image, byte_array_icon;
    QBuffer buffer_image(&byte_array_image), buffer_icon(&byte_array_icon);
    
    buffer_image.open(QIODevice::WriteOnly);
    buffer_icon.open(QIODevice::WriteOnly);
    
    image->save(&buffer_image, format.toLocal8Bit().data());
    icon->save(&buffer_icon, "JPEG");
    
    int partlen, parts_count = 0;
    save_succeed = false;
    
    while (!save_succeed && ++parts_count < 10)
    {
         partlen = byte_array_image.size();
         partlen = byte_array_image.size() / parts_count;
    
         query.clear();
         query.prepare("INSERT INTO pictures (id, id_act, date_time, name, part_one, parts_count, icon, src, src_act) "
                          "VALUES (:id, :id_act, :date_time, :name, :part_one, :parts_count, :icon, :src, :src_act)");
         ........
         query.bindValue(":part_one", byte_array_image.mid(0, partlen));
         query.bindValue(":parts_count", parts_count);
         query.bindValue(":icon", byte_array_icon);
    
         try
         {
             save_succeed = query.exec();
         }
         catch (std::bad_alloc &error)
         {
             if (parts_count < 10)
                 continue;
    
             emit database_failed(QUERY_HEAP_ERROR_HEADER, QUERY_HEAP_ERROR_TEXT);
    
             return false;
         }
    
     }
    
     if (save_succeed)
     {
        QStringList part_names = { "part_one", "part_two", "part_three", "part_four", "part_five",
                                       "part_six", "part_seven", "part_eight", "part_nine", "part_ten"};
    
        for (int current_part = 1; current_part < parts_count; ++current_part)
        {
            query.clear();
    
            query.prepare("UPDATE pictures SET "+part_names[current_part]+" = :current_part WHERE id = :id AND src = :src");
            query.bindValue(":current_part", current_part == parts_count - 1 ? byte_array_image.mid(current_part*partlen) : byte_array_image.mid(current_part*partlen, partlen));
            ........
            save_succeed = query.exec();
        }
     }
    
     byte_array_image.clear();
     byte_array_icon.clear();
    

    Am I correct with the problem source? Is there anything I can do to fix it? 36 mb for image is not so much.

    W 1 Reply Last reply 26 Aug 2019, 18:04
    0
    • S Offline
      S Offline
      SGaist
      Lifetime Qt Champion
      wrote on 23 Aug 2019, 20:09 last edited by
      #2

      Hi,

      You should check the current memory consumption of your application. Since you are building a 32bit application, you are limited to 2GB of RAM available to your process.

      Interested in AI ? www.idiap.ch
      Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

      A 1 Reply Last reply 23 Aug 2019, 20:34
      2
      • S SGaist
        23 Aug 2019, 20:09

        Hi,

        You should check the current memory consumption of your application. Since you are building a 32bit application, you are limited to 2GB of RAM available to your process.

        A Offline
        A Offline
        August Wiener
        wrote on 23 Aug 2019, 20:34 last edited by
        #3

        @sgaist
        Hello and thank you. I checked in task manager, On start my application consumes around 400 mb, when I trying to take snap, this amount grows to 900 mb, but then drops to~ 600. So with every snap I loose about 200 mb.
        I guess that is the problem. Can provided code leads to mem leak?

        1 Reply Last reply
        0
        • S Offline
          S Offline
          SGaist
          Lifetime Qt Champion
          wrote on 23 Aug 2019, 20:49 last edited by
          #4

          What do you mean by snap ?

          What kind of image are you manipulating ?
          From the looks of it, you are making at list one copy of your image in a format that may or may not be compressed.
          I don't know about the icon.

          You should profile the application to see what is going on.

          Interested in AI ? www.idiap.ch
          Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

          A 1 Reply Last reply 25 Aug 2019, 14:23
          0
          • S SGaist
            23 Aug 2019, 20:49

            What do you mean by snap ?

            What kind of image are you manipulating ?
            From the looks of it, you are making at list one copy of your image in a format that may or may not be compressed.
            I don't know about the icon.

            You should profile the application to see what is going on.

            A Offline
            A Offline
            August Wiener
            wrote on 25 Aug 2019, 14:23 last edited by August Wiener
            #5

            @sgaist
            By snap I mean image, which I'm pulling from camera
            I've noticed when I'm using try-catch block on qsqlquery::exec() it leads to memory leak. I mean is there a possability that when bad_alloc occurs inside qsqlquery class, program exits from exec() function without propper freeing memory up ?

            1 Reply Last reply
            0
            • Christian EhrlicherC Offline
              Christian EhrlicherC Offline
              Christian Ehrlicher
              Lifetime Qt Champion
              wrote on 25 Aug 2019, 14:55 last edited by
              #6

              Why do you use a QImage pointer - where do you free it?

              Qt Online Installer direct download: https://download.qt.io/official_releases/online_installers/
              Visit the Qt Academy at https://academy.qt.io/catalog

              A 1 Reply Last reply 25 Aug 2019, 16:44
              0
              • Christian EhrlicherC Christian Ehrlicher
                25 Aug 2019, 14:55

                Why do you use a QImage pointer - where do you free it?

                A Offline
                A Offline
                August Wiener
                wrote on 25 Aug 2019, 16:44 last edited by
                #7

                @christian-ehrlicher
                I call this function from another one where QImage declared as a static variable.

                1 Reply Last reply
                0
                • Christian EhrlicherC Offline
                  Christian EhrlicherC Offline
                  Christian Ehrlicher
                  Lifetime Qt Champion
                  wrote on 25 Aug 2019, 18:03 last edited by Christian Ehrlicher
                  #8

                  And you're sure you don't allocate it again on every call?

                  /edit: And why needs it to be a pointer at all?

                  Qt Online Installer direct download: https://download.qt.io/official_releases/online_installers/
                  Visit the Qt Academy at https://academy.qt.io/catalog

                  1 Reply Last reply
                  0
                  • A August Wiener
                    23 Aug 2019, 20:05

                    Hello there!
                    I have a problem with QSqlQuery, when trying to save relatively large (~36 mb) image. When I call QSqlQuery::exec(), I get std::bad_alloc exeption. When I work with smaller images, everything is ok. Need to be sayed - this code executes in other instance of QThread in order to prevent GUI freezing, all code compiled in MinGW-32 compiler, there are 8 GB RAM on PC. And some times (very rarely) I've got this code running without this exception even with images this size (36 mb)

                    here is the code:

                    ......
                    QByteArray byte_array_image, byte_array_icon;
                    QBuffer buffer_image(&byte_array_image), buffer_icon(&byte_array_icon);
                    
                    buffer_image.open(QIODevice::WriteOnly);
                    buffer_icon.open(QIODevice::WriteOnly);
                    
                    image->save(&buffer_image, format.toLocal8Bit().data());
                    icon->save(&buffer_icon, "JPEG");
                    
                    query.prepare("INSERT INTO pictures (id, id_act, date_time, name, picture, icon) VALUES (:id, :id_act, :date_time, :name, :picture, :icon)");
                    ......
                    query.bindValue(":picture", byte_array_image);
                    query.bindValue(":icon", byte_array_icon);
                    byte_array_image.clear();
                    byte_array_icon.clear();
                    
                    save_succeed = query.exec();
                    
                    if (!save_succeed)
                        emit database_failed(QUERY_ERROR_HEADER, QUERY_ERROR_SAVE_IMAGE + query.lastError().text());
                    
                    emit query_finished();
                    
                    return save_succeed;
                    

                    As far as I can see, problem can be related with lack of heap space for QApplication to create instance of QString included all bytecode of image. In order to fix that, I ve tried to split this image up to ten parts, but it doesn't help.

                    Here is ten-parts-splitted query code:

                    ............
                    QByteArray byte_array_image, byte_array_icon;
                    QBuffer buffer_image(&byte_array_image), buffer_icon(&byte_array_icon);
                    
                    buffer_image.open(QIODevice::WriteOnly);
                    buffer_icon.open(QIODevice::WriteOnly);
                    
                    image->save(&buffer_image, format.toLocal8Bit().data());
                    icon->save(&buffer_icon, "JPEG");
                    
                    int partlen, parts_count = 0;
                    save_succeed = false;
                    
                    while (!save_succeed && ++parts_count < 10)
                    {
                         partlen = byte_array_image.size();
                         partlen = byte_array_image.size() / parts_count;
                    
                         query.clear();
                         query.prepare("INSERT INTO pictures (id, id_act, date_time, name, part_one, parts_count, icon, src, src_act) "
                                          "VALUES (:id, :id_act, :date_time, :name, :part_one, :parts_count, :icon, :src, :src_act)");
                         ........
                         query.bindValue(":part_one", byte_array_image.mid(0, partlen));
                         query.bindValue(":parts_count", parts_count);
                         query.bindValue(":icon", byte_array_icon);
                    
                         try
                         {
                             save_succeed = query.exec();
                         }
                         catch (std::bad_alloc &error)
                         {
                             if (parts_count < 10)
                                 continue;
                    
                             emit database_failed(QUERY_HEAP_ERROR_HEADER, QUERY_HEAP_ERROR_TEXT);
                    
                             return false;
                         }
                    
                     }
                    
                     if (save_succeed)
                     {
                        QStringList part_names = { "part_one", "part_two", "part_three", "part_four", "part_five",
                                                       "part_six", "part_seven", "part_eight", "part_nine", "part_ten"};
                    
                        for (int current_part = 1; current_part < parts_count; ++current_part)
                        {
                            query.clear();
                    
                            query.prepare("UPDATE pictures SET "+part_names[current_part]+" = :current_part WHERE id = :id AND src = :src");
                            query.bindValue(":current_part", current_part == parts_count - 1 ? byte_array_image.mid(current_part*partlen) : byte_array_image.mid(current_part*partlen, partlen));
                            ........
                            save_succeed = query.exec();
                        }
                     }
                    
                     byte_array_image.clear();
                     byte_array_icon.clear();
                    

                    Am I correct with the problem source? Is there anything I can do to fix it? 36 mb for image is not so much.

                    W Offline
                    W Offline
                    wrosecrans
                    wrote on 26 Aug 2019, 18:04 last edited by
                    #9

                    @august-wiener said in How to save image (about 36 mb) to db via QSqlQuery?:

                    36 mb for image is not so much.

                    For a file, 36 MB is not too much. For a single field in a relational database, 36 MB is extremely unusual. You will probably have a lot more luck saving the image to a file, and just retaining the filename in the database as a short string. This is the much more common way to do it. The SQL database isn't really designed around storing large binary blobs, and neither are the associated libraries.

                    1 Reply Last reply
                    1

                    1/9

                    23 Aug 2019, 20:05

                    • Login

                    • Login or register to search.
                    1 out of 9
                    • First post
                      1/9
                      Last post
                    0
                    • Categories
                    • Recent
                    • Tags
                    • Popular
                    • Users
                    • Groups
                    • Search
                    • Get Qt Extensions
                    • Unsolved