How to save image (about 36 mb) to db via QSqlQuery?
-
Hello there!
I have a problem with QSqlQuery, when trying to save relatively large (~36 mb) image. When I call QSqlQuery::exec(), I get std::bad_alloc exeption. When I work with smaller images, everything is ok. Need to be sayed - this code executes in other instance of QThread in order to prevent GUI freezing, all code compiled in MinGW-32 compiler, there are 8 GB RAM on PC. And some times (very rarely) I've got this code running without this exception even with images this size (36 mb)here is the code:
...... QByteArray byte_array_image, byte_array_icon; QBuffer buffer_image(&byte_array_image), buffer_icon(&byte_array_icon); buffer_image.open(QIODevice::WriteOnly); buffer_icon.open(QIODevice::WriteOnly); image->save(&buffer_image, format.toLocal8Bit().data()); icon->save(&buffer_icon, "JPEG"); query.prepare("INSERT INTO pictures (id, id_act, date_time, name, picture, icon) VALUES (:id, :id_act, :date_time, :name, :picture, :icon)"); ...... query.bindValue(":picture", byte_array_image); query.bindValue(":icon", byte_array_icon); byte_array_image.clear(); byte_array_icon.clear(); save_succeed = query.exec(); if (!save_succeed) emit database_failed(QUERY_ERROR_HEADER, QUERY_ERROR_SAVE_IMAGE + query.lastError().text()); emit query_finished(); return save_succeed;
As far as I can see, problem can be related with lack of heap space for QApplication to create instance of QString included all bytecode of image. In order to fix that, I ve tried to split this image up to ten parts, but it doesn't help.
Here is ten-parts-splitted query code:
............ QByteArray byte_array_image, byte_array_icon; QBuffer buffer_image(&byte_array_image), buffer_icon(&byte_array_icon); buffer_image.open(QIODevice::WriteOnly); buffer_icon.open(QIODevice::WriteOnly); image->save(&buffer_image, format.toLocal8Bit().data()); icon->save(&buffer_icon, "JPEG"); int partlen, parts_count = 0; save_succeed = false; while (!save_succeed && ++parts_count < 10) { partlen = byte_array_image.size(); partlen = byte_array_image.size() / parts_count; query.clear(); query.prepare("INSERT INTO pictures (id, id_act, date_time, name, part_one, parts_count, icon, src, src_act) " "VALUES (:id, :id_act, :date_time, :name, :part_one, :parts_count, :icon, :src, :src_act)"); ........ query.bindValue(":part_one", byte_array_image.mid(0, partlen)); query.bindValue(":parts_count", parts_count); query.bindValue(":icon", byte_array_icon); try { save_succeed = query.exec(); } catch (std::bad_alloc &error) { if (parts_count < 10) continue; emit database_failed(QUERY_HEAP_ERROR_HEADER, QUERY_HEAP_ERROR_TEXT); return false; } } if (save_succeed) { QStringList part_names = { "part_one", "part_two", "part_three", "part_four", "part_five", "part_six", "part_seven", "part_eight", "part_nine", "part_ten"}; for (int current_part = 1; current_part < parts_count; ++current_part) { query.clear(); query.prepare("UPDATE pictures SET "+part_names[current_part]+" = :current_part WHERE id = :id AND src = :src"); query.bindValue(":current_part", current_part == parts_count - 1 ? byte_array_image.mid(current_part*partlen) : byte_array_image.mid(current_part*partlen, partlen)); ........ save_succeed = query.exec(); } } byte_array_image.clear(); byte_array_icon.clear();
Am I correct with the problem source? Is there anything I can do to fix it? 36 mb for image is not so much.
-
Hi,
You should check the current memory consumption of your application. Since you are building a 32bit application, you are limited to 2GB of RAM available to your process.
-
Hi,
You should check the current memory consumption of your application. Since you are building a 32bit application, you are limited to 2GB of RAM available to your process.
@sgaist
Hello and thank you. I checked in task manager, On start my application consumes around 400 mb, when I trying to take snap, this amount grows to 900 mb, but then drops to~ 600. So with every snap I loose about 200 mb.
I guess that is the problem. Can provided code leads to mem leak? -
What do you mean by snap ?
What kind of image are you manipulating ?
From the looks of it, you are making at list one copy of your image in a format that may or may not be compressed.
I don't know about the icon.You should profile the application to see what is going on.
-
What do you mean by snap ?
What kind of image are you manipulating ?
From the looks of it, you are making at list one copy of your image in a format that may or may not be compressed.
I don't know about the icon.You should profile the application to see what is going on.
@sgaist
By snap I mean image, which I'm pulling from camera
I've noticed when I'm using try-catch block on qsqlquery::exec() it leads to memory leak. I mean is there a possability that when bad_alloc occurs inside qsqlquery class, program exits from exec() function without propper freeing memory up ? -
Why do you use a QImage pointer - where do you free it?
-
Why do you use a QImage pointer - where do you free it?
@christian-ehrlicher
I call this function from another one where QImage declared as a static variable. -
And you're sure you don't allocate it again on every call?
/edit: And why needs it to be a pointer at all?
-
Hello there!
I have a problem with QSqlQuery, when trying to save relatively large (~36 mb) image. When I call QSqlQuery::exec(), I get std::bad_alloc exeption. When I work with smaller images, everything is ok. Need to be sayed - this code executes in other instance of QThread in order to prevent GUI freezing, all code compiled in MinGW-32 compiler, there are 8 GB RAM on PC. And some times (very rarely) I've got this code running without this exception even with images this size (36 mb)here is the code:
...... QByteArray byte_array_image, byte_array_icon; QBuffer buffer_image(&byte_array_image), buffer_icon(&byte_array_icon); buffer_image.open(QIODevice::WriteOnly); buffer_icon.open(QIODevice::WriteOnly); image->save(&buffer_image, format.toLocal8Bit().data()); icon->save(&buffer_icon, "JPEG"); query.prepare("INSERT INTO pictures (id, id_act, date_time, name, picture, icon) VALUES (:id, :id_act, :date_time, :name, :picture, :icon)"); ...... query.bindValue(":picture", byte_array_image); query.bindValue(":icon", byte_array_icon); byte_array_image.clear(); byte_array_icon.clear(); save_succeed = query.exec(); if (!save_succeed) emit database_failed(QUERY_ERROR_HEADER, QUERY_ERROR_SAVE_IMAGE + query.lastError().text()); emit query_finished(); return save_succeed;
As far as I can see, problem can be related with lack of heap space for QApplication to create instance of QString included all bytecode of image. In order to fix that, I ve tried to split this image up to ten parts, but it doesn't help.
Here is ten-parts-splitted query code:
............ QByteArray byte_array_image, byte_array_icon; QBuffer buffer_image(&byte_array_image), buffer_icon(&byte_array_icon); buffer_image.open(QIODevice::WriteOnly); buffer_icon.open(QIODevice::WriteOnly); image->save(&buffer_image, format.toLocal8Bit().data()); icon->save(&buffer_icon, "JPEG"); int partlen, parts_count = 0; save_succeed = false; while (!save_succeed && ++parts_count < 10) { partlen = byte_array_image.size(); partlen = byte_array_image.size() / parts_count; query.clear(); query.prepare("INSERT INTO pictures (id, id_act, date_time, name, part_one, parts_count, icon, src, src_act) " "VALUES (:id, :id_act, :date_time, :name, :part_one, :parts_count, :icon, :src, :src_act)"); ........ query.bindValue(":part_one", byte_array_image.mid(0, partlen)); query.bindValue(":parts_count", parts_count); query.bindValue(":icon", byte_array_icon); try { save_succeed = query.exec(); } catch (std::bad_alloc &error) { if (parts_count < 10) continue; emit database_failed(QUERY_HEAP_ERROR_HEADER, QUERY_HEAP_ERROR_TEXT); return false; } } if (save_succeed) { QStringList part_names = { "part_one", "part_two", "part_three", "part_four", "part_five", "part_six", "part_seven", "part_eight", "part_nine", "part_ten"}; for (int current_part = 1; current_part < parts_count; ++current_part) { query.clear(); query.prepare("UPDATE pictures SET "+part_names[current_part]+" = :current_part WHERE id = :id AND src = :src"); query.bindValue(":current_part", current_part == parts_count - 1 ? byte_array_image.mid(current_part*partlen) : byte_array_image.mid(current_part*partlen, partlen)); ........ save_succeed = query.exec(); } } byte_array_image.clear(); byte_array_icon.clear();
Am I correct with the problem source? Is there anything I can do to fix it? 36 mb for image is not so much.
@august-wiener said in How to save image (about 36 mb) to db via QSqlQuery?:
36 mb for image is not so much.
For a file, 36 MB is not too much. For a single field in a relational database, 36 MB is extremely unusual. You will probably have a lot more luck saving the image to a file, and just retaining the filename in the database as a short string. This is the much more common way to do it. The SQL database isn't really designed around storing large binary blobs, and neither are the associated libraries.