Opening .tiff geoimages consumes enormous memory

john_hobbyist

Hello,

I am very new to PyQt. I open huge .tiff images in a PyQt application. Each image is about 500MB. When it finally opens, I get this message:

cv2.error: OpenCV(4.4.0) /tmp/pip-req-build-h2062vqd/opencv/modules/core/src/alloc.cpp:73: error: (-4:Insufficient memory) Failed to allocate 24586133472 bytes in function 'OutOfMemoryError'

How can I fix this? without downsampling the image because I will use information...

Thank you for your time...

gde23

The out of memory error in openCV really means that your memory (RAM) is full.
As mentioned in the error message the loading of the image somehow needs about 25GB of RAM which your machine does not provide. So the first question would be why it needs so much memory when the image is only 500MB.

john_hobbyist

But the .tiff is compressed, isn't it?

I execute this code to open the .tiff

code.py

gde23

You're right. Didn't think about that.
What size (in pixels) does the image have and what datatype do you load it to?
Then you can easily calculate what size (in memory) it should have as a openCV mat object and if the image should fit into your PCs memory.

john_hobbyist

5000 x 7000 pixels. I use opencv.

mrjj

Hi
A 500 MB tiff is massive if high compression is used.
Unless you have 32+ GB ram, it might not be possible to open it.

The alternative is to load it in chunks
https://wiki.qt.io/Loading_Large_Images

But depending on what you need to do with it, it might not be a solution.
Also I have not tested this with the TIFF format.

Ah, never mind. you want use openCV to load the image.

update:
You might want to check out
https://github.com/pearu/pylibtiff
it's a python wrapper for libtiff which is supports tiles (chunks) directly.
So you might be able to load it via this and us openCV to display.
https://answers.opencv.org/question/86621/reading-geo-tiff/#86740

SGaist

Hi,

One thing to take into account: to represent a compressed image in a format like RGB, you will have to decompress it. You are mixing storage and usage. Depending on what you need from that image, you may have a thumbnail already embedded into it. In any case, you really should analyse what you need to do with that image and then adapt your code for that. Like suggested by my fellows there are tools to deal with these kind of data.

john_hobbyist

I am trying opening the tiff with this command:

QImageReader reader("/home/UbuntuUser/Desktop/UAV.tif");
print(reader)

but something is wrong... Any idea?

update:

File "code.py", line 547
    QImageReader reader("/home/UbuntuUser/Desktop/UAV.tif");
                 ^
SyntaxError: invalid syntax

SGaist

What does "but something is wrong" mean ?

mrjj

Hi
That looks like c++ syntax but you are using python, right ?
So is it not like
r = QImageReader("/home/UbuntuUser/Desktop/UAV.tif")

john_hobbyist

This post is deleted!

mrjj

@john_hobbyist

Hi
Why this old version ?

Anyway, what are you trying ?

Building PyQT4 from source ?

Also is there any reason you want to use the competing Qt binding from another company instead of the
official support one called pyside2 ?
https://pypi.org/project/PySide2/
https://doc.qt.io/qtforpython/

john_hobbyist

@mrjj

I don't know what goes wrong...

I just try to handle large tiff images instead of using opencv (that makes my program realy slow) with QTimagereader

Sorry, I didn't identify it was a competing company...just a result in google when tried to install PyQt4

Ok, I will look at it.

mrjj

@john_hobbyist
Hi
Np I just wondered why you want Qt4 version and build from the source when
PyQt5 is there and seems to be installable directly.
Also here is how to use it
https://programtalk.com/python-examples/PyQt5.Qt.QImageReader/
Do note none of them use setClipRect to load image in chunks but
should be easy to add.

Christian Ehrlicher

QtImagrReader is for sure not the correct way to handle geotiff. Either use openCV or better gdal for such tasks.

john_hobbyist

@Christian-Ehrlicher

I am trying GDAL in order to print the image:

from osgeo import gdal
import sys

ds = gdal.Open('UAV_image.tif')
print(ds)

and I get this error:

<osgeo.gdal.Dataset; proxy of <Swig Object of type 'GDALDatasetShadow *' at 0x7fb89062b840> >

Any idea what I am missing here?

Christian Ehrlicher

@john_hobbyist said in Opening .tiff geoimages consumes enormous memory:

Any idea what I am missing here?

Nothing. All is fine.

john_hobbyist

Why I cannot see the image?

SGaist

The Python print function only print strings or a string representation of the object.

Check if GDAL provides a widget or equivalent for that.

john_hobbyist

Ok, until now OpenCV requires ~24GB RAM, gdal does not work. Is there any QImageReader sample code to open the .tif? I am searching the google for this...nothing found.