Opening .tiff geoimages consumes enormous memory

gde23 · wrote on 24 Dec 2020, 17:33

The out of memory error in openCV really means that your memory (RAM) is full.
As mentioned in the error message the loading of the image somehow needs about 25GB of RAM which your machine does not provide. So the first question would be why it needs so much memory when the image is only 500MB.

john_hobbyist · wrote on 24 Dec 2020, 18:43

But the .tiff is compressed, isn't it?

I execute this code to open the .tiff

code.py

gde23 · wrote on 24 Dec 2020, 19:09

You're right. Didn't think about that.
What size (in pixels) does the image have and what datatype do you load it to?
Then you can easily calculate what size (in memory) it should have as a openCV mat object and if the image should fit into your PCs memory.

john_hobbyist · wrote on 24 Dec 2020, 19:13

5000 x 7000 pixels. I use opencv.

mrjj · wrote on 24 Dec 2020, 19:14

Hi
A 500 MB tiff is massive if high compression is used.
Unless you have 32+ GB ram, it might not be possible to open it.

The alternative is to load it in chunks
https://wiki.qt.io/Loading_Large_Images

But depending on what you need to do with it, it might not be a solution.
Also I have not tested this with the TIFF format.

Ah, never mind. you want use openCV to load the image.

update:
You might want to check out
https://github.com/pearu/pylibtiff
it's a python wrapper for libtiff which is supports tiles (chunks) directly.
So you might be able to load it via this and us openCV to display.
https://answers.opencv.org/question/86621/reading-geo-tiff/#86740

SGaist · wrote on 24 Dec 2020, 20:22

Hi,

One thing to take into account: to represent a compressed image in a format like RGB, you will have to decompress it. You are mixing storage and usage. Depending on what you need from that image, you may have a thumbnail already embedded into it. In any case, you really should analyse what you need to do with that image and then adapt your code for that. Like suggested by my fellows there are tools to deal with these kind of data.

john_hobbyist · wrote on 28 Dec 2020, 19:28

I am trying opening the tiff with this command:

QImageReader reader("/home/UbuntuUser/Desktop/UAV.tif");
print(reader)

but something is wrong... Any idea?

update:

File "code.py", line 547
    QImageReader reader("/home/UbuntuUser/Desktop/UAV.tif");
                 ^
SyntaxError: invalid syntax

SGaist · wrote on 28 Dec 2020, 19:29

What does "but something is wrong" mean ?

mrjj · wrote on 28 Dec 2020, 20:05

Hi
That looks like c++ syntax but you are using python, right ?
So is it not like
r = QImageReader("/home/UbuntuUser/Desktop/UAV.tif")

john_hobbyist · 29 Dec 2020, 08:59

This post is deleted!

mrjj · wrote on 29 Dec 2020, 08:59

@john_hobbyist

Hi
Why this old version ?

Anyway, what are you trying ?

Building PyQT4 from source ?

Also is there any reason you want to use the competing Qt binding from another company instead of the
official support one called pyside2 ?
https://pypi.org/project/PySide2/
https://doc.qt.io/qtforpython/

john_hobbyist · 29 Dec 2020, 09:38

@mrjj

I don't know what goes wrong...

I just try to handle large tiff images instead of using opencv (that makes my program realy slow) with QTimagereader

Sorry, I didn't identify it was a competing company...just a result in google when tried to install PyQt4

Ok, I will look at it.

mrjj · wrote on 29 Dec 2020, 09:38

@john_hobbyist
Hi
Np I just wondered why you want Qt4 version and build from the source when
PyQt5 is there and seems to be installable directly.
Also here is how to use it
https://programtalk.com/python-examples/PyQt5.Qt.QImageReader/
Do note none of them use setClipRect to load image in chunks but
should be easy to add.

Christian Ehrlicher · 30 Dec 2020, 15:35

QtImagrReader is for sure not the correct way to handle geotiff. Either use openCV or better gdal for such tasks.

john_hobbyist · wrote on 30 Dec 2020, 15:35

@Christian-Ehrlicher

I am trying GDAL in order to print the image:

from osgeo import gdal
import sys

ds = gdal.Open('UAV_image.tif')
print(ds)

and I get this error:

<osgeo.gdal.Dataset; proxy of <Swig Object of type 'GDALDatasetShadow *' at 0x7fb89062b840> >

Any idea what I am missing here?

Christian Ehrlicher · wrote on 30 Dec 2020, 16:25

@john_hobbyist said in Opening .tiff geoimages consumes enormous memory:

Any idea what I am missing here?

Nothing. All is fine.

john_hobbyist · wrote on 30 Dec 2020, 16:27

Why I cannot see the image?

SGaist · wrote on 30 Dec 2020, 16:29

The Python print function only print strings or a string representation of the object.

Check if GDAL provides a widget or equivalent for that.

john_hobbyist · 30 Dec 2020, 17:03

Ok, until now OpenCV requires ~24GB RAM, gdal does not work. Is there any QImageReader sample code to open the .tif? I am searching the google for this...nothing found.

mrjj · wrote on 30 Dec 2020, 17:03

@john_hobbyist
Hi
in what way did GDAL not work ?

Also, Mr @Christian-Ehrlicher made me realize that geotiff is a special tiff so
even QImageReader can read an image in sections. it might not really do what you want.

What do you need to do to this image ?