TIFF CMYKA image loses transparency on load/conversion to Format_ARGB32

Crin

Hello all,

We have a transparent, LZW compressed TIFF file (test.tif) which I'm trying to display as an ARGB32 image, with transparency, but it displays as an opaque image; it appears that the image::load() loses transparency on conversion from TIFF CMYKA to QImage::Format_ARGB32.

Testing using a PNG version of the image (test.png) produces the desired results, where the image is displayed as partly transparent. In this case, the image::load() converts to a QImage::Format_Indexed8.

Steps to reproduce:
I've been testing this using the Qt 5.3 Image Composition Example distributed with Qt Creator 3.2.1

On first startup this displays as shown below, with a transparent butterfly overlaid on a chequered background:
screenshot1
Clicking on the leftmost (butterfly) image and selecting 'test.tif' produces the following (BAD - image is completely opaque):
screenshot2
Using QImage::load(test.tif) converts the tiff file to a QImage::Format_ARGB32 (format 5) QImage, but the image is opaque. hasAlphaChannel() returns true.
Clicking again on the leftmost image and selecting 'test.png' produces the following (GOOD - image is partly transparent):
screenshot3
Using QImage::load(test.png) converts the tiff file to a QImage::Format_Indexed8 (format 3) QImage, and the image is partly transparent. hasAlphaChannel() returns true.

Does anyone out there have any suggestions as to how I might address this, for example using GIL, or another TIFF reader plugin?

Using GIL we are able to split the uncompressed alpha and CMYK channel data out from the CMYK into std::vector<unsigned char> variables, but I have no idea how to map/convert this into a QImage structure which displays correctly ...?

	std::vector<unsigned char> strImageDataUncompressed;
	std::vector<unsigned char> strImageDataAlpha
	cmyka8_image_t 
		cmyka_img;
	read_image(in, cmyka_img, tiff_tag());

	cmyk8_image_t
		dst_cmyk_image(cmyka_img.dimensions());
	cmyka8_planar_image_t
		dst_planar_image(cmyka_img.dimensions());

	// extract CMYK and alpha mask from CMYKA
	copy_and_convert_pixels( view( cmyka_img ), view( dst_cmyk_image ), convert_cmyka_to_cmyk() );
	copy_and_convert_pixels(view(cmyka_img), view(dst_planar_image));
	
	const unsigned char
		*cmyk_ptr = reinterpret_cast<const unsigned char *>(interleaved_view_get_raw_data(view(dst_cmyk_image))),
		// layer 4 below is for alpha data
		*alpha_ptr = reinterpret_cast<const unsigned char*>(planar_view_get_raw_data(view(dst_planar_image), 4));

	// store uncompressed image data in std::vector<..>s
	copy(cmyk_ptr, cmyk_ptr + (info._height * info._width * 4),std::back_inserter(strImageDataUncompressed()));
	copy(alpha_ptr, alpha_ptr + (info._height * info._width), std::back_inserter(strImageDataAlpha()));

Any ideas would be gratefully received ...

Thanks a million for reading this far.

Environment:
We're using Qt Creator 3.2.1 (opensource)
Based on Qt 5.3.2 (MSVC 2010, 32 bit)

mrjj

Hi and welcome
It could be a bug?
https://bugreports.qt.io/browse/QTBUG-50902

Can you test with Qt 5.7 or at least
5.6.1?

VRonin

Qt 5.7 tested and still affected

VRonin

when running it I get a warning on load:

TIFFReadDirectory: Warning, Unknown field with tag 37724 (0x935c) encountered.

So I tried with a .tif file I created on the fly with Gimp ( link here for 30 days) and it works perfectly.

The problem seems to be with your test.tif rather than anything in Qt

Crin

@VRonin Thank you for your update, VRonin

Crin

@mrjj Many thanks for your input mrjj, the patch for QTBUG-50902 looked very promising but according to vronin the behaviour is the same in Qt5.7 ...

mrjj

@Crin

well his TIFF did display correctly.
I also made a trans TIFF in photoshop and it worked too.

One diff i spotted with your test and his was CMYK vs RGB so maybe only some tiff is affected as
its clearly possible to make it work. (just test with his tiff)

Crin

@VRonin
Hi again VRonin,

Thanks for trying it. Your time and help is appreciated.

I should have said the sample TIFF was created in and saved from
Adobe Photoshop. When I use the Generic Image Library (outside Qt),
everything works fine, in the sense that I can load and extract the alpha channel
and image data. And I can import and display the image in
other applications. And stuff the image into PDF using the GIL
to separate the streams ...

AFAIK the warning message you saw does not indicate anything
wrong with the TIFF file per se, it's just a quirk in the way
the TIFF library handles legitimate but unrecognised tags. See, e.g.

imagemagick discussion

I don't know how Adobe uses tag 37724, but I don't think its presence
should cause a problem. Also no problems are suggested by

identify -verbose

The image you posted has (from Imagemagick's identify -verbose):

Colorspace: Gray
Compression: None
ICC-profile: (none)

My posted test image has:

Colorspace: CMYK
Compression: LZW
ICC-profile: Coated FOGRA27 (ISO 12647-2:2004) [557188 bytes!!!]

Unfortunately these kinds of images from Photoshop seem to be all
the rage these days, and if I tell people we can't handle them, they're
not impressed.

Thanks again

Crin

@mrjj Ah, interesting, the cmyk/rgb issue fits with what we've found. We opened and saved test.tif using gimp, it saved as an RGBA tiff and that displays fine (with transparency) in Qt. Opening it in photoshop vs 3, it displays as transparent in photoshop, (re)saving out as tiff, opting to keep transparency with LZW compression, the new image still shows as opaque in Qt...

mrjj

well actually I also had strange experience.
I took his gimp image and pasted ur image over it and saved.
Then transparent was lost.
I then tried to flipflop all TIFF option in PS but i never got it to work
again.

So it seems that Qt dont like some PS Tiff images with regards of keeping the alpha.

VRonin

Even Gimp, that is written in Gtk+ so nothing to do with Qt and being specialised image manipulator fails to load the transparency from your file. Photoscape, Inkscape and Microsoft Paint fail to load the image altogether.

I'm not rich enough to own Photoshop to test it but given 4 of the most popular image manipulator programs can't load the transparency I can't rule out the problem being only with the file.

Could you provide another file that presents the same issue? Was test.tif created on Mac?

mrjj

@VRonin
So most likely its the lib used by the tiff plugin that dont agree with PS tiff images.
Those other linux tools might use same lib?

Fun thing is that his image display fine in PS and other windows image viewers.

So yes, i agree, more test images would be great :)
and thx for helping testing!

concreteRick

@mrjj

Those other linux tools might use same lib?

I think that's likely a very good point--tho' not limited to
Linux tools. Even some Adobe products have problems reading some
Photoshop TIFF files (e.g. FrameMaker and maybe Fireworks).

Here is some background that may be relevant and helpful. This is
partially from memory, rather than based on solid evidence. So
corrections are very welcome.

The TIFF standard was initially developed and published by Aldus
and then Adobe. It may now be an ISO standard, ISO 12639:2004. See:

http://www.iso.org/iso/home/store/catalogue_ics/catalogue_detail_ics.htm?csnumber=34342

In the late 1980's early 1990's Sam Leffler of BSD and other fame
developed the brilliant libTIFF library in C and made the source code
freely available for any use whatsoever. Following that, a handy set
of 'TIFF Tools' was released, using libTiff. TIFF Tools was much used
on Unix systems, esp Sun and SGI machines. Development of libTIFF
was continued by a new group. See:

http://www.remotesensing.org/libtiff/

I'm fairly sure that versions of the libTIFF libraries are used in
many applications, on Windows, Mac, in open source and in closed source.

In 2002 Adobe Systems published a document called "Adobe Photoshop
TIFF Technical Notes", which explains the special use of tag number 37724.
See:

http://partners.adobe.com/public/developer/en/tiff/TIFFphotoshop.pdf

AFAIK the TIFF standard allows Adobe, and anyone else, to define
special TIFF tags. So the warning issued by some libraries that tag
37724 is 'unknown' might indicate one of the following:

The library was developed before 2002
The developers were unaware of the above Adobe document
The developers did not have time for or interest in supporting
several variants of TIFF files saved from Photoshop

In, or before 2006, Adobe open-sourced it's C++ Generic Image Library (GIL)
The GIL is now incorporated into Boost. See:

https://sourceforge.net/adobe/genimglib/home/Home/

Coming back to the original post, and follow ups, as I read them,
s/he is saying:

GIL seems to be able to handle Photoshop TIFF of the example type
supplied (test.tif, i.e. containing tag 37724 and transparency).
S/he has tried it, and used it to extract various color, alpha
and ICC profile streams, put them into the appropriate PDF dictionaries,
and the resulting PDF file shows the transparency correctly on-screen.
QImage::load(test.tif) does not handle it correctly (transparency
is lost) [There is no 'fault' in the test.tif file--it's just a PS
TIFF file using the documented 37724 tag that includes 8BIM
profile, inter alia.]

And then s/he is asking if anyone has any suggestions about how, e.g.
GIL may be used to overcome this issue with QImage::load() as a, maybe
very complicated, workaround: viz.

Using GIL we are able to split the uncompressed alpha and CMYK channel
data out from the CMYK into std::vector<unsigned char> variables, but
I have no idea how to map/convert this into a QImage structure which
displays correctly ...?

So "how to map/convert this into a QImage structure which displays
correctly" is the real question.

The problem, I guess, is that a huge number of professionals use Photoshop
and s/he needs to keep them happy.

If s/he finds a workaround to this issue, it could be a help to many
Qt users.

/Rick

PS. This:

https://bugs.scribus.net/print_bug_page.php?bug_id=2971

might imply that Scribus has overcome this issue. But I'm not sure.