performance of large image, that is mostly transparent
-
Using QML Image on embedded i.MX 8.
Does Qt or e.g. OpenGL detect when an image with lots of transparency could be cropped, without loss, and optimize for that?In figma/photoshop/etc, we draw gauges with different layers for the face, bezel, needle etc.
The bezel is naturally exported as a square png.
The needle could either be cropped to a rectangle for export, or exported within a square too, with lots of transparency.If the needle is exported without cropping all we need to do in QML is rotate around the center of the image. It "just works".
If instead we crop the needle to a rectangle, then the graphic designer and the developer need to be explicit about the magic number for Rotation origin.y, and carry that metadata around with the image somehow.
example:
KDAB Introduction to Qt / QML (Part 09) - Custom Transformations
https://www.youtube.com/watch?v=W_umKXqUL_U&t=112sBy the way, is there any standard way of associating the rotation origin metadata with each needle image?
Or if we simply export the needle without cropping - is there any ongoing performance penalty when drawing, due to all the extra transparent pixels. Or does Qt or OpenGL automatically crop/optimize those away anyway?
Thank you
-- Peter -
Normally if you work with multimedia such as games etc you have a content/asset packaging step.
In other words the content you get from the artists and their DCC tools is often not optimal for the engine to run so the packaging step performs not only packaging but also often optimization and other such steps such as
- texture resampling and combining
- audio resampling
- pre-computing content such as light maps, shadow maps
- etc
So to answer your question.
You'd need a step / tooling to produce the metadata and crop your data coming from the digital designers.
You then record the information that you need to work with the final asset in your runtime environment. In this case that would be your pivot point.
If your artist produce a square png for the needle for you gauge you already know the pivot point, the center and when you crop you can map the previous center to a point in the cropped image.
"is there any ongoing performance penalty when drawing, due to all the extra transparent pixels. Or does Qt or OpenGL automatically crop/optimize those away anyway?"
No, there's no automatic cropping.
Yes, there's a performance penalty.
- Increased memory usage for storage on the GPU
- Increased file size on disk
- Wasted computation on things such as mip map computation
- Wasted computation on blending transparent pixels
Does it matter? Depends how resource constrained you are. Generally speaking not cropping the incoming images is rather sloppy programming but understandable in some ad-hoc system without proper asset creation pipeline.
Free bonus tip:
If you're using OpenGL to draw and you are in fact scaling your content when drawing you probably want pre-multiplied alpha.
-
@SamiV123
Thank you for the comprehensive reply.
Yes, turns out even on disk, the uncropped png is noticeably larger:
39 KB for 480x480
29 KB for 163x284
Seems even png is not designed to detect that it could crop (and offset) most of the image without any loss.If instead, users have to record off-center pivot points explicitly, not sure that their tools make that easy and foolproof for them?
Even Figma, lacks this natively, and needs a rotate-origin plugin:
https://forum.figma.com/t/specify-center-of-rotation/1839/7Sounds like a common problem. But seems that Qt itself doesn't provide any specific advice or tooling for this scenario and workflow?
So sounds like the most friction-less workflow would be for us to create an in-house tool that takes images, crops them, and records the resulting offset.
Less than a day's work, but if anyone knows of an open source tool that does that, let me know?
-
@petero3 said in performance of large image, that is mostly transparent:
@SamiV123
Thank you for the comprehensive reply.
Yes, turns out even on disk, the uncropped png is noticeably larger:
39 KB for 480x480
29 KB for 163x284
Seems even png is not designed to detect that it could crop (and offset) most of the image without any loss.If instead, users have to record off-center pivot points explicitly, not sure that their tools make that easy and foolproof for them?
Even Figma, lacks this natively, and needs a rotate-origin plugin:
https://forum.figma.com/t/specify-center-of-rotation/1839/7Sounds like a common problem. But seems that Qt itself doesn't provide any specific advice or tooling for this scenario and workflow?
So sounds like the most friction-less workflow would be for us to create an in-house tool that takes images, crops them, and records the resulting offset.
Less than a day's work, but if anyone knows of an open source tool that does that, let me know?
If instead, users have to record off-center pivot points explicitly, not sure that their tools make that easy and foolproof for them?
Such a tool is not for the end users, it's for you basically. Assuming you're the programmer here, you get your content from the artist(s), apply your tool and package it etc for deployment as a product which is then used by the end user.
Sounds like a common problem. But seems that Qt itself doesn't provide any specific advice or tooling for this scenario and workflow?
Well no, Qt is a domain agnostic application development toolkit for the most part. This problem is pretty much application/domain/use-case specific. The solution (like you hinted) is really in higher level (the application) or a lower level (image compression).
Keep in mind that a general solution for cropping fully transparent pixels needs to ultimately record the information of pixel being transparent for every transparent pixel and you'd only get some compression benefit by combining large adjacent blocks into some kind of lookup table which you'd have to then consult then sampling to see if the pixel was discarded since it was transparent. This would mean that every pixel lookup would then have to check those "transparency blocks" before looking for the "non transparent pixel value".
This really is an image format/compression thing and it seems that it's just not worth it.Assuming that some .png file only contains a single shape that fits neatly inside a single rectangle is very domain and use-case specific.
So sounds like the most friction-less workflow would be for us to create an in-house tool that takes images, crops them, and records the resulting offset.
Yes, assuming the longevity of the project. If you plan to enhance and maintain this longer etc. then some tooling is the way to go. If this one off you can get by just by simply manually putting some JSON together or writing some crappy Python script etc.
Less than a day's work, but if anyone knows of an open source tool that does that, let me know?
Heh, yeah.. no.. really in any large multimedia project (games, movies) when you're setting up the production environment producing the tooling is the largest part of the work and any game studio working on their own engine has hordes of tools programmers to build all the content pipelines. So if you start this path and the project / product deliverables need to be maintained and further enhanced expect to spend a lot of time cranking out the tooling as the scope grows ;-)
Again free bonus tip:
You already hinted about possible solution by mentioning figma plugin, it's possible that in your use case all you need to do is to create some plugin/scripting for an existing tool such as Figma or Blender etc. So you don't necessarily need to write your own app from scratch.
-
If instead, users have to record off-center pivot points explicitly, not sure that their tools make that easy and foolproof for them?
Such a tool is not for the end users, it's for you basically. Assuming you're the programmer here, you get your content from the artist(s), apply your tool and package it etc for deployment as a product which is then used by the end user.
Yes, initially, our base application would have me, the developer, running the cropping tool on what our graphic designer gives me.
But ultimately, our customers/end users also want to be able to create their own custom skins for our dashboards/gauges themselves.
We can either:
A)
tell them to create images in which the pivot point of the needle is in the center of the image, and accept that technically performance (and disk/memory usage is sub-optimal). orB)
we can explain to them how to crop the image, and measure the new offset of the pivot point, and record that in the metadata or filename. But I guess their tools (Figma, Photoshop etc) do not help make that workflow as foolproof and user friendly as A.C)
provide them with a command line tool to automate the cropping and recording of the new offset. (I'd run it on internal assets from our graphic designer, but customers would have to run it themselves on their own custom skins).D)
make our application seamlessly manage custom assets and do the cropping that the command line tool would do itself -
Your customer deployment / product package strategy would depend on what is the expected technical expertise level of the person using your product.
For more non-technical people it'd probably be best to have a graphical tool that allows them easily to select images for their skins and set properties with simple UX such as mouse clicks in the image etc.
If we're talking about a digital dashboard (for example for a vehicle) for a really nice UX I think you'd have a visual tool that
- shows the layout of the dashboard, wireframe, outline etcs, the components that make up the final visual
- allows to move things around, drag & drop, hide show
- allows to adjust properties of things
- allows to change the visual appeareance easily by dragging and dropping images. etc.
- thousand other features
But this is then basically a full blown "editor" similar to https://rightware.com/
b) only works for the engineering types, technical people (programmers or technical artists) who can read and follow written rules.
c) could work if you have simple expected output formats for your input data and the script is just simple "do-it.py" which then spits out what you need, i.e. the running is trivial.
d) just means putting the tooling inside your deployed executable, it works too but in general is not ideal, plus you still need the meta information about what part to crop unless you just hard code the solution for all the problems inside it.
-