Packaging for Mac: macOS's tar seems to mess up binaries?!
-
Hi all,
I provide a Mac App package for one of my projects (although I'm really not a Mac user …). Currently, I build on macOS 10.14 "Mojave", using a static build of Qt 5.15.16 and qmake. This yields an .app folder I create a .tar.xz archive from (to get better compression). This seemed to work fine for the last few years.
However, I now played around with Qt 6 (Qt 5 won't be around forever ;-). Using macOS 12.7.4 "Monterey", I could could build a static version of Qt 6.7.3, as well as my project. But after unpacking the xz tarball my release script creates, I could not start the program: The icon was overlayed with some "stop" sign and when starting it, I get an error saying "This program can't be started because it is not supported on this Mac" or such.
I first thought this must be due to my Qt 6 build, but apparently, it's tar:
I can run the built app folder without a problem. But as soon as it's tared (no matter if it's compressed using xz or not), the binary is altered after it's unpacked again:
Here's the original app folder's binary:
% ls -l total 29856 -rwxr-xr-x 1 tobias staff 21822704 19 Nov 22:29 Muckturnier % file Muckturnier Muckturnier: Mach-O 64-bit executable x86_64 % md5 Muckturnier MD5 (Muckturnier) = 8883bfcd40d39d5b3459490dfe720be9And this is what comes out when a tarball is created from the app folder and it's unpacked again:
% ls -l total 29856 -rwxr-xr-x 1 tobias staff 21822704 19 Nov 22:29 Muckturnier % file Muckturnier Muckturnier: data % md5 Muckturnier MD5 (Muckturnier) = f57e510f88bce9d1fed0e0295104fea7The byte size is still the same, but somehow, the file itself seems to be messed up!
What on earth happens here?! Can somebody with more insight of that Mac stuff give me a hint? Thanks for all help!
-
L l3u_ deleted this topic on
-
L l3u_ restored this topic on
-
When I scp the .app folder to my Linux host, create the tarball there and scp it back to macOS, I can unpack and use it …
Does this multi-billion-dollar company really package a broken tar command and doesn't fix it?!
-
Hi,
Taking aside the tar issue, macOS rather uses dmg files for distribution. Qt provides macdeployqt which builds such things and that could simplify your life a bit.
-
Yeah I know … however, AFAIK before macOS 10.15 "Catalina", decent compression (LZMA) for DMG images wasn't available.
The oldest version I could use to build Qt 5 back then was macOS 10.14 "Mojave", which would only create ZIP compressed DMG files that were way bigger than a tar.xz archive. However, I could unpack such an archive there by simply double clicking on it … I thus thought it would be okay to save space on my server, backups etc. and of course traffic by using tar.xz.
With macOS 12.7.4 "Monterey", I can pack my stuff using
hdiutil create -format ULMO -srcfolder some_folder project-xyz.dmgThe result is still a bit larger than the same data compressed via xz, but it's way better than the old version. However, such a DMG image can't be opened using "Mojave". And I'm not sure if I can expect everybody to have at least "Catalina" – I really have no idea what those Apple folks use ;-)
@SGaist So: Asking the other way round: Could I be relatively sure to reach the majority of Mac users when I create an LZMA compressed DMG image?
-
I just realized something: Qt follows the same policies as Apple for backward compatibility. This means the current version of macOS and the two previous. So there might also be that to take into account for your own supported versions of macOS.
Hence postpone a bit the retiring of Qt 5 for the people using older versions of macOS that you still would like to support.
It's not unseen to have two versions (sometimes even more) versions of an application to support various flavors of macOS. Think PPC to x86_64 migration and now x86_64 to ARM migration.
-
Thanks for the input! I wanted to keep Qt 5 for my project as long as possible anyway, because with Qt 6, I can't support Windows 7 anymore (and heaven knows how many people still use that on their old notebooks – and old notebooks are what are used for this very project).
They do migrate to ARM now?! Oh my, I think I have to dive a bit into this … I literally have nothing to do with all that Apple stuff, left aside I build this project on Mac …
-
Not exactly now, we are currently at the fourth iteration of their ARM processors for desktop and laptop machines.
-
So … would you say that I'm at least safe for now if I create such "universal" binaries for x86_64 and ARM (due to the cost that they take almost double the space) for now?!
I'm a bit concerned about the sheer (double) space such a release would take and I'm wondering if it's worth it …
-
I still have one Mac running Mojave. Visual Studio Code just told me that they are fading out support (newer versions will not be able to install). Microsoft Office also does not provide updates for Mojave anymore. It's nice if you still support Mojave, but it is certainly not a requirement anymore.
The transition to ARM is a few years now. You don't necessarily have to provide universal binaries. For now, there is still Rosetta to translate x86_64 binaries to ARM on the fly (your costumers won't notice it unless it is the first time they are using Rosetta; first time it will show a dialog to install). Performance with Rosetta is exceptionally good (the first Apple ARM processor was as fast as the then-current Intel 12900KS with Rosetta in our tests).
Far more concerning is that you have to properly sign and distribute your app. If you are doing it through the App Store it should work. But otherwise it is really hard to run an app not properly signed on the most current version of macOS (you have to allow that specific app somewhere deep in the system settings).
-
@SimonSchroeder The signing issue depends on the architecture. x86_64 macOS is much more forgiving with unsigned apps. The situation changes drastically on arm. There is more or less the requirement to have the app properly signed with an apple account.
@l3u_ If it were my app I would stay on x86 and let them use Rosetta. I tried your app on my M4 Arm MacBook and it worked flawlessly following your signing workaround. This will change with a native app. -
Oh, thanks a lot for this input! Then I think I'll stay with x86_64 for now. @DerReisende thanks for the insider info ;-)
@SimonSchroeder The problem with Mojave is that it can't handle LZMA compressed DMG files. I first simply created a .tar.xz archive from the app dir, because packing the app in a DMG image resulted in a way bigger archive. One could simply unpack the tar.xz file and run the app (maybe allowing it to be executed first).
Then I learned about Apple users to be expecting DMG images with a drag and drop interface to install an app. And I learned about DMG images with LZMA compression.
The current status is that I can create an LZMA compressed DMG image for my app, with a nice drag and drop "installer". On Catalina. Mojave can't decompress the image. As far as I can grasp it the app itself could run on Mojave though …
Maybe there's a way to get such an image decompressed on Mojave, and I just didn't find it?! Or: Should one still care about this, or is your case a special one and the big majority of Apple users don't use Mojave anymore anyway?!
-
Yeah, and about the signing: I know that it's more and more difficult. But to properly sign a Mac app, I think you need an Apple Developer Account. And you won't get it for free. Correct me if I'm wrong, but they charge 99$/year for that. I don't earn a single cent with this app. I'm not willing to pay Apple so that I can sign the app in a format they like … I mean I do sign it with my PGP key …
-
Oh, thanks a lot for this input! Then I think I'll stay with x86_64 for now. @DerReisende thanks for the insider info ;-)
@SimonSchroeder The problem with Mojave is that it can't handle LZMA compressed DMG files. I first simply created a .tar.xz archive from the app dir, because packing the app in a DMG image resulted in a way bigger archive. One could simply unpack the tar.xz file and run the app (maybe allowing it to be executed first).
Then I learned about Apple users to be expecting DMG images with a drag and drop interface to install an app. And I learned about DMG images with LZMA compression.
The current status is that I can create an LZMA compressed DMG image for my app, with a nice drag and drop "installer". On Catalina. Mojave can't decompress the image. As far as I can grasp it the app itself could run on Mojave though …
Maybe there's a way to get such an image decompressed on Mojave, and I just didn't find it?! Or: Should one still care about this, or is your case a special one and the big majority of Apple users don't use Mojave anymore anyway?!
@l3u_ said in Packaging for Mac: macOS's tar seems to mess up binaries?!:
Or: Should one still care about this, or is your case a special one and the big majority of Apple users don't use Mojave anymore anyway?!
That is what I was saying: There is enough (important) software out there which will not update on Mojave anymore. Users are heavily pushed to update to a newer version of macOS. It is therefore reasonable enough to drop support for Mojave. Mojave has been last updated in 2021, so Apple dropped support a long time ago.
