Sandbox app for the Mac App Store with Qt 5.8 and QtWebEngineProcess

Wolosocu

@SGaist said in Sandbox app for the Mac App Store with Qt 5.8 and QtWebEngineProcess:

Hi,

The QtWebEngine macOS platform notes contains the key to your trouble.

This looks promising. My only question is, how do I specify that option when building Qt? If I look here (http://doc.qt.io/qt-5/osx-building.html) I see no mention of running qmake. Should I just run that qmake-command after configure and before make?

SGaist

You don't need to rebuild the whole Qt, just the module itself. That's why qmake is mentioned in the notes.

Wolosocu

@SGaist said in Sandbox app for the Mac App Store with Qt 5.8 and QtWebEngineProcess:

You don't need to rebuild the whole Qt, just the module itself. That's why qmake is mentioned in the notes.

Ok, so then:

./configure
./qtbase/bin/qmake WEBENGINE_CONFIG+=use_appstore_compliant_code
make module-qtwebengine

Sound about right?

Wolosocu

I am still unable to get it running. Here's what I did:

Last night I downloaded the Qt source and run the following commands in order:

./configure (and accepted all the license stuff)
./qtbase/bin/qmake WEBENGINE_CONFIG+=use_appstore_compliant_code (using the qmake that ./configure must've built)
make --jobs=16 (this was before I saw that I only need to rebuild the QtWebEngine module)

This ran overnight. I woke up this morning and this completed overnight. Great, so then I:

sudo make install

And now I have Qt in /usr/local/Qt5.8.0. Perfect. So I reset my CMake paths (I'm using CMake), generate the Xcode project and verify through Xcode's settings that I'm linking this new Qt build (which I am). Excellent., so I build the .app.

It gets built, no problem. Then I run my packaging script which does the following:

chdir into macdeployqt's folder, which may or not may not be necessary some people on Stackoverflow mentioned they were having trouble without doing this, so I make sure to do it.
Run macdeployqt <fullpath>/MyApp.app -appstore-compliant -qmldir=<my-projects-qml-folder> (Making sure to use the macdeployqt that I just built.)
chdir back to where I was before
Run /usr/bin/codesign --deep --sign <signature> --entitlements <entitlements-file> MyApp.app

Code signing reports no errors and runs fine. I launch the app and I see the same results. And Console reports the same thing:

4/23/17 09:00:49.103 sandboxd[135]: ([14028]) QtWebEngineProce(14028) deny forbidden-sandbox-reinit
4/23/17 09:00:49.122 sandboxd[135]: ([14029]) QtWebEngineProce(14029) deny forbidden-sandbox-reinit

So I tried inserting a step before the last one:

/usr/bin/codesign --deep --sign <signature> --entitlements <web-process-entitlements-file> <path-to>/QtWebEngineProcess.app

But still, the same problem persists. Suggestions?

Wolosocu

I think my problem is that I'm not setting up the config right. If I run the following:

./qtbase/bin/qmake WEBENGINE_CONFIG+=no_spellcheck QtWebEngine

I see:

Info: creating cache file /Users/Me/Downloads/qt-everywhere-opensource-src-5.8.0/.qmake.cache

Running configuration tests...
Done running configuration tests.

Configure summary:

Qt WebEngine:
  Proprietary Codecs ..................... no
  Spellchecker ........................... yes
  ALSA ................................... no
  PulseAudio ............................. no

Running configure tests
Checking for alsa... no
Checking for khr... no
Checking for libvpx... no
Checking for snappy... no
Checking for srtp... no
Configurable features:
  Proprietary codecs (H264, MP3) ..... Not enabled         (Default, enable with WEBENGINE_CONFIG+=use_proprietary_codecs)
  Geolocation ........................ Enabled
  Mac App Store Compliant ............ Not enabled         (Default, enable with WEBENGINE_CONFIG+=use_appstore_compliant_code)
Native Spellchecker .............. Not enabled         (Default, enable with WEBENGINE_CONFIG+=use_native_spellchecker)

And I see the spellchecker is still enabled. So yeah, I need to figure out how to set this config item and then build.

Wolosocu

I deleted the .qmake.cache file in my Qt source folder and then ran:

./qtbase/bin/qmake WEBENGINE_CONFIG+=use_appstore_compliant_code QtWebEngine

And I saw the output I was hoping to see:

Info: creating cache file /Users/Me/Downloads/qt-everywhere-opensource-src-5.8.0/.qmake.cache

Running configuration tests...
Done running configuration tests.

Configure summary:

Qt WebEngine:
  Proprietary Codecs ..................... no
  Spellchecker ........................... yes
  ALSA ................................... no
  PulseAudio ............................. no

Running configure tests
Checking for alsa... no
Checking for khr... no
Checking for libvpx... no
Checking for snappy... no
Checking for srtp... no
Configurable features:
  Proprietary codecs (H264, MP3) ..... Not enabled         (Default, enable with WEBENGINE_CONFIG+=use_proprietary_codecs)
  Geolocation ........................ Enabled
  Mac App Store Compliant ............ Enabled
Native Spellchecker .............. Not enabled         (Default, enable with WEBENGINE_CONFIG+=use_native_spellchecker)

With the most important part being Mac App Store Compliant ............ Enabled. I tried then running make module-qtwebengine but it returned an error saying there is no such make target. So, I went ahead and ran make and it looks like it's just rebuiling QtWebEngine. Once it's done I'll try it out. (Keeping my fingers crossed).

Wolosocu

And now I got the following compiler error:

../../../qtwebengine/src/webengine/plugin/plugin.cpp:140:10: fatal error: 'plugin.moc' file not found
#include "plugin.moc"
         ^
1 error generated.
make[3]: *** [.obj/debug/plugin.o] Error 1
make[2]: *** [debug-all] Error 2
make[1]: *** [sub-webengine-plugin-make_first] Error 2
make: *** [sub-src-make_first] Error 2

For giggles I've make clean and am trying make again. Guess I'll know in a couple hours.

Wolosocu

make finished quicker than I thought. As far as I can tell QtWebEngine we rebuilt and I see the config option being passed around all over the build process. For example a snippet:

( test -e Makefile.gyp_configure_host || /Users/Me/Downloads/qt-everywhere-opensource-src-5.8.0/qtbase/bin/qmake -o Makefile.gyp_configure_host /Users/Me/Downloads/qt-everywhere-opensource-src-5.8.0/qtwebengine/src/core/gyp_configure_host.pro WEBENGINE_CONFIG+=use_appstore_compliant_code ) && /Applications/Xcode.app/Contents/Developer/usr/bin/make -f Makefile.gyp_configure_host 
make[4]: Nothing to be done for `first'.

But yet the problem persists:

4/23/17 11:03:34.000 kernel[0]: Sandbox: QtWebEngineProce(30098) deny(1) forbidden-sandbox-reinit

I'm still not convinced I'm building QtWebEngine how I should be Google'ing is not being helpful.

What am I doing wrong?

Wolosocu

Ok, I think I figured out that qmake QtWebEngine will create makefiles for the QtWebEngine module only. But even when I build the makefiles using qmake WEBENGINE_CONFIG+=use_appstore_compliant_code QtWebEngine and then build the module, I still see the QtWebEngineProcess.app in the QtWebEngineCode.framework. Is this expected? If so, then how it should be signed? How should my bundle in general be signed?

SGaist

There's no need to configure Qt, you can use the qmake from your pre-built package if you'd rather use that one.

macdeployqt has two options that will be of interest:

codesign=<ident> : Run codesign with the given identity on all executables
appstore-compliant: Skip deployment of components that use private API

On a side, you're likely in possession of a multi-core machine, to speed things up when building you can do make -jX where X is (2 * number_fo_core) + 1,

Wolosocu

@SGaist So I should use macdeployqt to sign the bundle instead of doing it manually? I will try that.

Also, will QtWebEngineProcess.app still be included in the version of QtWebEngineCore built with the use_appstore_compliant_code?

Wolosocu

And if I'm using macdeployqt to sign the bundle, then how do I specify the entitlements file?

SGaist

The switch doesn't remove the build of the helper application, it disable some code that is using APIs not authorised in App Store applications.

Wolosocu

@SGaist Thank you.

I still do not understand how using macdeployqt to sign the bundle will apply an entitlements file for sandboxing, though.

Wolosocu

I am still unable to get this to work. I am seeing a new error on Console, though:

4/23/17 17:20:31.148 QtWebEngineProcess[2551]: [0423/172031:ERROR:mach_port_broker.mm(43)] bootstrap_look_up: Permission denied (1100)

I have three steps I am toying with:

Run macdeployqt with -codesign as suggested.
Manually codesign QtWebProcessEngine.app, with the entitlements file from above.
Manually codesign MyApp.app bundle, with my app's entitlements file.

I have tried various combinations, such as just #1, or #1 and #3. I've tried switching the order of #2 and #3. I've even tried signing QtWebProcessEngine.app with my bundle identifier.

I am running out of ideas. Please help!

Wolosocu

@SGaist said in Sandbox app for the Mac App Store with Qt 5.8 and QtWebEngineProcess:

The switch doesn't remove the build of the helper application, it disable some code that is using APIs not authorised in App Store applications.

Do you know if these API were present in 5.7? My next thought is to try this with 5.7 since I am having so many problems with 5.8.

Wolosocu

I was finally able to get my app to run in sandbox mode with Qt 5.6.2. I have not tried 5.7 yet, but I did have to reduce the version of QtQuick I was using from 2.7 to 2.6. Luckily I am not using any 2.7 specific features.

I was also able to get my app past the Application Loader portion of the app submission. It hasn't been approved yet but we'll see.

I wanted to mention a few things for anyone else that might have a problem with this.

macdeployqt's -codesign option did NOT work. The only way I was able to sign my app correctly was to write a script that manually walked through my app bundle and called codesign on every binary with the entitlements I wanted/needed.
macdeployqt also seems to create a lot of *.dSYM files. This is something I noticed in with 5.8 as well and Apple's Application Loader will reject these. I didn't pay much attention to the error but I just made sure my packaging script deleted them (before signing).
QtWebProcessEngine.app 's Info.plist file needs to be changed before you sign it. The CFBundleIdentifier needs to match that of your signature.

Like I said, my app has only passed Application Loader's loading process. I've had problems in the past with Qt and the App Store system (see http://stackoverflow.com/questions/25694066/submitting-qt-5-3-app-for-review-on-mac-app-store) before I could even submit the app for review, so I'm skeptical that I'm out of the woods yet.

I'll keep this thread updated.

Wolosocu

Attempting Qt 5.7.1 brought its own problem. Namely that on startup the application gives the error:

This application failed to start because it could not find or load the Qt platform plugin "cocoa"
in "".

Available platform plugins are: cocoa.

Reinstalling the application may fix this problem.
Abort trap: 6

I did a little research and tried a couple minor things but nothing worked.

Meanwhile my app was rejected because of an unused sandbox permission. So I've corrected that and resubmitted a new build still using Qt 5.6.2.

Really wish I had been able to get 5.8 to work or even 5.7.

SGaist

That strange, do you have a qt.conf file in your application bundle ?

Wolosocu

@SGaist Yes.

So, my app was rejected using Qt 5.6.2 because of the following:

Your app uses or references the following non-public API(s):

framework: '/System/Library/Frameworks/AppKit.framework/Versions/C/AppKit'
: NSAccessibilityUnregisterUniqueIdForUIElement
: _NSAppendToKillRing
: _NSDrawCarbonThemeBezel
: _NSDrawCarbonThemeListBox
: _NSInitializeKillRing
: _NSNewKillRingSequence
: _NSPrependToKillRing
: _NSSetKillRingToYankedState
: _NSYankFromKillRing

framework: '/System/Library/Frameworks/ApplicationServices.framework/Versions/A/ApplicationServices'
: CGSSetDenyWindowServerConnections
: CGSShutdownServerConnections
: CTFontCopyDefaultCascadeList

The use of non-public APIs is not permitted on the App Store as it can lead to a poor user experience should these APIs change.

Which looks like this is addressed by this: https://doc.qt.io/qt-5/qtwebengine-platform-notes.html#mac-app-store-compatibility

So I tried Qt 5.7 again and for whatever reason it didn't crash this time! However, it has the same exact problem I was seeing with Qt 5.8.

I have built and rebuilt and rebuilt QtWebEngine and yet I still cannot get it to work. Is there anyway to verify that my build of QtWebEngine is actually applying the use_appstore_compliant_code flag?

It seems like this has been a problem for a while: https://bugreports.qt.io/browse/QTBUG-41611