Sandbox app for the Mac App Store with Qt 5.8 and QtWebEngineProcess



  • I have run into the same problem that a few others have run into however their fixes are either ambiguous or did not help me.

    My application uses QtWebEngine. I run macdeployqt on my app and all is well. However, once I sign the .app bundel, the app loads and everything EXCEPT the QWebEngine portions look fine. The logs say what the problem is pretty clearly:

    QtWebEngineProce(1818) deny forbidden-sandbox-reinit

    And the process never starts. So obviously this has something to do with how I am signing the everything.

    My build process is pretty simple:

    1. I build my Release build (MyProgram.app) through Xcode, with no signing.
    2. I run the resulting bundle through macdeployqt. This puts the QtWebEngineProcess into my app's bundle, as well as many other files.
    3. Next, I sign that QtWebEngineProcess specifically with these QtWebEngineProcess-specific entitlements:
    <?xml version="1.0" encoding="UTF-8"?>
    <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
    <plist version="1.0">
    <dict>
    	<key>com.apple.security.app-sandbox</key>
    	<true/>
    	<key>com.apple.security.inherit</key>
    	<true/>
    </dict>
    </plist>
    

    My command looks like this (Python code)

    subprocess.call([r'/usr/bin/codesign', 
        r'--deep',
        r'--verify',
        r'--verbose',
        r'--sign',
        appSignature,	
        r'-i',
        r'com.yourcompany.QtWebEngineProcess',
        r'--entitlements',
        webProcessEntitlementsFile,
        bundleFolder + '/MyProgram.app/Contents/Frameworks/QtWebEngineCore.framework/Versions/5/Helpers/QtWebEngineProcess.app/Contents/MacOS/QtWebEngineProcess'
    
    1. Then I the only other thing I sign is my app's main bundle:
    subprocess.call([r'/usr/bin/codesign', 
        r'--deep',
        r'--force',
        r'--verify',
        r'--verbose',
        r'--sign',
        appSignature,	
        r'--entitlements',
        entitlements,
        bundleFolder + '/MyProgram.app'
    ])
    

    Signing goes through without an error. Yet, when I run the program my C++ and Qml widgets all render perfectly. The QWebEngine based control never renders anything. Presumably because QtWebEngineProcess fails to run. Which I can see in Console as:

    4/21/17 23:55:06.020 sandboxd[135]: ([4543]) QtWebEngineProce(4543) deny forbidden-sandbox-reinit
    4/21/17 23:55:06.023 sandboxd[135]: ([4544]) QtWebEngineProce(4544) deny forbidden-sandbox-reinit
    

    How exactly should everything in the package be signed? Is doing one signing of MyProgram.app ok? Do I need to sign everything individually? Maybe certain frameworks need different entitlements? I don't know. Any help would be appreciated!

    Is there any documentation or psuedo code for writing these build/signing procedures?

    Thank you!


  • Lifetime Qt Champion

    Hi,

    The QtWebEngine macOS platform notes contains the key to your trouble.



  • @SGaist said in Sandbox app for the Mac App Store with Qt 5.8 and QtWebEngineProcess:

    Hi,

    The QtWebEngine macOS platform notes contains the key to your trouble.

    This looks promising. My only question is, how do I specify that option when building Qt? If I look here (http://doc.qt.io/qt-5/osx-building.html) I see no mention of running qmake. Should I just run that qmake-command after configure and before make?


  • Lifetime Qt Champion

    You don't need to rebuild the whole Qt, just the module itself. That's why qmake is mentioned in the notes.



  • @SGaist said in Sandbox app for the Mac App Store with Qt 5.8 and QtWebEngineProcess:

    You don't need to rebuild the whole Qt, just the module itself. That's why qmake is mentioned in the notes.

    Ok, so then:

    1. ./configure
    2. ./qtbase/bin/qmake WEBENGINE_CONFIG+=use_appstore_compliant_code
    3. make module-qtwebengine

    Sound about right?



  • I am still unable to get it running. Here's what I did:

    Last night I downloaded the Qt source and run the following commands in order:

    1. ./configure (and accepted all the license stuff)
    2. ./qtbase/bin/qmake WEBENGINE_CONFIG+=use_appstore_compliant_code (using the qmake that ./configure must've built)
    3. make --jobs=16 (this was before I saw that I only need to rebuild the QtWebEngine module)

    This ran overnight. I woke up this morning and this completed overnight. Great, so then I:

    1. sudo make install

    And now I have Qt in /usr/local/Qt5.8.0. Perfect. So I reset my CMake paths (I'm using CMake), generate the Xcode project and verify through Xcode's settings that I'm linking this new Qt build (which I am). Excellent., so I build the .app.

    It gets built, no problem. Then I run my packaging script which does the following:

    1. chdir into macdeployqt's folder, which may or not may not be necessary some people on Stackoverflow mentioned they were having trouble without doing this, so I make sure to do it.
    2. Run macdeployqt <fullpath>/MyApp.app -appstore-compliant -qmldir=<my-projects-qml-folder> (Making sure to use the macdeployqt that I just built.)
    3. chdir back to where I was before
    4. Run /usr/bin/codesign --deep --sign <signature> --entitlements <entitlements-file> MyApp.app

    Code signing reports no errors and runs fine. I launch the app and I see the same results. And Console reports the same thing:

    4/23/17 09:00:49.103 sandboxd[135]: ([14028]) QtWebEngineProce(14028) deny forbidden-sandbox-reinit
    4/23/17 09:00:49.122 sandboxd[135]: ([14029]) QtWebEngineProce(14029) deny forbidden-sandbox-reinit

    So I tried inserting a step before the last one:

    /usr/bin/codesign --deep --sign <signature> --entitlements <web-process-entitlements-file> <path-to>/QtWebEngineProcess.app

    But still, the same problem persists. Suggestions?



  • I think my problem is that I'm not setting up the config right. If I run the following:

    ./qtbase/bin/qmake WEBENGINE_CONFIG+=no_spellcheck QtWebEngine

    I see:

    Info: creating cache file /Users/Me/Downloads/qt-everywhere-opensource-src-5.8.0/.qmake.cache
    
    Running configuration tests...
    Done running configuration tests.
    
    Configure summary:
    
    Qt WebEngine:
      Proprietary Codecs ..................... no
      Spellchecker ........................... yes
      ALSA ................................... no
      PulseAudio ............................. no
    
    Running configure tests
    Checking for alsa... no
    Checking for khr... no
    Checking for libvpx... no
    Checking for snappy... no
    Checking for srtp... no
    Configurable features:
      Proprietary codecs (H264, MP3) ..... Not enabled         (Default, enable with WEBENGINE_CONFIG+=use_proprietary_codecs)
      Geolocation ........................ Enabled
      Mac App Store Compliant ............ Not enabled         (Default, enable with WEBENGINE_CONFIG+=use_appstore_compliant_code)
    Native Spellchecker .............. Not enabled         (Default, enable with WEBENGINE_CONFIG+=use_native_spellchecker)
    

    And I see the spellchecker is still enabled. So yeah, I need to figure out how to set this config item and then build.



  • I deleted the .qmake.cache file in my Qt source folder and then ran:

    ./qtbase/bin/qmake WEBENGINE_CONFIG+=use_appstore_compliant_code QtWebEngine

    And I saw the output I was hoping to see:

    Info: creating cache file /Users/Me/Downloads/qt-everywhere-opensource-src-5.8.0/.qmake.cache
    
    Running configuration tests...
    Done running configuration tests.
    
    Configure summary:
    
    Qt WebEngine:
      Proprietary Codecs ..................... no
      Spellchecker ........................... yes
      ALSA ................................... no
      PulseAudio ............................. no
    
    Running configure tests
    Checking for alsa... no
    Checking for khr... no
    Checking for libvpx... no
    Checking for snappy... no
    Checking for srtp... no
    Configurable features:
      Proprietary codecs (H264, MP3) ..... Not enabled         (Default, enable with WEBENGINE_CONFIG+=use_proprietary_codecs)
      Geolocation ........................ Enabled
      Mac App Store Compliant ............ Enabled
    Native Spellchecker .............. Not enabled         (Default, enable with WEBENGINE_CONFIG+=use_native_spellchecker)
    

    With the most important part being Mac App Store Compliant ............ Enabled. I tried then running make module-qtwebengine but it returned an error saying there is no such make target. So, I went ahead and ran make and it looks like it's just rebuiling QtWebEngine. Once it's done I'll try it out. (Keeping my fingers crossed).



  • And now I got the following compiler error:

    ../../../qtwebengine/src/webengine/plugin/plugin.cpp:140:10: fatal error: 'plugin.moc' file not found
    #include "plugin.moc"
             ^
    1 error generated.
    make[3]: *** [.obj/debug/plugin.o] Error 1
    make[2]: *** [debug-all] Error 2
    make[1]: *** [sub-webengine-plugin-make_first] Error 2
    make: *** [sub-src-make_first] Error 2
    

    For giggles I've make clean and am trying make again. Guess I'll know in a couple hours.



  • make finished quicker than I thought. As far as I can tell QtWebEngine we rebuilt and I see the config option being passed around all over the build process. For example a snippet:

    ( test -e Makefile.gyp_configure_host || /Users/Me/Downloads/qt-everywhere-opensource-src-5.8.0/qtbase/bin/qmake -o Makefile.gyp_configure_host /Users/Me/Downloads/qt-everywhere-opensource-src-5.8.0/qtwebengine/src/core/gyp_configure_host.pro WEBENGINE_CONFIG+=use_appstore_compliant_code ) && /Applications/Xcode.app/Contents/Developer/usr/bin/make -f Makefile.gyp_configure_host 
    make[4]: Nothing to be done for `first'.
    

    But yet the problem persists:

    4/23/17 11:03:34.000 kernel[0]: Sandbox: QtWebEngineProce(30098) deny(1) forbidden-sandbox-reinit

    I'm still not convinced I'm building QtWebEngine how I should be Google'ing is not being helpful.

    What am I doing wrong?



  • Ok, I think I figured out that qmake QtWebEngine will create makefiles for the QtWebEngine module only. But even when I build the makefiles using qmake WEBENGINE_CONFIG+=use_appstore_compliant_code QtWebEngine and then build the module, I still see the QtWebEngineProcess.app in the QtWebEngineCode.framework. Is this expected? If so, then how it should be signed? How should my bundle in general be signed?


  • Lifetime Qt Champion

    There's no need to configure Qt, you can use the qmake from your pre-built package if you'd rather use that one.

    macdeployqt has two options that will be of interest:

    • codesign=<ident> : Run codesign with the given identity on all executables
    • appstore-compliant: Skip deployment of components that use private API

    On a side, you're likely in possession of a multi-core machine, to speed things up when building you can do make -jX where X is (2 * number_fo_core) + 1,



  • @SGaist So I should use macdeployqt to sign the bundle instead of doing it manually? I will try that.

    Also, will QtWebEngineProcess.app still be included in the version of QtWebEngineCore built with the use_appstore_compliant_code?



  • And if I'm using macdeployqt to sign the bundle, then how do I specify the entitlements file?


  • Lifetime Qt Champion

    The switch doesn't remove the build of the helper application, it disable some code that is using APIs not authorised in App Store applications.



  • @SGaist Thank you.

    I still do not understand how using macdeployqt to sign the bundle will apply an entitlements file for sandboxing, though.



  • I am still unable to get this to work. I am seeing a new error on Console, though:

    4/23/17 17:20:31.148 QtWebEngineProcess[2551]: [0423/172031:ERROR:mach_port_broker.mm(43)] bootstrap_look_up: Permission denied (1100)

    I have three steps I am toying with:

    1. Run macdeployqt with -codesign as suggested.
    2. Manually codesign QtWebProcessEngine.app, with the entitlements file from above.
    3. Manually codesign MyApp.app bundle, with my app's entitlements file.

    I have tried various combinations, such as just #1, or #1 and #3. I've tried switching the order of #2 and #3. I've even tried signing QtWebProcessEngine.app with my bundle identifier.

    I am running out of ideas. Please help!



  • @SGaist said in Sandbox app for the Mac App Store with Qt 5.8 and QtWebEngineProcess:

    The switch doesn't remove the build of the helper application, it disable some code that is using APIs not authorised in App Store applications.

    Do you know if these API were present in 5.7? My next thought is to try this with 5.7 since I am having so many problems with 5.8.



  • I was finally able to get my app to run in sandbox mode with Qt 5.6.2. I have not tried 5.7 yet, but I did have to reduce the version of QtQuick I was using from 2.7 to 2.6. Luckily I am not using any 2.7 specific features.

    I was also able to get my app past the Application Loader portion of the app submission. It hasn't been approved yet but we'll see.

    I wanted to mention a few things for anyone else that might have a problem with this.

    1. macdeployqt's -codesign option did NOT work. The only way I was able to sign my app correctly was to write a script that manually walked through my app bundle and called codesign on every binary with the entitlements I wanted/needed.

    2. macdeployqt also seems to create a lot of *.dSYM files. This is something I noticed in with 5.8 as well and Apple's Application Loader will reject these. I didn't pay much attention to the error but I just made sure my packaging script deleted them (before signing).

    3. QtWebProcessEngine.app 's Info.plist file needs to be changed before you sign it. The CFBundleIdentifier needs to match that of your signature.

    Like I said, my app has only passed Application Loader's loading process. I've had problems in the past with Qt and the App Store system (see http://stackoverflow.com/questions/25694066/submitting-qt-5-3-app-for-review-on-mac-app-store) before I could even submit the app for review, so I'm skeptical that I'm out of the woods yet.

    I'll keep this thread updated.



  • Attempting Qt 5.7.1 brought its own problem. Namely that on startup the application gives the error:

    This application failed to start because it could not find or load the Qt platform plugin "cocoa"
    in "".
    
    Available platform plugins are: cocoa.
    
    Reinstalling the application may fix this problem.
    Abort trap: 6
    

    I did a little research and tried a couple minor things but nothing worked.

    Meanwhile my app was rejected because of an unused sandbox permission. So I've corrected that and resubmitted a new build still using Qt 5.6.2.

    Really wish I had been able to get 5.8 to work or even 5.7.


  • Lifetime Qt Champion

    That strange, do you have a qt.conf file in your application bundle ?



  • @SGaist Yes.

    So, my app was rejected using Qt 5.6.2 because of the following:

    Your app uses or references the following non-public API(s):
    
    framework: '/System/Library/Frameworks/AppKit.framework/Versions/C/AppKit'
    : NSAccessibilityUnregisterUniqueIdForUIElement
    : _NSAppendToKillRing
    : _NSDrawCarbonThemeBezel
    : _NSDrawCarbonThemeListBox
    : _NSInitializeKillRing
    : _NSNewKillRingSequence
    : _NSPrependToKillRing
    : _NSSetKillRingToYankedState
    : _NSYankFromKillRing
    
    framework: '/System/Library/Frameworks/ApplicationServices.framework/Versions/A/ApplicationServices'
    : CGSSetDenyWindowServerConnections
    : CGSShutdownServerConnections
    : CTFontCopyDefaultCascadeList
    
    The use of non-public APIs is not permitted on the App Store as it can lead to a poor user experience should these APIs change.
    

    Which looks like this is addressed by this: https://doc.qt.io/qt-5/qtwebengine-platform-notes.html#mac-app-store-compatibility

    So I tried Qt 5.7 again and for whatever reason it didn't crash this time! However, it has the same exact problem I was seeing with Qt 5.8.

    I have built and rebuilt and rebuilt QtWebEngine and yet I still cannot get it to work. Is there anyway to verify that my build of QtWebEngine is actually applying the use_appstore_compliant_code flag?

    It seems like this has been a problem for a while: https://bugreports.qt.io/browse/QTBUG-41611


  • Lifetime Qt Champion

    The symbol problem is pretty surprising, are you sure you are using your build of the QtWebEngine module that is app store compliant ?



  • @SGaist said in Sandbox app for the Mac App Store with Qt 5.8 and QtWebEngineProcess:

    The symbol problem is pretty surprising, are you sure you are using your build of the QtWebEngine module that is app store compliant ?

    I was using 5.6.2. Should I build 5.6.2 QtWebEngine with the use_appstore_compliant_code?



  • @SGaist I'm back to trying Qt 5.8 again. I've gone so far as to manually copy over QtWebEngineCore.framework that is in my build folder and I'm getting the same results:

    4/26/17 20:33:04.000 kernel[0]: Sandbox: QtWebEngineProce(86853) deny(1) mach-lookup org.chromium.Chromium.rohitfork.86852
    4/26/17 20:33:04.019 QtWebEngineProcess[86853]: [0426/203304:ERROR:mach_port_broker.mm(43)] bootstrap_look_up: Permission denied (1100)
    4/26/17 20:33:04.000 kernel[0]: Sandbox: QtWebEngineProce(86854) deny(1) mach-lookup org.chromium.Chromium.rohitfork.86852
    4/26/17 20:33:04.020 QtWebEngineProcess[86854]: [0426/203304:ERROR:mach_port_broker.mm(43)] bootstrap_look_up: Permission denied (1100)
    4/26/17 20:33:04.000 kernel[0]: Sandbox: QtWebEngineProce(86854) deny(1) forbidden-sandbox-reinit
    4/26/17 20:33:04.000 kernel[0]: Sandbox: QtWebEngineProce(86853) deny(1) forbidden-sandbox-reinit
    

    Again, is there anyway by inspecting the QtWebEngineCore.framework to make sure is being built with the app store compliant code?



  • Just built QtWebEngine again on 5.8 and made sure to use the QtWebEngineCore.framework that just got built. No luck.

    I see the use_appstore_compliant_code config item was added in Qt 5.7. Guess I'll try building that and see if I have any luck. Heck, I may even try 5.9 at this point just to see.

    Seriously, has anyone successfully published an app that uses QtWebEngine to the MAS?



  • Built 5.7 with use_appstore_compliant_code and am getting the same errors from my app:

    4/27/17 07:10:53.000 kernel[0]: Sandbox: QtWebEngineProce(20764) deny(1) mach-lookup org.chromium.Chromium.rohitfork.20763
    4/27/17 07:10:53.000 kernel[0]: Sandbox: QtWebEngineProce(20765) deny(1) mach-lookup org.chromium.Chromium.rohitfork.20763
    4/27/17 07:10:53.627 QtWebEngineProcess[20764]: [0427/071053:ERROR:mach_broker_mac.mm(52)] bootstrap_look_up: Permission denied (1100)
    4/27/17 07:10:53.627 QtWebEngineProcess[20765]: [0427/071053:ERROR:mach_broker_mac.mm(52)] bootstrap_look_up: Permission denied (1100)
    4/27/17 07:10:53.000 kernel[0]: Sandbox: QtWebEngineProce(20764) deny(1) forbidden-sandbox-reinit
    
    

    Are there any apps on the MAS that use Qt 5.x and QtWebEngine?


  • Lifetime Qt Champion

    That I don't know.

    This thread might link to some clues that could help untangle your situation.



  • @SGaist Thank you for trying to help. I discovered that link a few days ago while researching this problem. It did lead me to add the com.apple.security.application-groups entitlement to my app's entitlements file but the problem persists.

    I am beginning to wonder if this is a problem with QtWebEngineProcess. I have posted this issue to the QtWebEngine mailing list but so far there has been no response.

    I'm at a loss. :(



  • As an alternative, is it possible to invoke C++ from a QtWebView? At this point I'm thinking of abandoning QtWebEngine altogether, though I would really like to avoid that.


  • Lifetime Qt Champion

    Do you mean from the QtWebView module ?



  • @SGaist Yes. That does not use QtWebEngineProcess correct?

    What I need is for a link in the HTML of the control to invoke some C++. I'm able to do this in QtWebEngine with QWebChannel. From what I've read, QtWebView doesn't really support that but maybe it's possible?


  • Lifetime Qt Champion

    No it doesn't use it.

    AFAIK, QtWebView uses the platform native web views so it should be able to run with the QWebChanne module. Note that I haven't personally tested that combo but it's well worth a shot since it will likely make your application more lightweight.


Log in to reply