[SOLVED] Qt GUI in VST plug-in - clicking menu bar crashes the host

[vlk23]

Hi,
I have a strange problem using Qt GUI for VST plug-ins on Mac OS. I am using QMacNativeWidget mechanism to get Qt into Carbon window as shown in several examples. The problem is that when I remove the plug-in from the host and click host`s menu bar, the host application usually crashes. There is no problem using Cubase, but for example Ableton Live or Reaper crash frequently.
I am using the flag:
@QApplication::instance()->setAttribute(Qt::AA_MacPluginApplication)@

to keep hosts menu bar before qApp initialization. I have also used "this patch":https://bugreports.qt-project.org/browse/QTBUG-11934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel when building Qt libraries. *If I do not create QMacNativeWidget and let the host to show just an empty window, it doesnt crash at all*.

Here is the main part of code:
@
#include <qapplication.h>
#include <QWidget>
#include <QMacNativeWidget>

static int numEditorInstances = 0;
static QApplication* g_myQtApp = 0;
static bool g_ownAppInstance = false;
static bool g_firstAppInit = false;

CAFXQtVST2Editor::CAFXQtVST2Editor (void _plugInDSP):AEffEditor(0)
{
numEditorInstances++;
mWidget = NULL;
}
/----------------------------------------------------------------------------*/
CAFXQtVST2Editor::~CAFXQtVST2Editor()
{
if (mWidget)
{
mWidget->close();
delete mWidget;
mWidget = NULL;
}

numEditorInstances--;

if (numEditorInstances == 0)
{
if (g_ownAppInstance)
{
delete g_myQtApp;
}
}
}
/----------------------------------------------------------------------------/
bool CAFXQtVST2Editor::open(void *parent)
{
if (!g_firstAppInit)
{
g_firstAppInit = true;
if (!qApp)
{
int argc = 0;
g_ownAppInstance = true;
QApplication::instance()->setAttribute(Qt::AA_MacPluginApplication);
g_myQtApp = new QApplication(argc, 0);
}
else
{
g_ownAppInstance = false;
g_myQtApp = qApp;
}
}

mWindowRef = (WindowRef) parent;

if (!mWidget)
{
mWidget = new QMacNativeWidget();
mWidget->setFixedSize(400, 400);
mWidget->setPalette (QPalette (Qt::red));
mWidget->setAutoFillBackground (true);
}

// find provided content view (from host)
HIViewRef contentView = 0;
GetRootControl((HIWindowRef)parent, &contentView);
if (contentView)
{
HIViewRef child = reinterpret_cast<HIViewRef> (mWidget->winId());
OSStatus err = HIViewAddSubview (contentView, child);

Rect bounds;
GetControlBounds (contentView, &bounds);
mWidget->move (bounds.left, bounds.top);
mWidget->show();
}
return true;
}
/----------------------------------------------------------------------------/
void CAFXQtVST2Editor::close()
{
if (mWidget)
{
HIViewRemoveFromSuperview((HIViewRef) mWidget->winId());
mWidget->hide();
}
}
@

And the crash report for the crashed thread:
@
Thread 0 Crashed: Dispatch queue: com.apple.main-thread
0 com.apple.CoreFoundation 0x951e8ce5 CFEqual + 53
1 com.apple.CoreFoundation 0x951ecf04 __CFDictionaryCallback + 420
2 com.apple.CoreFoundation 0x951ed2b9 ___CFBasicHashFindBucket1 + 537
3 com.apple.CoreFoundation 0x951f542c CFBasicHashFindBucket + 252
4 com.apple.CoreFoundation 0x951f52f3 CFDictionaryGetValue + 131
5 com.apple.HIToolbox 0x947a48ae HIObjectClass::Lookup(__CFString const*, unsigned char) + 104
6 com.apple.HIToolbox 0x947a46d0 HIObject::Create(__CFString const*, OpaqueEventRef*, HIObject**) + 38
7 com.apple.HIToolbox 0x947a4687 HIObjectCreate + 76
8 com.apple.HIToolbox 0x947ede7d MenuData::CreateMenuContentView() + 195
9 com.apple.HIToolbox 0x947eddab MenuData::GetModalView() + 25
10 com.apple.HIToolbox 0x947edd47 HIMenuGetContentView + 41
11 com.apple.shortcut 0x930b631a -[SCTSearchManager installShortcutMenuHandlers:] + 222
12 com.apple.shortcut 0x930b6114 shortcutBeginTrackingEventHandler + 121
13 com.apple.HIToolbox 0x947a9c2f DispatchEventToHandlers(EventTargetRec*, OpaqueEventRef*, HandlerCallRec*) + 1567
14 com.apple.HIToolbox 0x947a8ef6 SendEventToEventTargetInternal(OpaqueEventRef*, OpaqueEventTargetRef*, HandlerCallRec*) + 411
15 com.apple.HIToolbox 0x947a8d55 SendEventToEventTargetWithOptions + 58
16 com.apple.HIToolbox 0x947eda8c SendTrackingStatus(unsigned long, MenuData*, unsigned long, unsigned long, unsigned long, OpaqueEventRef*, unsigned long, unsigned char) + 433
17 com.apple.HIToolbox 0x947ed883 SendBeginTracking(MenuData*, unsigned long, unsigned long, MenuSelectData*) + 70
18 com.apple.HIToolbox 0x947eccb8 SetupMenuTracking(MenuSelectData&, unsigned char, Point, double, MenuData*, unsigned long, unsigned short, Rect const*, Rect const*, unsigned int, Rect const*, __CFString const*) + 1915
19 com.apple.HIToolbox 0x947ec07d MenuSelectCore(MenuData*, Point, double, unsigned long, OpaqueMenuRef**, unsigned short*) + 227
20 com.apple.HIToolbox 0x947eb8bb _HandleMenuSelection2 + 465
21 com.apple.HIToolbox 0x947eb6d9 _HandleMenuSelection + 53
22 com.apple.AppKit 0x92523f96 _NSHandleCarbonMenuEvent + 285
23 com.apple.AppKit 0x924f8b46 _DPSNextEvent + 2304
24 com.apple.AppKit 0x924f7dd6 -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] + 156
25 com.apple.AppKit 0x924ba1f3 -[NSApplication run] + 821
26 com.ableton.live 0x014e2fc2 std::out_of_range::~out_of_range() + 3448594
27 com.ableton.live 0x014e0004 std::out_of_range::~out_of_range() + 3436372
28 com.ableton.live 0x00002dd2 0x1000 + 7634
29 com.ableton.live 0x00002cf9 0x1000 + 7417
@

My config:
Qt 4.8 (just updated from 4.7.4, the problem remains)
Mac OS 10.6.8, XCode 3.2.6.

I am trying to solve the problem for more than a month so I would be happy if somebody could help me.

[vlk23]

The problem disappeared when I changed static link to framework link and compiled customized Qt frameworks.