QObject connect and shared from this

Fleshbits

I've made myself a client and server based on QtWebSocket. The server needs to delete the client connection after the client closes. Both classes are derived from std::enable_shared_from_this and have factory methods to assure they are created as shared pointers. This allows the client to be deleted once no one has a reference to it anymore.

Well, Qt slots don't use shared pointers, but use a raw pointer to this. Therefore when the slot for client closing is called, the ref count is not incremented. The heap gets corrupted and valgrind gets very angry.

The trace looks like:
onClientClosed slot called
....calls onClientConnectionClosed handler on server
.......erases the shared pointer to the client from collection. Afterward Ref = 1
....unwind Ref = 0, trigger delete
unwind we are still in the client instance onClientClosed slot that just got deleted.

code snippets:

WebsocketClientQtImpl::WebsocketClientQtImpl(QWebSocket* connectedSocket)
    : QObject()
    , m_qtWebsocket(connectedSocket)
{
    QObject::connect(m_qtWebsocket, &QWebSocket::connected, this, &WebsocketClientQtImpl::onConnected);
    QObject::connect(m_qtWebsocket, &QWebSocket::disconnected, this, &WebsocketClientQtImpl::onClosed);
}
WebsocketClientQtImpl::~WebsocketClientQtImpl()
{
    // HEAP gets corrupted here
    delete m_qtWebsocket;
}

Notice the disconnected slot uses 'this'
Meanwhile, clientimpl has its lifetime managed by the stored shared_ptr:

void WebsocketServerQtImpl::onAccepted()
{
    QWebSocket* qtWebsocket = m_qtWebsocketServer->nextPendingConnection();
    // Store the connection
    std::shared_ptr<WebsocketClientQtImpl> client = WebsocketClientQtImpl::create(qtWebsocket);
    auto result = m_clients.emplace(client->getId(), client);
    if(!result.second)
    {
        // handle error
        return;
    }
    // Register handlers
    client->registerHandlerOnClosed(std::bind(&WebsocketServerQtImpl::onClientClosed, this, std::placeholders::_1));        
...

void WebsocketServerQtImpl::onClientClosed(std::shared_ptr<WebsocketClientQtImpl> client)
{
    if(m_handlerOnClientClosed)
    {
        m_handlerOnClientClosed(shared_from_this(), client);
    }
    // Remove from storage
    // This triggers the destructor of the client
    m_clients.erase(client->getId());
...

I end up in the following method, after the clientimpl was already deleted

void WebsocketClientQtImpl::onClosed()
{
    if(m_handlerOnClosed)
    {
        m_handlerOnClosed(shared_from_this());
    }  
    // I UNWIND TO HERE AFTER DELETION.
    // Anything that happens here corrupts the heap.
    // I need to be kept alive by a shared pointer in the original connect call
    }
}

Any suggestions on how to handle this situation or am I simply doomed to not use shared pointers? Even if I used a raw pointer, I'd have to know when it is safe to delete and so would users of my classes.