How to make Qt reinitialize the OpenSsl engine, or let me take ownership of it?

Hristo Konstantinov

I'm using OpenSsl and load pkcs11 engine for my application, since the client Ssl should be built with the private key from the Hardware Security Module (a.k.a. a flash drive with a sim card - a requirement from the government website I'm building the client for). I also need it for some XML digital signatures.
I can't control the life time of the session, since the user can plug and unplug the usb whenever he wants. If I don't use QNetworkAccessManager, everything works okay.If I see that the x509 certificate from the HSM is empty, it means that the usb has been unplugged and I try to initialize it again, whenever user needs it. The message I get in the debug console on destruction is as follows:

Exception thrown at 0x00007FFF1A4B474C (KernelBase.dll) in PisHis.exe: 0x80100002: The action was cancelled by an SCardCancel request.
Exception thrown at 0x00007FFF1A4B474C (KernelBase.dll) in PisHis.exe: 0x0000071A: The remote procedure call was canceled, or if a call time-out was specified, the call timed out.
Exception thrown at 0x00007FFF1A4B474C in PisHis.exe: Microsoft C++ exception: unsigned long at memory location 0x0000006E83AFEEF8.
Exception thrown at 0x00007FFF1A4B474C in PisHis.exe: Microsoft C++ exception: unsigned long at memory location 0x0000006E83AFF024.
Exception thrown at 0x00007FFF1A4B474C in PisHis.exe: Microsoft C++ exception: unsigned long at memory location 0x0000006E834FF7F8.
'PisHis.exe' (Win32): Unloaded 'C:\Dev\IDPrimePKCS1164.dll'
'PisHis.exe' (Win32): Unloaded 'C:\Dev\pkcs11.dll'

Obviously the pkcs11 engine and the hsm module (IDPrime by Gemalto) have been unloaded and everything's cool.

HOWEVER if I send just one single QNetworkRequest, everything goes to hell (the destructor of my custom openssl engine wrapper doesn't free the libraries), and I cannot reinitialize the engine. I'm guessing somehow Qt gets ownership of the engine and starts using it, so reinitialization of the engine by me is not possible (it gives me "could not bind to the requested symbol name" error). Is there any way to make Qt ditch the engine or just let me handle it by myself? One workaround I could think of is to keep my engine as static and notify the user with "The HSM could not be loaded, please restart the program", when the usb is accidentally unplugged but that's just plain stupid.

My custom OpenSsl class constructor:

CryptoSslEngine::CryptoSslEngine()
{
   
    ERR_load_crypto_strings();
    SSL_load_error_strings();
    OpenSSL_add_all_algorithms();
    SSL_library_init();
    ENGINE_load_dynamic();
    ERR_clear_error();
 
    p_engine = ENGINE_by_id("dynamic");
    if (!p_engine) {
        throw std::exception(
            ERR_reason_error_string(ERR_get_error())
        );
    }

    if (!ENGINE_ctrl_cmd_string(p_engine, "SO_PATH", "C:/Dev/pkcs11.dll", 0) ||
        !ENGINE_ctrl_cmd_string(p_engine, "ID", "pkcs11", 0) ||
        !ENGINE_ctrl_cmd_string(p_engine, "LIST_ADD", "1", 0) ||
        !ENGINE_ctrl_cmd_string(p_engine, "LOAD", NULL, 0) ||
        !ENGINE_ctrl_cmd_string(p_engine, "MODULE_PATH", "c:/dev/IDPrimePKCS1164.dll", 0) ||
        !ENGINE_ctrl_cmd_string(p_engine, "VERBOSE", NULL, 1) ||
        !ENGINE_init(p_engine)// ||
      //  !ENGINE_set_default(p_engine, ENGINE_METHOD_ALL)
        )
    {
        ENGINE_free(p_engine);

        throw std::exception(
            ERR_reason_error_string(ERR_get_error()) //gives "could not bind to the requested symbol name"
        );
    }
}

The destructor (which DOESN'T work, if QNetworkAccessManager has been used at least once):

CryptoSslEngine::~CryptoSslEngine()
{
    if (p_engine) {
        ENGINE_unregister_RSA(p_engine);
        ENGINE_unregister_ciphers(p_engine);
        ENGINE_unregister_digests(p_engine);
        ENGINE_finish(p_engine);
        ENGINE_remove(p_engine);
        ENGINE_free(p_engine);
        ENGINE_cleanup();
    }
  
}

And finally, how I use the whole thing:

void PisHis::sendRequest()
{
     //the QNetworkAccessManager is a member variable and those two functions at least prevent some crashes
     manager.clearAccessCache();
     manager.clearConnectionCache();

    CryptoSslEngine engine;
 
    auto certString = engine.Ssl_x509cert();

    if (certString.empty()){
        showMessage("No HSM found");
        return; //the end of the scopes calls the destructor and releases the engine
    }

    QSslCertificate cert(certString.data());
    QSslKey key = QSslKey(Qt::HANDLE(engine.getPrivateKey()));
    
    if (key.isNull()){
        qDebug() << "PIN dialog from the module has been canceled";
        return;
    }

    QSslConfiguration config = QSslConfiguration::defaultConfiguration();
    config.setProtocol(QSsl::SslProtocol::TlsV1_2);
    config.setLocalCertificate(cert);
    config.setPrivateKey(key);

    QUrl url("https://pis.nhif.bg/ws/PISService");
   
    QNetworkRequest request(url);
    request.setSslConfiguration(config);
    request.setHeader(QNetworkRequest::KnownHeaders::ContentTypeHeader, "text/xml;charset=\"utf-8\"");
    request.setRawHeader("SOAPAction", "\"http://pis.technologica.com/view\"");
    request.setRawHeader("accept", "\"application/xml\"");

    manager.post(request, getQuery());
    
}

I've seen the whole idea from this project here: https://github.com/iksaif/qsslkey-p11
However the author states that he uses a PATCHED version of Qt, which I assume fixes the problem, but the link to it is no longer available.

Hristo Konstantinov

I kind of managed to make it work . The freeing of the engine has to be followed by deleting the QNetworkAccessManager and creating a new one. However I still have problems with programmatically managing the engine. The destructor has to track whether the private key from the HSM has to be accessed or not:

CryptoSslEngine::~CryptoSslEngine()
{
        if (prvKeyHasBeenAccessed)
        {
          ENGINE_finish(engine);
          ENGINE_free(engine);
          ENGINE_remove(engine);

          ENGINE_unregister_ciphers(engine);
          ENGINE_unregister_digests(engine);
          ENGINE_finish(engine);

          
        }
        else
        {
            ENGINE_finish(engine);
            ENGINE_free(engine);
            ENGINE_remove(engine);
        }
}

I have no idea why I have to call those functions in this particular order, but that way it works. I've noticed that only when accessing the HSM private key the module gets loaded, and then if I want to destroy the engine I have to call ENGINE_free() twice (once for the module and once for the pkcs11 engine). Probably I'll have to try and use Libp11 from OpenSC to manage these things for me, but that's not part of the Qt anyway.

Hristo Konstantinov

Ok, it seems that the problem is not in QNetworkAccessManager, but in QSslKey itself. The constructor which takes a HANDLE to the key from the pkcs11 hsm doesn't release it afterwards(as written in the documentation) and freeing the engine manually has no effect, since its reference count is held up by the QNetwork backend. If there was only a way to re-initialize the whole QNetwork module, or make the SslKey release the handle, it would be great!

SGaist

Hi,

I don't have a solution at hand but did you try to contact the author of the original code ? He may still have his patches lying around.

kshegunov

@Hristo-Konstantinov said in How to make Qt reinitialize the OpenSsl engine, or let me take ownership of it?:

QSslKey key = QSslKey(Qt::HANDLE(engine.getPrivateKey()));

This will take ownership of the handle, make sure that the engine is not going to delete it. Perhaps there's something like takePrivateKey() or something?

Otherwise the code looks decent to me.

Hristo Konstantinov

I kind of managed to make it work . The freeing of the engine has to be followed by deleting the QNetworkAccessManager and creating a new one. However I still have problems with programmatically managing the engine. The destructor has to track whether the private key from the HSM has to be accessed or not:

CryptoSslEngine::~CryptoSslEngine()
{
        if (prvKeyHasBeenAccessed)
        {
          ENGINE_finish(engine);
          ENGINE_free(engine);
          ENGINE_remove(engine);

          ENGINE_unregister_ciphers(engine);
          ENGINE_unregister_digests(engine);
          ENGINE_finish(engine);

          
        }
        else
        {
            ENGINE_finish(engine);
            ENGINE_free(engine);
            ENGINE_remove(engine);
        }
}

I have no idea why I have to call those functions in this particular order, but that way it works. I've noticed that only when accessing the HSM private key the module gets loaded, and then if I want to destroy the engine I have to call ENGINE_free() twice (once for the module and once for the pkcs11 engine). Probably I'll have to try and use Libp11 from OpenSC to manage these things for me, but that's not part of the Qt anyway.

Hristo Konstantinov

UPDATE: I've ditched completely the engine part, by just using libp11. Wrote a simple C++ wrapper around it, and now everything works without problems. However I still have to "reset" QNetworkAccessManager, but calling ClearAccessCache() does the job perfectly.