Fun and games (?!) building QSQLITE for Qt 5.15.0

AMGAMG

I am developing a 32-bit Windows application that needs an encrypted version of SQLite, so I have downloaded the SQLite sources and am attempting to rebuild the database driver.

After many false starts, I believe I have successfully built the driver as follows:

Download the sources for Qt 5.15.0 (to C:\Qt\5.15.0\Src)
Copy the new sqlite3.c and sqlite3.h files to C:\Qt\5.15.0\Src\qtbase\src\3rdparty\sqlite
cd \Qt\5.15.0\Src\
configure -prefix C:\Qt\5.15.0\Src\qtbase -nomake tests -nomake examples -sql-sqlite -opensource -confirm-license
cd qtbase\src\plugins\sqldrivers
\Qt\5.15.9\mingw81_32\bin\qmake "INCLUDEPATH+=\Qt\5.15.0\Tools\OpenSSL\Win_x86\include" "LIBS+=-L%\Qt\5.15.0\Tools\OpenSSL\Win_x86\lib -l:libssl.lib -l:libcrypto.lib"
mingw32-make sub-sqlite
mingw32-make sub-sqlite-install_subtargets

However, when I run my application, it fails at the first call to QSqlDatabase::addDatabase ("QSQLITE", <filename>) with the error

QSqlDatabase: QSQLITE driver not loaded
QSqlDatabase: available drivers: QSQLITE QODBC QODBC3 QPSQL QPSQL7

Questions:

What have I done wrong?
The operations I'm doing to build all seem reasonable to me, although the final make target: sub-sqlite-install_subtargets looks odd (I wouldn't expect the name to include the final _subtargets).
I notice that I don't have a file named qsqlited.dll in C:\Qt\5.15.0\qtbase\plugins\sqldrivers at the end of the process, and I do have a file called qsqlite.dll.debug, which is different to what I had with Qt version 5.11.3)
Why does the error message say that it can't load QSQLITE driver, but then list it as an available driver?

jsulm

@AMGAMG said in Fun and games (?!) building QSQLITE for Qt 5.15.0:

Why does the error message say that it can't load QSQLITE driver, but then list it as an available driver?

Set QT_DEBUG_PLUGINS environment variable before running your app and check its output (see https://doc.qt.io/qt-5/debug.html).

AMGAMG

There are several yards of output. Anything in particular I should look for, rather than clutter this up by posting everything?

JonB

@AMGAMG
The one which is right at the end, that says why it has failed :)

Christian Ehrlicher

Why do you build QSQLite by your own at all? It's in the binary distribution.

AMGAMG

@JonB You'd think so, wouldn't you?

Here are the last few lines:

Got keys from plugin meta data ("generic")
QFactoryLoader::QFactoryLoader() checking directory path "C:/development-gitlab/TotalVet/impl/build-Qt_5_15_0-mingw81_32-Debug/bin/bearer" ...
loaded library "C:/Qt/5.15.0/mingw81_32/plugins/bearer/qgenericbearer.dll"
Message: "Starting Application"
QFactoryLoader::QFactoryLoader() checking directory path "C:/Qt/5.15.0/mingw81_32/plugins/accessiblebridge" ...
QFactoryLoader::QFactoryLoader() checking directory path "C:/development-gitlab/TotalVet/impl/build-Qt_5_15_0-mingw81_32-Debug/bin/accessiblebridge" ...
15:12:40: The program has unexpectedly finished.
15:12:40: The process was ended forcefully.
15:12:40: C:\development-gitlab\TotalVet\impl\build-Qt_5_15_0-mingw81_32-Debug\bin\ttv.exe crashed.

I don't see anything obvious there about why it failed. (I do wonder why it's looking at or for "accessiblebridge" which certainly doesn't exist.)

However, searching for the word "error" gets me

Got keys from plugin meta data ("QPSQL7", "QPSQL")
QFactoryLoader::QFactoryLoader() checking directory path "C:/development-gitlab/TotalVet/impl/build-Qt_5_15_0-mingw81_32-Debug/bin/sqldrivers" ...
Cannot load library C:\Qt\5.15.0\mingw81_32\plugins\sqldrivers\qsqlite.dll: Unknown error 0x000000c1.
QLibraryPrivate::loadPlugin failed on "C:/Qt/5.15.0/mingw81_32/plugins/sqldrivers/qsqlite.dll" : "Cannot load library C:\Qt\5.15.0\mingw81_32\plugins\sqldrivers\qsqlite.dll: Unknown error 0x000000c1."
QSqlDatabase: QSQLITE driver not loaded
QSqlDatabase: available drivers: QSQLITE QODBC QODBC3 QPSQL QPSQL7

Searching for "windows error 0x000000c1" appears to give me some ways to fix this error for (other) specific failures. I can't find anything that says what this error is, which strikes me as a prerequisite for any attempt to fix it.

Christian Ehrlicher

Use the qsqlite plugin provided by Qt
if you don't want to use it for whatever reason (I don't see any) then use Dependency Walker to see what dependent dlls are needed.
Use the qsqlite plugin provided by Qt

AMGAMG

@Christian Ehrlicher. Apologies for slow response, the forum will only let me post once every 10 minutes.

I'm building QSQLite by my own because I need encryption. I believe I need to rebuild to use the SQLite Encryption Extension Please tell me I'm wrong.

JonB

@AMGAMG
I cannot guide you about what you need to do for your requirements.

But the error line we are interested in is indeed:

QLibraryPrivate::loadPlugin failed on "C:/Qt/5.15.0/mingw81_32/plugins/sqldrivers/qsqlite.dll" : "Cannot load library C:\Qt\5.15.0\mingw81_32\plugins\sqldrivers\qsqlite.dll: Unknown error 0x000000c1."

You should indeed try a Dependency Walker as @Christian-Ehrlicher says. I think you have something wrong in your build! For 0x000000c1, is there any chance you have the wrong architecture, as per https://stackoverflow.com/questions/61080829/cannot-load-library-dll-unknown-error-0x000000c1 ? Or plain dependencies missing, like https://stackoverflow.com/questions/64915865/qmlplugindump-error-when-building-with-mingw-64bits-qt-5-13-1 ? Seems Qt plugin loading likes to throw this error code :)

AMGAMG

@AMGAMG said in Fun and games (?!) building QSQLITE for Qt 5.15.0:

mingw32-make sub-sqlite

Thanks @JonB. I'll check out those two StackOverflow links.

Christian Ehrlicher

@AMGAMG said in Fun and games (?!) building QSQLITE for Qt 5.15.0:

configure -prefix C:\Qt\5.15.0\Src\qtbase -nomake tests -nomake examples -sql-sqlite -opensource -confirm-license

This is not needed and therefore wrong

cd qtbase\src\plugins\sqldrivers
\Qt\5.15.9\mingw81_32\bin\qmake "INCLUDEPATH+=\Qt\5.15.0\Tools\OpenSSL\Win_x86\include" "LIBS+=-L%\Qt\5.15.0\Tools\OpenSSL\Win_x86\lib -l:libssl.lib -l:libcrypto.lib"

L% and -l: doesn't look right. Also are you sure you're using a 32bit binary build of Qt?

Please follow the documentation.

AMGAMG

@Christian-Ehrlicher OK, thanks. I didn't realise I already had a .pro file in Qt\5.15.0\Src\qtbase\src\plugins\sqldrivers.

My make line is wrong, I'm running a batch file with environment variables in and trying to translate on-the-fly as I post here. The second argument to qmake is actually "LIBS+=-L\Qt\5.15.0\Tools\OpenSSL\Win_x86\lib -l:libssl.lib -l:libcrypto.lib". I understand that -l:libssl.lib links against libssl.lib, whereas -lssl would attempt to link against libssl.a. This is advice given by a colleague, I assumed that since the dll built without errors, it was good advice; if you'd like me to try something else, please tell me. If you could say how I can "make clean" as well, so I don't need to wait 45 minutes while I remove and re-download the sources, I would be very grateful.

I'm not sure what build of Qt I'm using, to be honest, but I am sure it's a 32-bit kit. As well as reading and writing SQLite databases, the application has to read MS Access database written with a 32-bit version and when we tried building with a 64-bit kit, we couldn't read them.

I'm on the point of trying again, making sure that \Qt\5.15.0\Tools.OpenSSL\Win_x86\lib is in PATH at run-time. I will use the sqldrivers.pro that came with the source distribution.

It's getting late in the UK so I won't be able to post results until the morning because of the "only one post every 10 minutes" thing.

(Later - also some edits above about the use of -l:)

Adding the directory to PATH didn't fix the problem.

I now have this piece of code, which demonstrates the problem:

        const QString dllName ("C:/Qt/5.15.0/mingw81_32/plugins/sqldrivers/qsqlite.dll");
        QLibrary lib (dllName);
        const bool loaded (lib.load ());
        if (loaded)
        {
            qDebug ().noquote () << QString ("%1 loaded OK").arg (dllName);
        }
        else
        {
            qDebug ().noquote () << QString ("%1 failed to load - '%2'").arg (dllName).arg (lib.errorString ());
        }

I also understand why you are suspicious that I'm not building a 32-bit dll. After reading this answer on superuser.com, I've run file against my driver:

$ file /cygdrive/c/Qt/5.15.0/mingw81_32/plugins/sqldrivers/qsqlite.dll
/cygdrive/c/Qt/5.15.0/mingw81_32/plugins/sqldrivers/qsqlite.dll: PE32 executable (DLL) (GUI) Intel 80386, for MS Windows

I've never run the utility before, but that looks to me like a 32-bit image.

AMGAMG

I believe I have solved this now and have been able to build a encryption-aware SQLite driver on Qt 5.15.0. My head is still spinning from relief so I am uncertain, but this is what I believe I did to build a version using the SQLite Encryption Extension and OpenSSL (I will recheck and repeat, and post back here if I find that I actually did something different):

Download sqlite3-see-aes256-openssl.c and sqlite3.h from the SQLite website - note that this requires a licence from SQLite.org, and use them to overwrite sqlite3.c and sqlite3.h in the Qt sources tree (C:\Qt\5.15.0\Src\qtbase\src\3rdparty\sqlite on my machine.
Move to the database drivers source directory (C:\Qt\5.15.0\Src\qtbase\src\plugins\sqldrivers on my machine)
Run qmake, spcifying some extra dependencies:

qmake "INCLUDEPATH+=C:\Qt\Tools\OpenSSL\Win_x86\include" "LIBS+=-LC:\Qt\Tools\OpenSSL\Win_x86\lib -l:libssl.lib -l:libcrypto.lib"

Run make to build the database driver.

mingw32-make sub-sqlite

Copy the new driver from the source tree to kit (on my machine, from C:\Qt\5.15.0\Src\qtbase\src\plugins\sqldriver\plugins\sqldriver to C:\Qt\5.15.0\mingw81_32\plugins\sqldrivers)
Ensure that the OpenSSL crypto library is on the PATH when you run the application (on my machine, I need to add C:\Qt\Tools\OpenSS:\Win_x86\bin)

Many thanks to everyone, particularly @Christian-Ehrlicher, @jsulm and @JonB for their help.

I believe that the Qt Documentation could be improved in this area and will probably raise a bug report to that effect.

My process currently only builds qsqlite.dll which I assume is a Release version. I was hoping also to get a Debug version, probably called qsqlited.dll, but since what I have works with a Debug build of my software, and I have absolutely no desire to debug the driver I will leave it here.

Christian Ehrlicher

@AMGAMG said in Fun and games (?!) building QSQLITE for Qt 5.15.0:

I believe that the Qt Documentation could be improved in this area and will probably raise a bug report to that effect.

In which way? The steps you did are exactly the same as documented: https://doc.qt.io/qt-5/sql-driver.html#compile-only-a-specific-sql-driver

AMGAMG

@Christian-Ehrlicher

To judge by the number of threads on these forums, and elsewhere on the internet, dealing with queries on how to build a Qt database driver something is wrong.

The documentation section "Building the dirvers" starts with a discussion of the output of configure. I agree that there is no instruction to run configure, but neither is there any specific instruction not to build the driver. A week has passed now, but so far as I recall it was only when you stated above, in response to my publishing the command that I was using, that "This is not needed and therefore wrong", that it even occurred to me not to run configure.

I'm still having trouble - I am currently trying to circulate instructions on how to build the driver around the project team, and I find that I am unable to rebuild the driver from a cygwin window, or from a MinGW window started by selecting "Git-bash here" from the windows explorer context menu. I am able to build it from the Windows command prompt. I suspect this is nothing whatever to do with Qt, and am NOT asking for any assistance to resolve this problem here.

I'm sorry not to have answered your question about documentation before now. If you can advise me how to set up my Qt Forum account so that I am emailed when someone else contributes to a thread I have contributed to (or even only one that I've started) I would be very grateful.

Back to my suspicion that the Qt Documentation could be improved... I notice that you have responded to a large proportion of the threads about building database drivers. Why do you think there are so many?

Thnks again for pointing me in the right direction.

Christian Ehrlicher

I personally don't understand why anyone will create a sqldatabase driver plugin by them self when there are basic c++ and compiling knowledge is missing. Why not simply use the drivers by Qt instead? Why all people want to use a open-source unfriendly database like MySQL at all?

And if you think you can improve the documentation - feel free to provide a patch.

AMGAMG

@Christian-Ehrlicher: I tried to explain some days ago.

I am not using MySQL. I am using SQLite.
The application I am building requires an encrypted database. The SQLite driver that is bundled with Qt does not support encryption; at least, it is not documented to support it.
SQLite.org do provide an encryption extension, which is available on payment of an additional licence fee.
On payment of the fee, you are supplied with "drop-in" replacements for the public domain sqlite3.c and sqlite3.h. You must then recompile your software using those sources.

I have the greatest respect for your knowledge of Qt as well as your patience. I would appreciate it if you would extend me a similar courtesy.

Sprezzatura

@Christian-Ehrlicher AMGAMG clearly answered your question on 15 Dec: ENCRYPTION. He's building the encrypted version of SQLite, which is only available as source code, and you have to compile it yourself. The built-in Qt version does not handle SQLite encryption.