How to properly include openssl for a windows deployment when using mingw?

ThirdStrand

@SGaist Ah, but if it only actually worked!

I have gone round and round here and in the "exchanges" and "stacks" to no avail. My QSslSocket application runs like a champ inside Qt Creator, but refuses to run on the same PC outside of Qt Creator.

This really should not be this hard!

Application loads the co-located libcrypto and libssl DLL's in the application directory. Confirmed using LoadedDllsView application.

QMqttClient first state: Connecting, second state, Disconnected.

Error number: 256 which corresponds to:

QMqttClient::TransportInvalid

Explanation from Qt Docs:
The underlying transport caused an error. For example, the connection might have been interrupted unexpectedly.

This has become rather maddening, and rarely can one find any reliable information for how to deploy the two little OpenSSL libs that will work with Qt apps.

Qt 5.15.0 w/MinGW 64 8.1.0 compiler.

I have tried 64 bit OpenSLL libs from multiple sources to no avail, including the ones packaged with QtCreator.

SGaist

Hi,

Did you check that you deployed all the OpenSSL dependencies as well ?

ThirdStrand

@SGaist Sorry, I hit Submit before I finished my post and now I am stuck in "new guy" 10 minute hell. :) Please review my edits above.

RE: OpenSSL dependencies, can anyone point to what they actually might be?

SGaist

Use Dependency Walker or some similar tool to check what these libraries are using.

ThirdStrand

@SGaist Yessir, that's what I have been doing for two days. It cannot be a secret. ;) But it hides from me nonetheless.

I know for libcrypto, nothing external is required. I use it all the time for symmetrical encryption of data. Just drop libcrypto into the Deployment folder. So it must be with libssl somewhere that my issue resides.

When run OUTSIDE Qt Creator NO TLS connection is possible:

Filename	Full Path	File Size	
Qt5Core.dll	C:\Users\mitch\Deploy\CAPBone\Qt5Core.dll	8,174,864	
Qt5Gui.dll	C:\Users\mitch\Deploy\CAPBone\Qt5Gui.dll	9,684,704	
Qt5Mqtt.dll	C:\Users\mitch\Deploy\CAPBone\Qt5Mqtt.dll	275,348	
Qt5Network.dll	C:\Users\mitch\Deploy\CAPBone\Qt5Network.dll	2,692,000	
Qt5SerialPort.dll	C:\Users\mitch\Deploy\CAPBone\Qt5SerialPort.dll	155,472	
Qt5Sql.dll	C:\Users\mitch\Deploy\CAPBone\Qt5Sql.dll	425,104	
libgcc_s_seh-1.dll	C:\Users\mitch\Deploy\CAPBone\libgcc_s_seh-1.dll	76,288	
libstdc++-6.dll	C:\Users\mitch\Deploy\CAPBone\libstdc++-6.dll	1,417,216	
libwinpthread-1.dll	C:\Users\mitch\Deploy\CAPBone\libwinpthread-1.dll	52,224	
Qt5Widgets.dll	C:\Users\mitch\Deploy\CAPBone\Qt5Widgets.dll	8,655,264	
qwindows.dll	C:\Users\mitch\Deploy\CAPBone\platforms\qwindows.dll	2,891,000	
qwindowsvistastyle.dll	C:\Users\mitch\Deploy\CAPBone\styles\qwindowsvistastyle.dll	314,048	
qgif.dll	C:\Users\mitch\Deploy\CAPBone\imageformats\qgif.dll	78,848	
qicns.dll	C:\Users\mitch\Deploy\CAPBone\imageformats\qicns.dll	98,872	
qico.dll	C:\Users\mitch\Deploy\CAPBone\imageformats\qico.dll	82,456	
qjpeg.dll	C:\Users\mitch\Deploy\CAPBone\imageformats\qjpeg.dll	486,776	
qsvg.dll	C:\Users\mitch\Deploy\CAPBone\imageformats\qsvg.dll	75,320	
Qt5Svg.dll	C:\Users\mitch\Deploy\CAPBone\Qt5Svg.dll	573,688	
qtga.dll	C:\Users\mitch\Deploy\CAPBone\imageformats\qtga.dll	72,272	
qtiff.dll	C:\Users\mitch\Deploy\CAPBone\imageformats\qtiff.dll	527,656	
qwbmp.dll	C:\Users\mitch\Deploy\CAPBone\imageformats\qwbmp.dll	69,880	
qwebp.dll	C:\Users\mitch\Deploy\CAPBone\imageformats\qwebp.dll	678,608	
libssl-1_1-x64.dll	C:\Users\mitch\Deploy\CAPBone\libssl-1_1-x64.dll	925,400	
libcrypto-1_1-x64.dll	C:\Users\mitch\Deploy\CAPBone\libcrypto-1_1-x64.dll	3,245,272	
qsqlite.dll	C:\Users\mitch\Deploy\CAPBone\sqldrivers\qsqlite.dll	1,266,224

When run INSIDE Qt Creator EVERYTHING works as expected. I even put the OpenSSL libs into the build folder to force them to be loaded:

Filename	Full Path	File Size	
libssl-1_1-x64.dll	C:\Users\mitch\build\CAPBone-Desktop_Qt_5_15_0_MinGW_64_bit-Release\release\libssl-1_1-x64.dll	925,400	
libcrypto-1_1-x64.dll	C:\Users\mitch\build\CAPBone-Desktop_Qt_5_15_0_MinGW_64_bit-Release\release\libcrypto-1_1-x64.dll	3,245,272	
Qt5Core.dll	C:\Qt\5.15.0\mingw81_64\bin\Qt5Core.dll	8,174,864	
Qt5Gui.dll	C:\Qt\5.15.0\mingw81_64\bin\Qt5Gui.dll	9,684,704	
Qt5Mqtt.dll	C:\Qt\5.15.0\mingw81_64\bin\Qt5Mqtt.dll	275,348	
Qt5Network.dll	C:\Qt\5.15.0\mingw81_64\bin\Qt5Network.dll	2,692,000	
Qt5SerialPort.dll	C:\Qt\5.15.0\mingw81_64\bin\Qt5SerialPort.dll	155,472	
Qt5Sql.dll	C:\Qt\5.15.0\mingw81_64\bin\Qt5Sql.dll	425,104	
libgcc_s_seh-1.dll	C:\Qt\5.15.0\mingw81_64\bin\libgcc_s_seh-1.dll	76,288	
libstdc++-6.dll	C:\Qt\5.15.0\mingw81_64\bin\libstdc++-6.dll	1,417,216	
libwinpthread-1.dll	C:\Qt\5.15.0\mingw81_64\bin\libwinpthread-1.dll	52,224	
Qt5Widgets.dll	C:\Qt\5.15.0\mingw81_64\bin\Qt5Widgets.dll	8,655,264	
qwindows.dll	C:\Qt\5.15.0\mingw81_64\plugins\platforms\qwindows.dll	2,891,000	
qwindowsvistastyle.dll	C:\Qt\5.15.0\mingw81_64\plugins\styles\qwindowsvistastyle.dll	314,048	
qgif.dll	C:\Qt\5.15.0\mingw81_64\plugins\imageformats\qgif.dll	78,848	
qicns.dll	C:\Qt\5.15.0\mingw81_64\plugins\imageformats\qicns.dll	98,872	
qico.dll	C:\Qt\5.15.0\mingw81_64\plugins\imageformats\qico.dll	82,456	
qjpeg.dll	C:\Qt\5.15.0\mingw81_64\plugins\imageformats\qjpeg.dll	486,776	
qsvg.dll	C:\Qt\5.15.0\mingw81_64\plugins\imageformats\qsvg.dll	75,320	
Qt5Svg.dll	C:\Qt\5.15.0\mingw81_64\bin\Qt5Svg.dll	573,688	
qtga.dll	C:\Qt\5.15.0\mingw81_64\plugins\imageformats\qtga.dll	72,272	
qtiff.dll	C:\Qt\5.15.0\mingw81_64\plugins\imageformats\qtiff.dll	527,656	
qwbmp.dll	C:\Qt\5.15.0\mingw81_64\plugins\imageformats\qwbmp.dll	69,880	
qwebp.dll	C:\Qt\5.15.0\mingw81_64\plugins\imageformats\qwebp.dll	678,608	
qsqlite.dll	C:\Qt\5.15.0\mingw81_64\plugins\sqldrivers\qsqlite.dll	1,266,224

Looking at /var/log/syslog on the MQTT broker (mosquitto) on AWS:

Jun 15 20:54:28 ip-xxxxxxxxxxxxxx mosquitto[2057]: 1623790468: New connection from xx.xx.xx.xx on port 8883.
Jun 15 20:54:28 ip-xxxxxxxxxxxxxx mosquitto[2057]: 1623790468: Socket error on client <unknown>, disconnecting.
Jun 15 20:54:48 ip-xxxxxxxxxxxxxx mosquitto[2057]: 1623790488: New connection from xx.xx.xx.xx on port 8883.
Jun 15 20:54:48 ip-xxxxxxxxxxxxxx mosquitto[2057]: 1623790488: New client connected from xx.xx.xx.xx as 9487cba7cfa54c76a207341 (c1, k60).

The first two lines are the fail of OUTSIDE Qt Creator.
The second two lines are the success INSIDE Qt Creator.

I shall keep rooting around in Depends (sounds about right!) to see what gives. If anyone has a clue for me, please chime in.

SGaist

Just to be sure, your application is 64 bit as well ?

nick_uu

Hi there, I have a same problem like @ThirdStrand had. First i am a newbie about QT.

QT version: 6.2.0 - 64
system: windows 10
mingw810_64

I have a project which use websocket. I run the project very perfectly when In Qt Creator by tap run button.But when I run the exe file that deployed by windeployqt , QWebsocket::open this method will emit an error of SSL Sockets are not supported on this platform.
I have tried copy <QTDIR\Tools\OpenSSL\Win_x64\bin> libcrypto-1_1-x64.dll and libssl-1_1-x64.dll to Deployment folder and aside the exe file, but it did not work.

How could i sovle this problem? Thank you.

ThirdStrand

@SGaist Yes, definitely a 64-bit MinGW application, and 64-bit OpenSSL libs.

For anyone searching for mingw compiled OpenSSL binaries, see: https://curl.se/windows/

There is an engines-1_1 folder with 2 included with the OpenSSL distributions. Having it in the deploy folder makes no difference however.

For the OUTSIDE run program, these show up as "missing" dependencies in Depends. Makes no sense whatsoever, since it is running on the same machine!?!? And by "makes no sense", naturally, I mean, "makes no sense to me".

API-MS-WIN-CORE-FILE-L1-1-0.DLL
API-MS-WIN-CORE-PROCESSTHREADS-L1-1-0.DLL
API-MS-WIN-CORE-PROCESSTHREADS-L1-1-1.DLL
API-MS-WIN-CORE-SYNCH-L1-1-0.DLL
API-MS-WIN-CORE-SYNCH-L1-2-0.DLL
API-MS-WIN-CORE-SYSINFO-L1-1-0.DLL
API-MS-WIN-CRT-PRIVATE-L1-1-0.DLL
API-MS-WIN-EVENTING-CONTROLLER-L1-1-0.DLL
KERNEL32.DLL
SHCORE.DLL
SHLWAPI.DLL
IEFRAME.DLL

The OUTSIDE run path at the cmd prompt is shown below. If one runs the "openssl" command at the cmd prompt, the one from Strawberry PERL is the one which runs. If left to it's own devices with no deployed OpenSSL libraries in the deploy folder, running the program will get the libs from C:\WINDOWS.

I think at this point I may uninstall ALL OpenSSL from this dev machine, other than the libs included by C:\Qt\Tools... If it works in Qt Creator, there is something being done in Qt Creator to use these libs properly that simply needs to be sussed out and DOCUMENTED (for goodness sake!).

Path=
C:\Program Files (x86)\VMware\VMware Player\bin\;
C:\Program Files\AdoptOpenJDK\jdk-8.0.252.09-hotspot\bin;
C:\WINDOWS\system32;
C:\WINDOWS;
C:\WINDOWS\System32\Wbem;
C:\WINDOWS\System32\WindowsPowerShell\v1.0\;
C:\WINDOWS\System32\OpenSSH\;
C:\Program Files (x86)\Bitvise SSH Client;
C:\Program Files\PuTTY\;
C:\Strawberry\c\bin;
C:\Strawberry\perl\site\bin;
C:\Strawberry\perl\bin;
C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit\;
C:\Program Files\Git\cmd;
C:\Program Files\GeographicLib-1.51\bin;
C:\Users\mitch\AppData\Local\Microsoft\WindowsApps;
C:\Users\mitch\AppData\Local\GitHubDesktop\bin;
C:\Users\mitch\AppData\Local\Microsoft\WindowsApps;
C:\Users\mitch\Microsoft VS Code\bin;
C:\Program Files\JetBrains\IntelliJ IDEA Community Edition 2021.1.1\bin;

ThirdStrand

@nick_uu This is what has led me to believe that there are some sort of settings or environment variables in use by Qt Creator, which packages it's own OpenSSL libraries (on demand install by the developer). I'll see if I can have the application here print out the envionment in use by Qt Creator when the project is run. Maybe that will lead to some discovery.

In the case of Windows std. install, C:\Qt\Tools\OpenSSL\ and following.

One would suspect that if the entire Qt OpenSSL deployed tree were included with an application that it should work properly. Just like the rest of Qt libraries. That assumes proper paths are made available to the running application of course.

ThirdStrand

Well folks, here's a lesson in persistence and attention to detail!

The old RHEL8 AWS server had a cert file named "mqtt-client.p12". RHEL is bitchy and I got tired of fighting with it (gratuitous jab), so I changed to a Debian server (ahhhhhh) on AWS and reconfigured my MQTT broker and nginx and made all new keys/certs of course. New filename is "mqtt_client.p12".

Inside the Qt Creator IDE my build folder had the correct new client .p12 file with the "_". Outside of Qt Creator my deploy folder had the old RHEL client .p12 file with the "-".

So what looked like was happening truly was what was happening. It tried to set up the TLS socket and it failed with the wrong cert/key/CA combination and the server correctly shut down the pipe!

I profusely apologize for any wasted time by @SGaist or others, but at least we have a partial answer (that being "operator head-space"). Now to deploy to another PC and re-test after a belt of Bourbon and some rest.

Don't "solve" this one yet, because I'd like to report on Deployment success/fail on another PC, and get someone else to do the same. The Qt application will correctly load libssl and libcrypto from the exe's folder before looking elsewhere. I just want to ensure that there are no other external dependencies lingering before I call this one "solved" from my perspective.

flashch

Hi,
Actually, I'm facing the exact same issue:
My goal is to have a standalone software which use QNetwork which needs OpenSSL in Windows 64 bits.
The OpenSSL version that I am using is the one that can be installed by the Qt Maintenance tool.

My software with OpenSSL works inside QtCreator with Qt 6.2.0 but as soon as I want to deploy it, I get the following error:
"qt.network.ssl: No functional TLS backend was found" and "qt.network.ssl: No TLS backend is available"
and if I print:

QSslSocket::supportsSsl() -> 1 inside QtCreator, 0 outside
QSslSocket::sslLibraryBuildVersionString() -> "OpenSSL 1.1.1k 25 Mar 2021" inside QtCreator , "" outside
QSslSocket::sslLibraryVersionString() -> "OpenSSL 1.1.1j 16 Feb 2021" inside QtCreator, "" outside

I have OpenSSL installed in .../Qt/Tools/OpenSSL with the Win_x64 folder.
This is currently the only OpenSSL library available. (I renamed the Anaconda3 Folder where I have another OpenSSL install to remove the access to it from PATH).

I agree with @ThirdStrand that it is probably an issue with a path and dependencies.

I tried to add to the Windows PATH the OpenSSL Win_x64/bin folder and launch my app but without success. (I print the cmd "where openssl" and the path to the correct folder is displayed. So, it seems that it can have access to this folder)
I tried to copy the OpenSSL folder to my deployment folder without success.
I tried to add the libcrypto-1_1-x64.dll and libssl-1_1-x64.dll files to the deployment folder (In addition to the usual libgcc_s_seh-1.dll libstdc++-6.dll and libwinpthread-1.dll) but without success. I also added the dll from the Win_x64/lib/engines-1_1 but also have no success (confirming the results obtained by @ThirdStrand in a previous message)
I tried to use windeployqt to populate automatically the deployment folder (and adding the files listed in the above mentionned point) without success.

Looking at the dependencies in Depends gives me similar results than @ThirdStrand regarding the "missing" dependencies: API_MS_WIN... dll files are missing, which also doesn't make any sense to me.

I already managed to deploy several apps on my computer so no issues in this direction.

I also tried to add the static, openssl and openssl-linked flag to the CONFIG parameter in my .pro file, as well as adding the libssl and libcrypto to the LIBS parametr, and declaring the OPENSSL_INCDIR, OPENSSL_LIBDIR and OPENSSL_LIBS.
In the OPENSSL_LIBS var, I have defined the libraries that this webpage recommands: -lWs2_32 -lGdi32 -lAdvapi32 -lCrypt32 -lUser32.

Edit:
I installed OpenSSL 1.1.1k and copied the two files libcrypto-1_1-x64.dll and libssl-1_1-x64.dll to the deployment folder. For testing, I removed the access to the OpenSSL version located in Qt/Tools/OpenSSL (to force using the files located in the deployment folder). With this, I am able to run my program in QtCreator: it detects the correct OpenSSL install. I believe this was due to the difference between the version j and k of OpenSSL between the one built in Qt and the one located in the folder /Qt/Tools.
However, once I copy the QCore.dll (as well as the other) in the deployment folder, the OpenSSL is not visible anymore, as if the QCore was removing the OpenSSL support.

I am currently out of ideas.

flashch

I finally manage to make it work: it was an issue with dll (as it could be expected) that were in the Qt folder but not in the path, and that were not preventing the app launch.

Solution:

Copy the libssl-1_1-x64.dll and libcrypto-1_1-x64.dll files to your deployment folder.
Copy the dll files located in C:\Qt\6.2.0\mingw81_64\plugins\tls to your deployment folder. (Of course, adapt to your Qt path, version and compiler)

Procedure to find it:

Launch the app from Qt
Using the ListDLL tool from Windows, in a cmd, type "Listdlls.exe YOUR_EXE_NAME.exe", copy the output to a text file.
Close you app
Launch the app from outside Qt
Repeat step 2 and copy the output to a second text file
Compare the dll list obtained in the two text files in step 2 and 5 and copy the missing dll from the Qt folder to your deployment folder (I copied the same folder structure (e.g. plugins/tls) but it's not necessary).

I also added the other dll that were missing (such as styles/qwindowsvistastyle.dll all the dll in plugins/imageformats folder, iconenines/qsvgicon.dll and Qt6Svg.dll for my app)

I hope this can help someone.

ThirdStrand

@flashch MANTRA: "It's always the path. It's always the permissions. It's always the path. It's always the permissions."

As @SGaist has noted many times in these forums, OpenSSL is NOT managed in any way by "windeployqt.exe". Thus if you need those libs, and any supporting Qt libs, you pretty well have to do that part yourself. I did note that when the "mqtt" directive is added to the QT += line of the .pro file, suddenly the SSL libs show up demand loaded inside Qt Creator. If you take out "mqtt", they are unloaded. One would have expected the SSL support to be included with the "network" directive since one of it's classes is QSslSocket as well as other SSL support classes.

I can report that I packaged my program with the libssl and libcrypto dll's, and sent it to my primary tester. He installed it and was able to connect to the MQTT broker immediately via TLS. I have not yet confirmed whether or not he has OpenSSL installed, but I do not think that he does. This would confirm that only those two files are necessary for this type of functionality, which is what is expected.

This is a MinGW 64 bit application and I used the MinGW 64 bit OpenSSL libs as well (from the curl project noted in a previous post).

More deployment testing to come.

ThirdStrand

@flashch the C:\Qt\<version>\<platform>\bin\windeployqt.exe pretty well does all of that for you, except for the OpenSSL libs.

in my case:

C:\Qt\5.15.0\mingw81_64\bin\windeployqt.exe \users\myname\Deploy\MyProg\MyProg.exe

That ensures that the necessary libs are gotten into the folder that holds the EXE file. Then you have to add your own libs and support from there. Most of the time, when you have no other libs, there is nothing else to do.

Regards...

Hai_China

拷贝那两个dll文件是没有用的

SGaist

@Hai_China said in How to properly include openssl for a windows deployment when using mingw?:

拷贝那两个dll文件是没有用的

Please use English on the main forums.

There are language specific sub forum for other languages like Chinese in the international section.