SMTP works fine when running on Qt, issue when application is packaged

maverick13

Hi,

I have used windeploy to extract the dll.

Also, I know this is not the logging you had suggested, but I was having some trouble implementing it and wanted to get something.

I added some message box logs within the demo application, and I found out where it is hanging up, although I do not understand how/why..

This is within the .cpp of the demo program. It is in the slot for pressing the send button, to begin sending the email.

    errorMessage("connecting to host");
    smtp.connectToHost();
    errorMessage(QString::fromStdString(std::to_string(smtp.waitForReadyConnected())));
    if (!smtp.waitForReadyConnected())
    {
        errorMessage("Connection Failed");
        return;
    }

    errorMessage("authenticating");
    smtp.login(user, password);
    if (!smtp.waitForAuthenticated())
    {
        errorMessage("Authentification Failed");
        return;
    }

The first box pops up telling me we are about to connect to the host, and then the second one which is returning the bool value of the waitForReadyConnect() function never returns.

The code for that function is (waitForReadyConnect())

bool SmtpClient::waitForReadyConnected(int msec) {
    if (state == UnconnectedState)
        return false;

    if (isReadyConnected)
        return true;

    waitForEvent(msec, SIGNAL(readyConnected()));

    return isReadyConnected;
}

waitForEvent()

void SmtpClient::waitForEvent(int msec, const char *successSignal)
{
    QEventLoop loop;
    QObject::connect(this, successSignal, &loop, SLOT(quit()));
    QObject::connect(this, SIGNAL(error(SmtpClient::SmtpError)), &loop, SLOT(quit()));

    if(msec > 0)
    {
        QTimer timer;
        timer.setSingleShot(true);
        connect(&timer, SIGNAL(timeout()), &loop, SLOT(quit()));
        timer.start(msec);
    }

    loop.exec();
}

I assume this has to be some issue with what I am doing since this repository has been used widely with no known issues like what I am experiencing.

I also double checked, again, the values I am using and they are

SMTP Server: smtp.gmail.com
Port Number: 465
Email: myemail@gmail.com
Password: my OTP

I assume there are not any issues with the credentials being I use them when running through Qt Creator and they work just fine, then once it is installed on my machine it seems to never connect to the host...not sure if the logging would really assist here being it does not even get to the point where an error message will be logged..

JonB

@maverick13

which is returning the bool value of the waitForReadyConnect() function never returns

Your timer.setSingleShot() will never timeout or fire any signal since it is destroyed before loop.exec() is reached!

That may have nothing to do with why you never get success or error signals, but it's a start to fix that....

I assume there are not any issues with the credentials

You are trying to send emails using the Gmail smtp.gmail.com connector? Correct me if I am wrong but is it not the case that:

You have (had?) to enable a setting in Gmail to allow this.
If you have 2FA on your account you have to do more than just username+password.

Not to mention I think you have to have SSL stuff for your Qt app.

I think you say you have some system which works from Creator but not when "installed" or "deployed"? is the install/deploy to a different machine or using a different Gmail account? If everything is on one machine with one account I guess this may not apply.

maverick13

Hi @JonB ,

So I believe the code that I posted (excluding the error messages) is as it should be, although I could be wrong being you are bringing up a valid point. The only reason I believe this, is because @piervalli has used the same repository (https://github.com/bluetiger9/SmtpClient-for-Qt) and it seems to be used very widely within Qt for sending emails.

I think you say you have some system which works from Creator but not when "installed" or "deployed"?

Correct.

is the install/deploy to a different machine or using a different Gmail account? If everything is on one machine with one account I guess this may not apply.

Negative, although good thought. I actually have set up everything you talked about here (minus 2FA):

You have (had?) to enable a setting in Gmail to allow this.

If you have 2FA on your account you have to do more than just username+password.

I set up a one time password in order to have a random string and it works perfectly, as you had brought up within Creator with the same credentials (I have them within a demo_vars.h file, as it is used in the GitHub Repository). So there should not be an issue with that I don't think.

JonB

@maverick13 said in SMTP works fine when running on Qt, issue when application is packaged:

So I believe the code that I posted (excluding the error messages) is as it should be, although I could be wrong being you are bringing up a valid point. The only reason I believe this, is because @piervalli has used the same repository (https://github.com/bluetiger9/SmtpClient-for-Qt) and it seems to be used very widely within Qt for sending emails.

If by "the code that I posted" you include the if (msec > 0) { ... } then it is "wrong" in the sense that you will never get a timeout to terminate the later loop.exec(). Maybe other users never needed a timeout because they always got success/error signals instead? (Or they did not get as far as waitForEvent().) But you say you "hang"/never return. Why do you think that is? Why don't you move the QTimer timer; statement to above the if(msec > 0) to see what happens? You can always move it back if you think I am wrong and they are right.... [EXTRA: Could you post the link to the exact file you are copying the timer.setSingleShot(true); code from please? Thanks.]

Next thing. You are calling waitForReadyConnected() twice. Why is that? Might your waitForReadyConnected() return on first call but not on second call? If you got the successful signal once would it get emitted the second time?

Next thing. What sets isReadyConnected = true;? I don't see that anywhere in the code you post.

Maybe this stuff is not to do with your problem. But I sure can't figure the code you show as-is, certainly for your "hanging" case.

maverick13

@JonB

[EXTRA: Could you post the link to the exact file you are copying the timer.setSingleShot(true); code from please? Thanks.]

Here is the .cpp for that method:

https://github.com/bluetiger9/SmtpClient-for-Qt/blob/v2.0/src/smtpclient.cpp#L557

You are calling waitForReadyConnected() twice. Why is that?

I called it twice as I was trying to implement message boxes as a "form" of debugging because I couldn't get my logging utility to work, and was experiencing some issues with qInstallMessageHandler, and wanted to try to debug ASAP. Regardless if it is hung up on the first call within the message box, or if I took that call out, it still is hung up on that call and it is an unexpected result, as it should return (unless your hunch is right)

Next thing. What sets isReadyConnected = true;? I don't see that anywhere in the code you post.

isReadyConnected is set to false in the constructor (https://github.com/bluetiger9/SmtpClient-for-Qt/blob/v2.0/src/smtpclient.cpp#L34), and then set to true in changeState(SmtpClient::ClientState state) (https://github.com/bluetiger9/SmtpClient-for-Qt/blob/v2.0/src/smtpclient.cpp#L300).

I didn't copy in the whole file, being it is cumbersome, but I will attach the GitHub link below to the specific .cpp, as it yields the same result just via GitHub.

https://github.com/bluetiger9/SmtpClient-for-Qt/blob/v2.0/src/smtpclient.cpp

If by "the code that I posted" you include the if (msec > 0) { ... } then it is "wrong" in the sense that you will never get a timeout to terminate the later loop.exec(). Maybe other users never needed a timeout because they always got success/error signals instead? (Or they did not get as far as waitForEvent().) But you say you "hang"/never return. Why do you think that is? Why don't you move the QTimer timer; statement to above the if(msec > 0) to see what happens? You can always move it back if you think I am wrong and they are right....

I am going to try this now! Thanks. I wasn't saying you were wrong, I was just under the assumption that the repository worked so I was not sure if changing anything would help, but I will definitely try this right now and update this post with the result.

JonB

@maverick13
Rmove this line:

errorMessage(QString::fromStdString(std::to_string(smtp.waitForReadyConnected())));

For the timer, you can either move it outside the { .. }s or you could call the static QTimer::singleShot() (whichever overload for your slot stuff) to avoid the scope issue. I suggest their code was not tested when the timeout was required!

This might hopefully aid debugging whatever the bad issue is.

maverick13

@JonB

Why don't you move the QTimer timer; statement to above the if(msec > 0) to see what happens?

When doing this I ended up getting a return value of 0 from the message box, as pictured below.

You are calling waitForReadyConnected() twice. Why is that?

I explained why in my previous reply, but I took it out and it yielded the same result (it just printed out the errorMessage(...) as put in the demo by bluetiger saying "Connection Failed". The reason I did not see the original one before this was probably due to what you had said prior about the QTimer timer neededing to be outside the loop.

The code to the demo that I am using is here

https://github.com/bluetiger9/SmtpClient-for-Qt/tree/v2.0/demos/demo2

The specific file/line we were speaking about with the message boxes is here:

https://github.com/bluetiger9/SmtpClient-for-Qt/blob/v2.0/demos/demo2/sendemail.cpp#L125

This all still works within Qt Creator, but not when creating an installer package of the application, and then installing it on my machine (the same machine in which I am running it via Qt Creator, and developing on). I have also tried to install and run on different machines, and it yields the same unexpected result.

JonB

@maverick13
OK, I think you are at a sensible place now, if I understand right.

The bad case now fails to connect but at least times out?

Since it works in Creator and you need to test the "deployed" build you can't use a debugger. If it were me I would go through whatever code this is that you are using and put whatever "debug" statements in until I identified exactly where the connection is failing to be made. That might be qDebug() statements if you can show them under Windows or printf()s or message boxes or log to file.

My hunch is that the deployed one is not using quite the same libraries as the development one? For example, the smtp connection is across SSL, where does that come from? Or some other difference in environment. But it could even be a bug that is in code and just happens to only show up in the installed case. You just don't know.

maverick13

@JonB

Awesome. Thanks for the help, I appreciate it. I won't close the thread, just becuase if I have any questions, you are definitely more versed to answer and speak on things I do not quite grasp or see at first glance. Really appreciate helping to this point. I'm going to do a bit more debugging and see if I get anywhere.

The worst part of software engineering is the debugging...the best part is figuring out the bug and seeing it work :D

edit: also, if I do reply to you via this thread, if I ever think I have a mini breakthrough -- do you get notified or is it ok to send you a PM?

JonB

@maverick13
Just post in thread. We get notified if you reply, but we look at them anyway.

maverick13

@JonB

For example, the smtp connection is across SSL, where does that come from? Or some other difference in environment.

Also, in regards to this I believe it is coming from QSslSocket, unless I am misunderstanding. I see in another method within smtpclient.cpp, this

void SmtpClient::setConnectionType(ConnectionType ct)
{
    this->connectionType = ct;

    switch (connectionType)
    {
    case TcpConnection:
        socket = new QTcpSocket(this);
        break;
    case SslConnection:
    case TlsConnection:
        socket = new QSslSocket(this);
        connect(socket, SIGNAL(encrypted()),
                this, SLOT(socketEncrypted()));
        break;
    }
}

(I originally wanted to use TLS over SSL [I believe it is more safe with research i've done?], but now that I can't get either working, might as well forget about being picky now)

which leads me to believe it uses a Qt library for the SSL connection. Although, I see there is no SSL connection code being ran in this..Which is odd, then doing a slight scroll down and looking at the changeState(SmtpClient::ClientState) method I see code being ran in the case for SslConnection enum. (this code will be linked below, if you want to look at the .cpp instead)

void SmtpClient::changeState(SmtpClient::ClientState state) {
    this->state = state;

#ifdef QT_NO_DEBUG
    // Emit stateChanged signal only for non-internal states
    if (state <= DisconnectingState) {
        emit stateChanged(state);
    }
#else
    // emit all in debug mode
    qDebug() << "[SmtpClient] State:" << staticMetaObject.enumerator(staticMetaObject.indexOfEnumerator("ClientState")).valueToKey(state);
//    std::string output(staticMetaObject.enumerator(staticMetaObject.indexOfEnumerator("ClientState")).valueToKey(state));
//    log.writeToLogFile("[SmtpClient] State: " + output, "Error");
    emit stateChanged(state);
#endif

    switch (state)
    {
    case ConnectingState:
        switch (connectionType)
        {
        case TlsConnection:
        case TcpConnection:
            socket->connectToHost(host, port);
            break;
        case SslConnection:
            ((QSslSocket*) socket)->connectToHostEncrypted(host, port);
            break;
        }
        break;

Note: some of the comments were mine, I was trying to get a debugger to work this morning for the installed version. They will not show up within the repo, obviously.

setConnectionType - https://github.com/bluetiger9/SmtpClient-for-Qt/blob/v2.0/src/smtpclient.cpp#L228

changeState - https://github.com/bluetiger9/SmtpClient-for-Qt/blob/v2.0/src/smtpclient.cpp#L246

JonB

@maverick13 said in SMTP works fine when running on Qt, issue when application is packaged:

which leads me to believe it uses a Qt library for the SSL connection.

Under Linux it uses OpenSSL libraries for this, and there can be issues if it picks up different versions. I don't know how this works under Windows.

Not sure what your "Although, I see there is no SSL connection code being ran in this" means. The port number is for SSL. Do make 100% sure the two instances are trying to connect on the same port number.

Not my area, but you might also use a tool like Wireshark to see whether/what communications are actually happening with the server.

maverick13

@JonB

Not sure what your "Although, I see there is no SSL connection code being ran in this" means.

Nevermind you can disregard that... I did not notice there wasn't a break statement after the 2nd case, so if it is SslConnection it will just fall through, sorry about that.

The port number is for SSL. Do make 100% sure the two instances are trying to connect on the same port number.

Yeah, I looked to verify that the port number is not being set anywhere within the code explicitly, which it is not. I'll go and get to debugging, thanks again.

I've used wireshark a bit in school. Might try to get it downloaded again and see if I can pick anything up. Thanks.

Under Linux it uses OpenSSL libraries for this, and there can be issues if it picks up different versions. I don't know how this works under Windows.

Also, I believe it uses OpenSSL for Windows as well. I had to download OpenSSL, as well as define OpenSSL within the build environment path for Qt Creator to get it to work "properly"

JonB

@maverick13
Literally, just put some debug/trace statements into code to see just where it gets to. Then run it from Creator and from installed. Compare where installed gets to before connection failure. Print out port number etc.

piervalli

This post is deleted!

maverick13

@JonB
One thing I wanted to look at was this,

My hunch is that the deployed one is not using quite the same libraries as the development one? For example, the smtp connection is across SSL, where does that come from? Or some other difference in environment.

Because I believe this is most likely the issue as well, and I might have noticed something I can't believe I didn't before. The DLL's for OpenSSL (libcrypto-1_1.dll & libssl-1_1.dll) were not in the data directory of the installer...obviously this caused an issue. I tried the following steps:

Cleared out the DLL's from the data directory
Placed myself within the C:\Qt\5.15.12\mingw81_32\bin\ directory within PowerShell
Ran the following command .\windeployqt.exe '..\..\..\..\Users\Admin\Documents\Qt Installers\TestInstaller\packages\com.email.test\data\TestingSMTP.exe', where TestingSMTP.exe is the executable created from running via Qt Creator. I ran this prior to this experiment, but wanted to run it again to cover all my bases.
I noticed that the OpenSSL DLLs were not being copied to the data directory...so I manually copied them in from the directory that I initialized within the Projects -> Build Environment -> Path (pictured below)

It works now, although I did switch which GitHub repository I was using, as I had found the original one we were debugging with after using a different one without MIME capabilities (I believe MIME is sending to multiple users? Might be the wrong word, but that's the capability the one I have working now was missing).

So, I am going to try this on the original one, which is the one I need to get working, but at least there is something.

Any idea why the OpenSSL DLLs did not copy over with the windeployqt command? Or does this command purposefully not copy these DLLs over?

Edit: Just tested it with the original repository, worked fine. Now just need to implement it within my actual project and not the demo, and hopefully all goes smoothly. Still curious about this though, if anyone knows.

Any idea why the OpenSSL DLLs did not copy over with the windeployqt command? Or does this command purposefully not copy these DLLs over?

JonB

@maverick13

So are you saying that now, with the locally-copied SSL DLLs, your packaged/deployed/installed SMTP client application finally works, as it did from Creator with the development directory? I think you are, because you have marked the topic as solved.

The perhaps it was picking up the SSL DLLs from elsewhere on your machine? Or not finding them at all, but not warning? Since the port is for SSL one might guess that a problem/incompatibility there could cause malfunction, though shame there was no apparent indication/feedback for this.

@maverick13 said in SMTP works fine when running on Qt, issue when application is packaged:

Any idea why the OpenSSL DLLs did not copy over with the windeployqt command? Or does this command purposefully not copy these DLLs over?

You will have to await someone else for this. I have never used linuxdeployqt. Maybe it expects SSL DLLs to be system-wide resources so not to copy/install locally?

maverick13

Yes, it ended up working fine on my actual project, as it did with the demo project I was using as a tester, within this thread. Thanks for your help @JonB, hopefully I can get some feedback for someone on the question below, but regardless, you've been very helpful in the debugging process!

Any idea why the OpenSSL DLLs did not copy over with the windeployqt command? Or does this command purposefully not copy these DLLs over?

Also, one more small question. I should be using TLS over SSL, correct? I believe from my prior research, TLS is more secure with encryption, and less prone to getting the email swiped/edited within transit, or did I misinterpret? While I am never sending sensitive data with this, I still like to look at the security side of things, always just to be safe.

piervalli

@maverick13
The standard going to TLS, for instance Microsoft on Azure the minimum standard Is TLS 1.1

maverick13

@piervalli

Awesome, thank you! I will move ahead with TLS as I expected. Thanks for your assistance earlier as well!