Gmail SMTP authentication

JonB

Not actually a Qt question, and I wasn't sure what category to put this in, but hoping for some help from the experty peeps around here.

We have a program, which happens to be written using Qt, which sends email via SMTP in certain circumstances. Note that this is "automated" sending of an email message, not user interaction to compose and send the mail.

The administrator sets up the credentials required to connect to the chosen mail provider, which could be any SMTP host, not just Gmail. They provide the hostname/IP plus username and password, which the program saves (securely) and uses to connect. This has worked for a long time for a variety of SMTP mail host providers.

Google has just sent an email with the following gist:

On 30 May, you may lose access to apps that are using less secure sign-in technology

To help keep your account secure, Google will no longer support the use of third-party apps or devices which ask you to sign in to your Google Account using only your username and password. Instead, you’ll need to sign in using Sign in with Google or other more secure technologies, like OAuth 2.0

An app or device that uses Simple Mail Transfer Protocol (SMTP) to send emails using your Google Account has less secure access to your Gmail. This might be an older device, like a printer or scanner. To continue using your Google Account with this app or device:

App – remove your Google Account from the app or device and sign in again using Sign in with Google

Do the experts think this will affect our situation? I'm not clear. The app does not presently "sign into Google". But it does (if the admin wants to use Gmail for SMTP sending) specify the Gmail smtp host, username and password, are they saying that will be disabled?

If so, do we need to change our Qt app to move to this "OAuth 2"? What is involved? The irritant is that at present it is all generic, regardless of SMTP provider, there would have to be special recognition of doing something different in the Gmail case....

SGaist

Hi,

Based on the text, I would say yes. The Qt network authorization module might help.

JonB

@SGaist
Thanks, will look into.

Someone else said:

<My app> has been registered as an app with google as it uses gmail to send emails to users
As I understand it, it means that we need to register <our app> with google, and probably update the code to request and submit access tokens

At some level, are we now looking at needing to do this sort of thing, needing to register something with google?

SGaist

That would be a task for the users of your application. They will need that to get an id/key combo for the auth process.

JonB

@SGaist
Hmmm, that's not good :( Thanks.

SGaist

I may have misunderstood your user case. Since you write the application, you usually create the "auth application" on the provider(s) you want to support. Then the personne that wants to use it must sign in through that provider and then the application gets the token to use to access whatever it is the service provides.

JonB

@SGaist
I believe we are on the same level, but with issues over wording.

The application presently sends email when the end user does something in it, rather than the end user sending the email himself/manually.
The application achieves this via an external SMTP server.
The administrator/installer of our application configures any chosen SMTP server, supplying our app with IP address/port/username/password. It might be the Google/gmail SMTP server, or it might be one from another provider.
This presently works generically, regardless of provider/server. Although the Administrator needs to have some email account set up there for use by the program, up until now there has been absolutely no need to "register the application with the SMTP provider*, it works "out-of-the-box" with no further configuring.
From end of May I am suspecting that Gmail SMTP server will "reject" such a connection with uysername+password.
We will now need to do something about "registering" the application with Google/Gmail? So that it can now obtain and swap "tokens" with the server to be allowed to log on for sending mail? Something like OAuth2?
I'm not sure whether the application itself will need registering, by us for any end-user sites, or maybe by the end-user sites themselves? But either way it will require code changes, and will/may not quite work "out-of-the-box"? We patch the app from time to time, will that require some kind of re-registration/update with Google?

artwaw

@JonB You don't need to register your program with google unless you want to make some API calls or something. If you don't want to mess with google OAuth (which, truth to tell, I never managed to implement in a way to inject stored credentials so the user doesn't need to auth upon each program run) you need to obtain separate credentials for your program:

the google account in the security settings needs to have "allow less secure apps" set.
then in the section responsible/listing those "less secure apps" you need to create a credentials. Write them down or copy/paste as once the view is closed you'll never see them again.
those credentials should be used with your program to access the mail server.

I use this process for my own automation access to send maintenance logs and for end-user software I write for my users to deliver me diagnostic data. Once set it works, no "refreshment" of credentials needed.
To keep things sane I have one credential pair per tool/instance, so in case of a security problem I can quickly kill access by deleting one of the "less secure app" accesses. And, of course, TLS/SSL.

EDIT: there is, of course, a question of how to store those credentials within the program. My dirty workaround is that I wrote a quick console tool making qCompress on the string, string is hardcoded in program and decompressed when needed. Not ideal but I don't need to hand out anything to the users.

JonB

@artwaw
Hi. This may be more (or less) than what I am asking about.

Remember we already have this working for any SMTP server accepting username+password authentication.

For Gmail (only) it does require the account holder to configure a setting to "allow less secure access" before it will accept SMTP connections. But that's fine, that's what the Administrator does if they want to use Gmail as their SMTP server.

the google account in the security settings needs to have "allow less secure apps" set.

Yes, we have that presently.

then in the section responsible/listing those "less secure apps" you need to create a credentials. Write them down or copy/paste as once the view is closed you'll never see them again.
those credentials should be used with your program to access the mail server.

That I am not at all familiar with. I do not believe there was any such thing when I last looked a couple of years ago. Do you perhaps have a screenshot?

You don't need to register your program with google unless you want to make some API calls or something. If you don't want to mess with google OAuth ...

That sounds better in principle. I do not want to move to Google APIs/OAuth, and I have no desire to "register" anything if I do not have to. I simply want to know what has to be done to allow it to continue to send email via SMTP, with as little changes/fuss as possible!

My question is what is Google telling me is changing on May 30?

artwaw

@JonB Apparently "less secure apps" is enabled for personal accounts by default.
Short path:

open myaccount.google.com (assuming you are logging in with the account in question).
on the left hand side you'll have "security" menu option. Click.
follow the help page https://support.google.com/accounts/answer/185833?hl=en-GB

And for non-managed google account that should be all.

If the account is managed by the Google Workspace type of organisation (former GSuite) admin of that needs to:

login to admin.google.com
security -> access and data control -> less secure apps
enable feature for user in question.

As for "what changes" on May 30th I have no idea but quick search found this: https://h30434.www3.hp.com/t5/Scanning-Faxing-Copying/After-May-30th-2022-what-will-be-the-way-to-scan-to-gmail/td-p/8320453

Which means that for the account in question google will phase out plain login without OAuth challenge. The right way to proceed is to create "less secure apps" credentials and continue as before.

JonB

@artwaw said in Gmail SMTP authentication:

follow the help page https://support.google.com/accounts/answer/185833?hl=en-GB

Under "Signing in to Google," select App Passwords. You may need to sign in. If you don’t have this option, it might be because:

2-Step Verification is not set up for your account.

For my own Gmail I do not see any "App Passwords". That may be because I personally do not have 2-step active, and don't wish to do so/try it out....

At the bottom, choose Select app and choose the app you using and then Select device and choose the device you’re using and then Generate.

If I (the end user) got this far, I don't know how "our app" would appear as one to be selected. Sounds more like a list of apps registered with Google?

Let's say this does all work. Now that means 2-step verification with mobile is enabled. An end user does something in our desktop app which causes it to want to send SMTP email. That might mean Google wants to send a code to mobile and have user enter it? Would that authentication appear on the desktop OK when run from a non-web desktop Python Qt program?

artwaw

@JonB said in Gmail SMTP authentication:

, I don't know how "our app" would appear as one to be selected

it does not matter. Select "custom", provide recognisable name:

After providing the name you'll see something like this:

The yellow is the password (no spaces). Make a copy as this disappears forever as you hit "done". Then just use the users full email and that password to login to smtp.

This circumvents 2FA, so nothing will be sent anywhere in that regard. As this hampers the security aspect of the account take special care not to share those credentials.

JonB

@artwaw
Thanks, this looks great!

I note it says "from Apps on devices that don't support 2-step". As I said earlier, I never get to see what you show

For my own Gmail I do not see any "App Passwords". That may be because I personally do not have 2-step active, and don't wish to do so/try it out....

so still not sure how I'm supposed to get to the screenshot you show, unfortunately....

artwaw

@JonB I am afraid that rolling 2FA is unavoidable for this.
(A side note outside of the topic is that 2FA should be mandatory wherever possible for obvious security reasons, but that's just my sysadmin persona speaking).

JonB

@artwaw said in Gmail SMTP authentication:

@JonB I am afraid that rolling 2FA is unavoidable for this.

OK, but the text says "for devices which do not support 2FA"! That's pretty confusing! Does it mean "You will need to use 2FA enabled on your account in order to set this up for your app, but then an end user will not need 2FA to use this way of connecting to Gmail SMTP once you have set it up"?

artwaw

@JonB said in Gmail SMTP authentication:

You will need to use 2FA enabled on your account in order to set this up for your app, but then an end user will not need 2FA to use this way of connecting to Gmail SMTP once you have set it up"?

That is my understanding of the situation, yes. (Please bear in mind that I did not work outside 2FA Google environment for quite some years now)

artwaw

@JonB
Take a look at the original message you've got from google:

"On May 30, you may lose access to apps that are using less secure sign-in technology. To help keep your account secure, Google will no longer support the use of third-party apps or devices which ask you to sign in to your Google Account using only your username and password. Instead, you’ll need to sign in using Sign in with Google or other more secure technologies, like OAuth 2.0."

That implies that logging in with google functionality will cease to work without 2FA challenge enabled. But in order to keep the functionality you have now, you need to enable 2FA and generate the "less secure app" credentials. That's the scope of the changes you face, if I read the situation correctly.

JonB

@artwaw
All good, thank you very much for your time. I will have to try this out --- or rather, get others to try it out.

I cannot be sure but am marking this topic as solved.

mchinand

I'm not sure how it works in the background for initiating this request for access, but for my Synology NAS I recently set up email notifications. In the process of configuring it, it went to a Google page asking to authorize access to send emails on my behalf. Going to the Security page of my Google account and viewing the 'Third-Party Apps with Account Access', I now have Synology listed; perhaps your app will have to obtain the same authorization.

JonB

@mchinand
Yeah, that (the need to "register" anything about the app with Google) is precsiely what I am hoping to avoid, which I believe/hope @artwaw's suggestions will allow me to do. The app is not released, not for sale, not for general consumption, so we don't need such hassles!