Friday 17 May 2019

gmail/google mail sending from Postfix

Here's a nasty little change that google/gmail sneaked in over the past couple of months in 2019, adding further authentication/authorisation checks for sending emails to anyone on a google hosted domain/gmail account.

The problem? Mails sent to google hosted/gmail accounts get bounced with a not very clear message:

relay=gmail-smtp-in.l.google.com[2a00:1450:400c:c09::1b]:25, delay=0.62, delays=0.08/0/0.26/0.28, dsn=5.7.1, status=bounced (host gmail-smtp-in.l.google.com[2a00:1450:400c:c09::1b] said: 550-5.7.1 This message does not have authentication information or fails to pass 550-5.7.1 authentication checks. To best protect our users from spam, the 550-5.7.1 message has been blocked. Please visit 550-5.7.1   https://support.google.com/mail/answer/81126#authentication for more 550 5.7.1 information.

So like a good user, you go to the url and lookup the address, which you're told all about bulk messages, mailing list best practices and pretty much it. It doesn't actually address the reasons specifically behind this error.
I suspect this is because it's a generic failure message that google are using, generally rejecting mail that they don't want to accept.

Now in this case, it's from an SMTP mail gateway/relay that several hundred people are behind it, this is a mail relay for an ISP connection, so the users behind it are of varying tech background, but are all on a known connection and various protection mechanisms are already in place to avoid spam and junk going over that connection. Therefore we believe the connection isn't generally junk/blacklisted. SORBS (urgh) and all the other mail checks show the server as clean and not listed, so WHY has google blocked/rejected mail?
Also to add to it. It's not all emails, just some get bounced like this. Send to the same recipient and sometimes it'll get through, sometimes it won't. No real recognition of why, different google gateways/IPs do the same. IPv4 or IPv6 just the same, so no real connection between which ones fail and which ones succeed.

The first thing to add was to create SPF records for the top-level mail relay server, and then get clients sending through it to add an include on their DNS records. This didn't appear to make any difference. Again some rejected some succeeded.
The only last option was to implement DKIM to the relay. This is kinda tricky as DKIM wasn't exactly designed for this scenario. Multiple sending domains, IPs, etc, all going via one mail relay. So the mail relay would need to add the DKIM signature based on it's own signing regardless of who sent it (Since we authenticate/verify senders via the server, we KNOW when an email is accepted to be relayed that it's legit).

To implement, first we have the current setup:
Postfix accepting on 25 for relaying mail out.
Multiple domains/IPs sending via Postfix, we don't KNOW all domains/IPs sending through us.
Debian/Ubuntu standard Postfix install/setup for mail relaying.


The first parts are standard installation steps for DKIM:
apt-get install opendkim opendkim-tools
Then you need to configure the base opendkim package:
/etc/opendkim.conf
/etc/default/opendkim

Configuration here is pretty basic, the key parts you need though are:
/etc/opendkim.conf:
Syslog                  yes
SyslogSuccess yes

LogWhy yes
Canonicalization        relaxed/simple
Mode                    s
SubDomains              yes
KeyTable        /etc/postfix/dkim_key_table
SigningTable    refile:/etc/postfix/dkim_signing_table
ExternalIgnoreList      refile:/etc/postfix/opendkim-TrustedHosts
InternalHosts   refile:/etc/postfix/opendkim-TrustedHosts

So the above sets Syslog to log success and failure reasons. We set Canonicalization to relaxed/Simple as this relaxes the signing. Mode is set to sign and not verify (We only relay outbound via this server, we don't want to check mail coming in for DKIM signatures/validation).
Then the Keytable, SigningTable, ExternalIgnoreList and InternalHosts contain the main parts to allow us to sign any mails going through.

/etc/postfix/dkim_key_table:
mail  serverrelay.mydomain.com:mail:/etc/postfix/dkim.key
Set "mail" as our signing name (keep it consistent throughout), define our servers name "serverrelay.mydomain.com" is our FQDN for our sending server. Again set our signing name and the path to our key.
(We'll generate the dkim.key file in a moment)

/etc/postfix/dkim_signing_table:
*  mail
This sets the domain/entries we want to sign, so the * means anything/all and again "mail" is our signing name setup above.

/etc/postfix/opendkim-TrustedHosts:
127.0.0.1
0.0.0.0/0
This defines what IPs are "trusted", i.e. allowed to be DKIM signed by us. In our instance we've set this to localhost and then to anything. This means that anything postfix is allowed to relay is allowed to be DKIM signed too. (Here we are assuming trust from postfix not being an open relay)

/etc/postfix/opendkim-TrustedHosts:
^^^ points to the above file as it's the same thing.

/etc/default/opendkim
Edit this and set the SOCKET to listen locally, by setting:
SOCKET="inet:8891@localhost"

As we want to use this to simplify our link into postfix (Since postfix is generally chroot'ed and using unix sockets/paths can get complex), easiest to use a tcp listen port.

Now we generate our signature:

cd /etc/postfix;opendkim-genkey -t -s mail -d serverrelay.mydomain.com

As above, "mail" is our signing name and our fqdn is at the end. This generates the files mail.private and mail.txt in the "current" directory, hence changing to /etc/postfix first. Then copy the files to be the ones you defined above in the config:
cp /etc/postfix/mail.private /etc/postfix/dkim.key
The /etc/postfix/mail.txt contains the DNS record you'll need to add to your DNS. So ours generated:

mail._domainkey IN TXT "v=DKIM1; k=rsa; t=y; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDKjCZppULxQyiblah+blah+blah" ; ----- DKIM key mail for serverrelay.mydomain.com

So create that in your DNS. I created it under the subdomain "serverrelay.mydomain.com" and the TLD "mydomain.com" to be sure.

Restart opendkim (service opendkim restart) and check it's running and listening on port 8891.
Then we can integrate into postfix:

milter_protocol = 2
milter_default_action = accept
milter_connect_macros = j {daemon_name} v {if_name} _
smtpd_milters = inet:127.0.0.1:8891
non_smtpd_milters = $smtpd_milters

Add this to your /etc/postfix/main.cf to incorporate the milter/opendkim into your install. If you already have other milters then append the inet line onto the end.
Restart postfix and then watch the mail logs for the opendkim signing taking place.

You should see:

May 17 10:48:59 serverrelay opendkim[24500]: 2A9823C156C: DKIM-Signature header added (s=mail, d=serverrelay.mydomain.com)

For each email sent through the server.

Good luck and hopefully that has helped you out! This was a combination of information I gleaned from several sources as none seemed to join all the parts together for this particular mail relaying scenario.

4 comments:

  1. Thanks for sharing this quality information with us. I really enjoyed reading. Will surely going to share this URL with my friends. Buy PVA Gmail Accounts

    ReplyDelete
  2. Lengkap sekali penjelasanya, saya sangat butuh penjelasan seperti ini.
    Anda bisa mengunjungi https://abimanaseo.com/jasa-backlink-ac-id-edu-murah/ bila memerlukan backlink kualitas kelas wahid.

    ReplyDelete
  3. very very well written content and information i found on your site.
    Very good real estate property for family resident available then contact kalinga keshari rath
    kalinga keshari rath
    kalinga keshari rath

    ReplyDelete

Note: only a member of this blog may post a comment.