The Simple Mail Transfer Protocol, used for sending e-mail messages, has been written in the 1970s. It was not designed to be secure.
SMTP is a pretty simple protocol, and it follows the idea of classic mail: You write the address on the front of the envelope, your own address on the back, and you send it. There is no way for the recipient to check if the address you’ve written on the back is really yours.
So, what’s the problem? It always worked like that.
E-mail has become the most important means of “serious” online communication. Accounts on most sites are connected to an e-mail address, including e-commerce. If you receive an e-mail from your bank, how do you know if it’s really your bank that has sent the e-mail?
You can’t, not automatically.
You could phone your bank and ask them if they’ve sent it. But that’s not something most people would do. People want to read their e-mail and usually trust the sender line shown in their mail client.
The solution for geeks!
There has been a solution for this problem for ages. Standards like S/MIME and OpenPGP allow you to sign e-mail messages. Everyone that trusts your public key can be pretty sure you are who you say you are.
Most desktop e-mail clients support S/MIME and OpenPGP, but the number of users is small. A select group of people that really cares about privacy and security uses it for sending e-mails. (OpenPGP is used for other things as well: A lot of GNU/Linux packages are signed using OpenPGP).
This solution isn’t suitable for the masses though. Keeping your private certificate secure (and maintaining a web of trust in the case of OpenPGP) is too much work. It should be easier!
The solution for the masses!
People that lack the required knowledge to set up their own e-mail server usually use the servers of their provider, or a webmail service. This is where we have to implement the authentication if we want to reach everyone.
A few standards have been developed to make this happen.
Sender Policy Framework (SPF)
SPF allows server administrators to keep a whitelist of servers allowed to send e-mails for their domain. The server receiving the e-mail can then check if it came from an allowed IP.
The whitelist is stored in a TXT or SPF record in DNS. As the domain administrator is (or should be) the only one able to change the DNS records for his domain, this is considered a safe place to store the whitelist.
For example, the SPF record for demuzere.be:
demuzere.be. IN TXT "v=spf1 include:_spf.google.com -all" demuzere.be. IN SPF "v=spf1 include:_spf.google.com -all"
This record basically tells you that you should read the SPF record on _spf.google.com, and disallow all other servers.
DomainKeys Identified Mail (DKIM)
The problem with SPF is that it breaks when an e-mail forwarder forwards your e-mail. In this case, the receiving server will see the IP of the forwarder, and not that of the original sender. For the same reason, this also breaks mailing lists.
That’s where DKIM comes in! It signs the e-mail body and a few critical headers on the server before sending it to the recipient.
The public key is read from DNS (for the same reasons the whitelist for SPF is stored there). The receiving server can validate the message using the public key and a signature in the e-mail headers. Forwarders and mailing lists keep these headers intact, and don’t break DKIM. (Unless they’re not configured correctly..)
For example, the DKIM record for demuzere.be:
demunet._domainkey.demuzere.be. IN TXT "v=DKIM1\;k=rsa\;p=MIGfMA [..] AQAB\;t=s"
DKIM allows you to set multiple public keys (the key name is “demunet” in the record shown above). The reason to allow this is simple: You might want to let a third party sign e-mails on your behalf. Companies using a third party service for sending bulk e-mail can give them a separate key, and revoke it if needed.
Domain-based Message Authentication, Reporting & Conformance (DMARC)
But how should the receiving server know if it should check for SPF and DKIM records? SPF can be checked easily: If there is no SPF record, the sender doesn’t use it. But what for DKIM? You have to know the public key name in order to check if it exists. There is no way to know if an unsigned e-mail should have been signed.
There have been a few proposals on how to solve this. One of them was Author Domain Signing Policy (ADSP). It wasn’t popular: The biggest webmail providers didn’t implement it, and then it simply doesn’t work.
The latest try is DMARC. Behind the ridiculously long name hides yet another DNS record. This time it allows you to specify a policy as to how the receiving server should threat your messages. The difference with ADSP is that DMARC also supports reporting. Receiving servers can send you reports about the number of messages it received from your domain, what they did with them, and why they decided to do that.
You can suggest what to do with messages that fail DKIM and SPF tests. Possible options are reject, quarantaine (sends them to the spambox most of the time) and none. This last option simply sends reports, but doesn’t stop the e-mails from reaching the inbox if something is wrong with them.
A last example, the DMARC record for demuzere.be:
_dmarc.demuzere.be. IN TXT "v=DMARC1\;pct=100\;p=reject\;adkim=s\;aspf=r"