From 6ce0bb9a8f9fd7d169cbb414a9537d68c5290aae Mon Sep 17 00:00:00 2001 From: Prefetch Date: Fri, 14 Oct 2022 23:25:28 +0200 Subject: Initial commit after migration from Hugo --- source/blog/2020/email-server-extras/index.md | 431 ++++++++++++ source/blog/2020/email-server/index.md | 729 +++++++++++++++++++++ source/blog/2022/email-server-revisited/index.md | 302 +++++++++ .../email-server-revisited/microsoft-bounce.png | Bin 0 -> 24184 bytes source/blog/2022/things-i-use/index.md | 156 +++++ source/blog/index.md | 30 + 6 files changed, 1648 insertions(+) create mode 100644 source/blog/2020/email-server-extras/index.md create mode 100644 source/blog/2020/email-server/index.md create mode 100644 source/blog/2022/email-server-revisited/index.md create mode 100644 source/blog/2022/email-server-revisited/microsoft-bounce.png create mode 100644 source/blog/2022/things-i-use/index.md create mode 100644 source/blog/index.md (limited to 'source/blog') diff --git a/source/blog/2020/email-server-extras/index.md b/source/blog/2020/email-server-extras/index.md new file mode 100644 index 0000000..b19819b --- /dev/null +++ b/source/blog/2020/email-server-extras/index.md @@ -0,0 +1,431 @@ +--- +title: "Setting up an email server in 2020 with OpenSMTPD and Dovecot: extras" +date: 2020-04-28 +layout: "blog" +toc: true +--- + +This sequel to my post +"[Setting up an email server in 2020 with OpenSMTPD and Dovecot](/blog/2020/email-server/)" +gives extra tips and tricks to extend your email setup. +See also the sequel's sequel, +"[Revisiting my email server in 2022](/blog/2022/email-server-revisited/)". + +Last updated on 2022-09-12. + + +## General + +### Multiple domains + +You can generalize your setup to handle multiple domains +with very little effort. In the following, +I'll assume that your two domains are called `foo.com` and `bar.com`. + + +#### DNS records + +There should be MX, SPF, DKIM and DMARC records for both domains, +as explained in the previous guide. Fortunately, these records +can have identical contents for both domains! + +However, it remains essential that the mail server's mailname +and reverse DNS domain name match up exactly, +so you should create MX records with that in mind. +Therefore, if the email server for both domains has `mx1.foo.com` +as reverse DNS name, the MX records should look like this: +```sh +foo.com. MX 42 mx1.foo.com. +bar.com. MX 42 mx1.foo.com. +``` +This is perfectly valid: the only thing that matters is that +what your SMTP server calls itself agrees with what reverse DNS +says that the server is actually called. + + +#### Dovecot + +To make Dovecot aware of multiple domains, +you only need to update the `/etc/dovecot/users` file +to add accounts for both domains. +However, in the original guide, I said to only write `user` +in the file, without the `@foo.com`, for an address `user@foo.com`. +Unsurprisingly, that isn't an option for multiple domains, +so you must put the full address in `/etc/dovecot/users`. + +Then update `/etc/dovecot/dovecot.conf` to reflect that, +by changing `%n` to `%u` in `username_format`: +```sh +userdb { + driver = passwd-file + args = username_format=%u /etc/dovecot/users + override_fields = uid=vmail gid=vmail home=/home/vmail/%d/%n +} +``` +Also note the change in the `home` setting: +the inbox of a user `user@foo.com` will now be stored +in `/home/vmail/foo.com/user`. +That's all you need to change. + + +#### OpenSMTPD + +To inform OpenSMTPD of all the domains, +create a new file `/etc/smtpd/domains`, +and in there put all desired names on their own line: +```sh +foo.com +bar.com +``` +And as I mentioned when discussing the DNS records, +you should check that `/etc/smtpd/mailname` agrees +with your server's reverse DNS. + +Then, in the main configuration file, tell OpenSMTPD to +use the new domains file when deciding whether to accept an message, +by declaring a new table and changing the `match` line for inbound mail: +```sh +table domains "/etc/smtpd/domains" +# ... +match from any for domain action "RECV" +``` + +#### Rspamd + +The last thing to do is to inform Rspamd of the multiple domains. +It's really easy: simply add multiple domain blocks: +```c +domain { + foo.com { + path = "/path/to/dkim/private.key"; + selector = "hello"; + } +} +domain { + bar.com { + path = "/path/to/dkim/private.key"; + selector = "world"; + } +} +``` + +### Advanced security + +SPF, DKIM and DMARC are email's traditional DNS-based security systems, +but in 2018 the IETF released [RFC 8460](https://tools.ietf.org/html/rfc8460) and [RFC 8461](https://www.rfc-editor.org/rfc/rfc8461.txt), +which respectively define TLSRPT and MTA-STS, +two fancy new systems focused on TLS-encrypted email transport. + +These security mechanisms are pretty new, +so you won't get a huge benefit from enabling them, +but big email providers' draconian spam filters might like it. + + +#### TLSRPT + +TLS reporting, or TLSRPT for short, is very simple: +all it does is provide a contact email address in case +somebody has trouble with the TLS configuration of your SMTP server. + +To enable it for your custom email domain `example.com`, +simply create a DNS TXT record for the `_smtp._tls` subdomain: +```sh +_smtp._tls.example.com. TXT "v=TLSRPTv1; rua=mailto:" +``` +Where `` is an email address of your choosing. +That's all! + + +#### MTA-STS + +MTA Strict Transport Security (MTA-STS) tells other servers +that you take TLS encryption of messages very seriously, +so they should avoid sending you unencrypted email, +and should only accept certain certificates from your side. + +Compared to the previously discussed DNS-based security extensions, +MTA-STS is a bit more work to set up, +because you'll also need an HTTP web server. + +The DNS part is still pretty simple: +create yet another DNS TXT record, +this time for the subdomain `_mta-sts`: +```sh +_mta-sts.example.com. TXT "v=STSv1; id=" +``` +The `` should identify the version of your policy, +so other servers can quickly see if something changed. +I recommend using today's date. + +For the next part, I'll assume that you already have +a web server running on a server with the IP address `1.2.3.4`. +I use [nginx](https://nginx.org/) for this, running +on the same server as OpenSMTPD and Dovecot, +but you don't have to do the same. + +Create an A record which binds your server +to the subdomain `mta-sts` (without underscore): +```sh +mta-sts.example.com. A 1.2.3.4 +``` +Set your web server to serve the file +`https://mta-sts.example.com/.well-known/mta-sts.txt` +(we'll discuss that file in a moment). +Note that this policy file **must** be served over HTTPS, +so you need a valid TLS certificate for that domain. + +The contents of the `mta-sts.txt` policy file are as follows, +where `mx1.example.com` and `mx2.example.com` are the hosts +mentioned in `example.com`'s DNS MX records: +```sh +version: STSv1 +mode: enforce +mx: mx1.example.com +mx: mx2.example.com +max_age: +``` +All MX servers must be mentioned this way. +If you're feeling cautious, you may want to set +`mode` to `testing` in the beginning. +This policy is valid for `` seconds, +which is recommended to be several weeks, +but to start with, I suggest using 86400 seconds (one day). +Finally, ensure that this file has CRLF Windows-style line endings. + +To correctly pass an MTA-STS test, the TLS certificate +presented by e.g. `mx1.example.com` should be valid for `mx1.exaple.com`. +To achieve this without needing to manage too many certificates, +you can specify multiple domains when requesting a certificate, +or you can use a wildcard domain (`*.example.com`). +Note, however, that MTA-STS testing tools don't like +the latter option, so I recommend the former. + +Once you're done, check your work by using either +[ESMTP](https://esmtp.email/tools/mta-sts/)'s or [Ayke](https://aykevl.nl/apps/mta-sts/)'s +online MTA-STS validation tools, +ignoring any warnings about DNSSEC or DANE. +If all is good, great! + +Even if you did everything correctly, +these tools will warn you that you're not using DNSSEC/DANE. +It might then be tempting to set that up for even more security, +but I recommend against that for private servers: take a look at [this](https://dane.sys4.de/common_mistakes). + + + +## OpenSMTPD + +### Client certificates (in addition to passwords) + +You can configure OpenSMTPD to request a client certificate +for sending emails, as a second factor for authentication. + +UPDATE: When I wrote this two years ago, it worked, +but now it doesn't anymore, and I can't figure out why. +It seems OpenSMTPD always rejects the client certificates for being self-signed, +even if they can manually be verified for our CA using the `openssl` tool. +I'm leaving this tutorial here for anyone who's interested, +but it's unlikely I'll fix it anytime soon. + + +#### Certificates + +We need to start with some cryptography to create and verify certificates. +I recommend that you do all of this on your trusted *client* device, +and only copy the necessary files to the server later. + +DISCLAIMER: +All the keys and certificates that we'll generate in this section are +for **private use** only, to handle a small number of trusted clients. +I'm not a cryptography expert, so you should **not** listen to me +for large-scale systems that may involve untrusted devices. + +The first step is to set up a private Certificate Authority (CA), +which issues the client certificates and can be used to verify them. +Start by generating an RSA private key, +which you should store in a safe place and not share with anyone: +```sh +$ openssl genrsa -out mailca.key 2048 +``` +Extract a public certificate from this key as follows. +Because we're lazy, we give it a lifetime of 36500 days: +```sh +$ openssl req -new -x509 -days 36500 -key mailca.key -out mailca.crt +``` +When running this command, OpenSSL will ask you some questions +about who this certificate is intended for. +Since this is for personal use, your answers don't matter, +so just use the defaults. +Some fields (I think only *Country Name* and *Organization Name*) +cannot be empty, but the others can. + +Moving on to the client, once again generate an RSA private key: +```sh +$ openssl genrsa -out mailclient.key 2048 +``` +From this private key, create a Certificate Signing Request (CSR) +as follows, where you'll be asked the same questions as before: +```sh +$ openssl req -new -key mailclient.key -out mailclient.csr +``` +By feeding this CSR to the CA, we can create a signed client certificate +that can be verified using the CA's public certificate. +```sh +$ openssl x509 -req -in mailclient.csr -out mailclient.crt \ + -days 36499 -CA mailca.crt -CAkey mailca.key +``` +If you want to multiple client certificates, +just repeat the last few steps for each one. + + +#### Server + +OpenSMTPD needs to verify the validity of client certificates +using the CA's public certificate, so you should copy that +to somewhere on the server, e.g. `/etc/smtpd/mailca.crt`, +and declare it to OpenSMTPD by adding this near +the top of `/etc/smtpd/smtpd.conf`: +```sh +ca "mailca" cert "/etc/smtpd/mailca.crt" +``` +Then replace the entire configuration for outbound mail as follows. +Note that this removes SMTPS support, leaving only STARTTLS: +```sh +# Outbound +listen on eth0 port 587 tls-require verify pki "example.com" ca "mailca" auth filter "rspamd" +action "SEND" relay srs +match from any auth for any action "SEND" +``` +The magic word here is "`verify`", which tells OpenSMTPD +to ask for a client certificate and to verify it using the given CA. + + +#### Client + +Now you won't be able to send emails if your client doesn't +present its certificate to the server! +Unfortunately, not all mail clients support this; personally +I use [Thunderbird](https://www.thunderbird.net/) with success. +I won't include any client-specific configuration here, +but I will say this: + +For some clients (like Thunderbird), you'll have an easier time +importing your client certificate if you encode it in the +[PKCS #12](https://en.wikipedia.org/wiki/PKCS_12) storage format: +```sh +$ openssl pkcs12 -export -in mailclient.crt -inkey mailclient.key \ + -certfile mailca.crt -out mailclient.pfx +``` +OpenSSL will ask you to set a password, which you'll need to +enter again when importing the certificate into the client. + + + +### ~~Client certificates (instead of passwords)~~ + +UPDATE: Don't do this. +As said above, OpenSMTPD's certificate verification is a mystery, +so for all I know, if you follow the instructions in this subsection, +you might find yourself running an *open* SMTP relay! +That would be bad, because anyone on the Internet +could send emails through your server with zero authentication. +In theory, the client certificates act as authentication, +but, again, the verification process is mysterious, +so I'm just not confident enough to say. + +If you really want to, you can use the client certificates +as a substitute for passwords. This is especially useful +if you set up a catchall inbox in Dovecot, +because this will allow you to send emails +from arbitrary addresses from your domain. + +To do this, follow the same procedure as in the previous section, +but with a slightly different OpenSMTPD configuration: +```sh +listen on eth0 port 587 tls-require verify pki "example.com" ca "mailca" filter "rspamd" tag "VALID" +action "SEND" relay srs +match from any tag "VALID" for any action "SEND" +``` +All incoming connections that present a good certificate +will be tagged as being `VALID`, and their mail will be relayed. + +Unfortunately, we're not quite done yet here, +because Rspamd is now very confused... + + +#### Rspamd + +When OpenSMTPD passes a message through Rspamd, it also includes +some metadata, most notably whether the sender has authenticated +successfully with OpenSMTPD... which is now no longer the case +for submissions, because we've removed the `auth` directive! + +Rspamd therefore starts regarding these outgoing emails +as *incoming* emails, because they don't seem +to come from a trusted user. So instead of signing them with DKIM +and handing them back to OpenSMTPD, it will do a full spam scan. +If they get a high spam score (which is likely for short test emails), +*your* spam filter, running on *your* server, +will be flagging *your* messages as spam! + +The solution is to whitelist your domain(s) in Rspamd, +so it won't scan them. To do this, create a new file +`/etc/rspamd/local.d/settings.conf` with these contents, +where `foo.com` and `bar.com` are the domains to whitelist: +```c +outbound { + priority = high; + from = "@foo.com"; + from = "@bar.com"; + apply { + actions { + add_header = 1000; + } + } +} +``` +Setting `priority` to `high` ensures that Rspamd checks +this rule before doing anything else. +You can add any number of `from` directives; +this rule will be applied if any of them match. +It only sets the threshold for the action `add_header` to `1000`. +That is, if the email doesn't get a spam score of at least 1000 +(the default is 6) Rspamd will not add any spam tags. + +Because Rspamd is still regarding your emails as inbound, +you also need to change the global settings of +the DKIM signer in `/etc/rspamd/local.d/dkim_signing.conf`, +such that they include the following: +```c +sign_inbound = true; +allow_hdrfrom_mismatch = true; +allow_username_mismatch = true; +``` +This tells Rspamd to add DKIM signatures to incoming emails, +which in this case includes yours. +Allowing these mismatches ensures that the messages still get signed, +even if you're sending from an arbitrary address. + + + +## Dovecot + +### Catchall inbox + +In Dovecot, you can create a catch-all inbox that will accept all +emails sent to your domain that don't match anyone in `/etc/dovecot/users`. +Just add another `userdb` block *after* the first: +```sh +userdb { + driver = static + args = uid=vmail gid=vmail home=/var/vmail/catchall allow_all_users=yes +} +``` +The `static` driver means there is no table file: +all configuration is directly within this `userdb` block. +If we don't specify `allow_all_users=yes`, then Dovecot +will check whether users exist using the `passdb` table, +and will conclude that the recipient is invalid. + + + diff --git a/source/blog/2020/email-server/index.md b/source/blog/2020/email-server/index.md new file mode 100644 index 0000000..962e6aa --- /dev/null +++ b/source/blog/2020/email-server/index.md @@ -0,0 +1,729 @@ +--- +title: "Setting up an email server in 2020 with OpenSMTPD and Dovecot" +date: 2020-04-27 +layout: "blog" +toc: true +--- + +So, you want to set up your own email server? In that case, welcome. + +There are many reasons to run a custom email server, +ranging from privacy concerns about providers like Google, +to just wanting to do it for fun and/or learning. +Since you're here, I assume you've already found a reason. + +Beware: this is a messy topic, and the available documentation +is even messier, so it could take a while before you get it to work properly. +I've compiled this guide according to my experiences +in an attempt to make this dark art more accessible, +but your mileage may vary considerably. I hope you find it useful. + +This guide is aimed at people who are comfortable with +the Linux/*BSD command line. + +When you're done (if you get that far), take a look at the sequels +"[Setting up an email server in 2020 with OpenSMTPD and Dovecot: extras](/blog/2020/email-server-extras/)" +and "[Revisiting my email server in 2022](/blog/2022/email-server-revisited/)" +for ideas on how to extend your setup. + +Last updated on 2022-09-12. + + + +## Preparation + +Setting up email is relatively complex compared to e.g. a static website, +because you need to configure not one, but *two* server programs, +*and* you need to shoehorn modern security features into email's Stone-Age design. +I'll start by explaining the general structure of a mail server setup. + + + +### How email works + +The programs involved in the exchange of emails are called [agents](https://en.wikipedia.org/wiki/Email_agent_(infrastructure)). +Officially, there are 5 different types of agent: MUA, MSA, MTA, MDA and MRA. +But fortunately, it's reasonable to treat the MRA and MSA +as being part of the MUA and MTA, respectively. + +The *Mail User Agent* (MUA) is simply the client on your device at home +that you use to send and receive emails, and this guide assumes +you already have a favourite program for this, e.g. [Thunderbird](https://www.thunderbird.net/en-US/). +Nowadays it's fashionable to use a web interface for emails, +but that's also beyond the scope of this guide. + +The *Mail Delivery Agent* (MDA) is a program that watches over +the server's copy of your mailbox: it manages your inbox, +remembers which messages you have or haven't read, +keeps a copy of your drafts, etc. +When you open your mailbox, your MUA will connect to +your server's MDA using the [IMAP](https://en.wikipedia.org/wiki/Internet_Message_Access_Protocol) protocol +(or [POP3](https://en.wikipedia.org/wiki/Post_Office_Protocol), but that one's [obsolete](https://pop2imap.com/)). + +The *Mail Transfer Agent* (MTA) is responsible for +making messages arrive at the right destination. +When you send an email, your MUA will pass it on to your server's MTA, +which will in turn pass it on to the recipient's mail server. +Likewise, when someone sends *you* an email from another server, +the MTA will receive it and hand it over to the MDA so you can read it later. +In both cases the MTA speaks the [SMTP](https://en.wikipedia.org/wiki/Simple_Mail_Transfer_Protocol) protocol. + +In this guide our MDA will be [Dovecot](https://dovecot.org/), +which is a very popular choice for that role. +As for the MTA, there exist several options, +the most popular being [Postfix](http://www.postfix.org/) and [Exim](https://exim.org/). +However, this guide uses the newer, lesser-known [OpenSMTPD](https://opensmtpd.org/), +which in my experience is *much* easier to set up: +Postfix and Exim have complex configurations and +are geared towards large-scale email providers, +whereas OpenSMTPD is more beginner-friendly. + + + +### Security + +The base email system is horribly insecure on its own, +so we still need to duct-tape on some security features. +In this context, "security" has two meanings: +spam protecion and privacy protection (encryption). + +Spam protection also means two things here: +defending yourself against spammers, and +preventing that *your* emails get flagged as spam. +The former is optional, but the latter is not: +big providers such as Google and Microsoft +use infamously strict spam filters, +and if they decide that your server is a spammer, +there's almost nothing you can do about it. +Spam protection techniques will be discussed +in more detail over the course of this guide. + +Privacy protection is important in the 21st century: +you don't want a random router in the Internet to read all your emails, +which may contain sensitive information such as +private conversations and account password reset links. +You should therefore try to make sure that emails are +transported over an encrypted channel. +To do this, you have two options for encryption: +*mandatory* and *opportunistic* encryption. + +Mandatory encryption is only practical for client-server +communication (not server-server), and is provided by IMAPS and SMTPS, +which wrap the IMAP and SMTP protocols in [TLS](https://en.wikipedia.org/wiki/Transport_Layer_Security), +in the same way that [HTTPS](https://en.wikipedia.org/wiki/HTTPS) does for [HTTP](https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol). + +For server-server communication, the only option is +opportunistic encryption in the form of [STARTTLS](https://en.wikipedia.org/wiki/STARTTLS), +where communication is only encrypted if both parties agree +after a short unencrypted discussion. +That last part is vulnerable to [MitM](https://en.wikipedia.org/wiki/Man-in-the-middle_attack) attacks, +where anyone along the path of the email servers' discussion +can alter the exchange to block the use of encryption, +which sometimes actually [happens](https://www.eff.org/deeplinks/2014/11/starttls-downgrade-attacks) in practice. + +The only way to make sure that STARTTLS is used in that case +is to refuse any exchange unless the servers agree to use encryption. +Unfortunately, that's a risky approach that I can't recommend, +because not all servers support encryption (unbelievable, right?). +For example, I've received airline booking confirmations, +full of personal details, and made with billion-dollar companies, +sent across the Internet without any protection. + +This guide includes intructions to enable encryption, +but assumes that you already have a TLS certificate for that. +If not, find a guide to get one from [Let's Encrypt](https://letsencrypt.org/) (it's free!), +and remember that you'll need to renew it every few months. +Using a self-signed certificate *may* work, but I don't recommend it. + +In the rest of this guide I'll assume that +you have a public full-chain TLS certificate at `/etc/ssl/certs/example.com.pem`, +and a private encryption key at `/etc/ssl/private/example.com.pem`. + + + +### Server + +Obviously, you'll need a server to run the MTA and MDA on. +You can host your own at home, but the more reliable option is +to rent one in a data center ([VPS](https://en.wikipedia.org/wiki/Virtual_private_server)). +This guide was written with a Linux server in mind, +but in theory it should also work on the BSDs +([OpenBSD](https://www.openbsd.org/), [FreeBSD](https://www.freebsd.org/), +[NetBSD](https://www.netbsd.org/), etc.) with minimal adaptation. + +The server must be online 24/7, you must have root SSH access, +it must have a static IP address, and TCP network port 25 must be open. +Especially check that last one: you may need to explicitly ask +your home ISP or the server provider to enable port 25, +because they often close it to prevent spam. +You can usually do this from their web interface. + +You also must have a domain name, which I'll call `example.com`. +This will be necessary for basically everything: +DNS records, TLS certificate, MTA network configuration, etc. +If you don't have one yet, you can choose between many registrars +to rent one from. Personally I use and can recommend [Gandi](https://www.gandi.net/). + +Note that it's a **bad** idea to use a domain like `foo.bar.com`, +where you control the `foo` part but *not* the `bar` part: +in that case, a spammer in control of `qux.bar.com` +could negatively affect *your* reputation +in the eyes of other email providers. + +Lastly, when setting up an email server, you also have the choice +between using to *system* users or *virtual* users. +With system users, if an email arrives for `john@example.com`, +then the MTA and MDA will expect that there exists +a `john` Unix user on the server to deliver it to. +With virtual users, you have much more flexibility, so that's what we'll use. +All email will be managed under a single Unix user/group called `vmail`. +Create it as follows: +```sh +# GNU CoreUtils: +$ groupadd vmail +$ useradd -g vmail vmail +# BusyBox: +$ addgroup vmail +$ adduser -D -G vmail vmail +# *BSD: +$ no clue, but it should be similar +``` + + +## DNS records + +Now we must set up all the necessary DNS records, which +is usually possible from the domain registrar's web interface. +It may take a while for your changes to propagate over the Internet, +so I recommend doing this section now and the rest tomorrow. + +Firstly, you should already have an A and/or AAAA record +to associate your domain `example.com` with the server's IP address. +For email it is **essential** that you also have [reverse DNS](https://en.wikipedia.org/wiki/Reverse_DNS_lookup) +set up correctly. If you're renting your server remotely, +you can often do this from the provider's configuration tool, +otherwise, you should create a PTR-type DNS record, +although that's beyond the scope of this guide. + +Once you're done, I recommend testing your DNS records +using the [MX Lookup](https://mxtoolbox.com/MXLookup.aspx) online tool. + + + +### MX + +To inform the rest of the Internet that your server is an email server, +create an MX (Mail eXchanger) DNS record for your domain. +Note the dot at the end of the domain name: +```sh +example.com. MX 42 example.com. +``` +When a message is sent to an email address ending in `@example.com`, +the sender will query DNS for any MX records for `example.com`. +There it will find a domain name (in this case `example.com` again), +for which it will look up the IP address using an A/AAAA record. +The domain name in the record must **not** have an associated CNAME record; +it must be directly translatable to an IP address. + +You may have multiple MX records, containing different domain names, +each with a preference number (`42` in the example above). +The sender will try MX records with *lower* numbers first, +and if that server is unavailable, it will try a higher number. +If you have multiple mail servers (which is a good idea), +you can thus declare those as follows: +```sh +example.com. MX 13 mx1.example.com. +example.com. MX 42 mx2.example.com. +``` +Here, a server sending an email to your domain `example.com` +will try to send it to the IP address of `mx1.example.com` first, +and if that fails, it will move on to `mx2.example.com`. +If both `mx1` and `mx2` have the same number, then the sender +will randomly choose one, which is useful for load balancing, +although that's probably overkill for a private server. + + + +### SPF + +The [Sender Policy Framework](https://en.wikipedia.org/wiki/Sender_Policy_Framework) (SPF), +is a feature which helps prevent spammers from impersonating +your server in an attempt to get around blacklists. +This security feature is **required** nowadays: +if you don't use it, you'll probably get flagged as spam. + +SPF works by specifying which IP addresses are authorized +to send emails from your domain name. +You must publish this information in a +TXT-type DNS record (**not** SPF-type, which also exists!) with the following contents: +```sh +example.com. TXT "v=spf1 mx -all" +``` +Everything after the version `v=spf1` is a list of *verification mechanisms* +for a spam filter to try out in the given order. +The `-all` at the end says to reject your email +if all of the previous mechanisms fail. +See the [SPF spec](https://tools.ietf.org/html/rfc7208) for details. + +I recommend only using the `mx` mechanism, which tells the verifier +to look at the A/AAAA addresses of the domains in your MX records. +This allows you to add, remove, or change your servers +without needing to update this record. + + + +### DKIM + +Then we have [DomainKeys Identified Mail](https://en.wikipedia.org/wiki/DomainKeys_Identified_Mail) (DKIM), +which is a more comprehensive form of anti-impersonation, +and, like SPF, is practically **mandatory** in the modern era. + +It adds a cryptographic signature to all emails from your server, +which the receiver's spam filter will verify using the email's contents, +and a public key that you need to publish in a DNS record. +Again, you should implement *both* SPF and DKIM, despite their overlap. + +To set up DKIM, create an RSA keypair, using the `openssl` utility: +```sh +$ openssl genrsa -out /path/to/dkim/private.key 2048 +$ openssl rsa -in /path/to/dkim/private.key -out /path/to/dkim/public.key +``` +The minimum size is 1024 bits, but I recommend 2048 bits. +Bigger is better, but because DNS is involved you can't stretch it +to 4096 bits without causing discomfort to [some](https://serverfault.com/questions/747185/dkim-can-i-use-a-rsa-key-larger-than-2048bit-i-e-4096) servers. +And I think it goes without saying that you should keep the private key private. + +Importantly, the DKIM DNS record **cannot** be +attached directly to your domain `example.com`; +instead, it should belong to a subdomain of the form +`._domainkey.example.com`, +where `` is an alphanumeric string you can choose (e.g. today's date), +just remember your choice for later when configuring the DKIM signer. +And if you change your key, keep the old record around +for a while so old emails can still be verified. + +Your DKIM policy must be published in a TXT record +as follows, where `` is the public RSA key `MI...AB` +stored in `/path/to/dkim/public.key`, with the newlines removed: +```sh +._domainkey.example.com. TXT "v=DKIM1; t=s; h=sha256; p=" +``` +Here, `v=DKIM1` is the version and must be the first tag. +The flag `t=s` enables strict mode, as recommended by the [DKIM spec](https://tools.ietf.org/html/rfc6376), +meaning that emails sent from subdomains immediately fail verification. +The optional tag `h=sha256` blocks the use of the old SHA1 algorithm. + + + +### DMARC + +Lastly, we have [Domain-based Message Authentication, Reporting and Conformance](https://en.wikipedia.org/wiki/DMARC) (DMARC), +which is *technically* optional, but *highly* recommended, +because it will make you look more legitimate in the eyes of Google and Microsoft. +It can modify the behaviour of SPF and DKIM, +and also provides advice about what a receiver should do +if one of your emails fails verification. + +To enable it, create yet another TXT record, which, +similarly to DKIM, **must** belong to the subdomain `_dmarc.example.com`, +and give it the following contents, +where `` is an email address of your choosing, +which may or may not belong to your domain: +```sh +_dmarc.example.com. TXT "v=DMARC1; p=reject; sp=reject; pct=100; aspf=s; adkim=s; fo=1; ruf=mailto:" +``` +The version tag `v=DMARC1` must come first, +followed by `p=` and `sp=`, which control what to do to unverified messages +coming from the main domain and subdomains, respectively. +Here, `reject` means that delivery should be refused, +`none` asks to let it through anyway, and `quarantine` tells +the filter to take a closer look or to put it in a spam folder. +The percentage `pct=100` says how many of your emails to apply the policy to. +Next, `aspf=s` and `adkim=s` enable strict mode for SPF and DKIM, +which blocks subdomains from passing. +Finally, `fo=1` asks for a forensic report if verification fails, +and `ruf=` gives an address to send it to. +If in doubt, see the [DMARC spec](https://tools.ietf.org/html/rfc7489). + + + +## MDA: Dovecot + +[Dovecot](https://www.dovecot.org/) is a very popular IMAP server, +focused on being lightweight, simple, and secure, +and has extensive and up-to-date documentation. +It's very flexible and scalable, and keeps up well +with the lastest security best-practices. + +If you installed Dovecot via a package manager, +you'll probably have lots of configuration files +in the `/etc/dovecot` directory. +I want you to delete all of them. Yes, `rm -rf` that crap. +Dovecot is simple to configure, and doesn't care where +you put its settings, so having all that chaos in `/etc/dovecot` +just makes things unnecessarily confusing. + + + +### Network + +Create a new configuration file `/etc/dovecot/dovecot.conf`, +and start by filling in the details of your TLS certificate, +making clear that unencrypted connections are unacceptable: +```sh +ssl = required +ssl_cert = ` could be anything you want, +not necessarily the same account name as in Dovecot: +```sh + +``` +Generate the password hash with this command for each user. +Again, the password doesn't need to be the same as in Dovecot, +but for your own convenience it's probably best if it is: +```sh +$ smtpctl encrypt '' +``` +Then, like with Dovecot, just delete the contents of +the main config file `/etc/smtpd/smtpd.conf`, +and start by putting in the following: +```sh +table aliases "/etc/smtpd/aliases" +table passwds "/etc/smtpd/passwds" +``` + + +### Network + +Write your domain name on a single line in `/etc/smtpd/mailname`. +This is the name that OpenSMTPD will use to introduce itself +to other servers, and it's important that this matches +the reverse DNS domain name of the server's IP: +```sh +example.com +# Or mx1.example.com or whatever +``` +Then continue in `/etc/smtpd/smtpd.conf` by importing your TLS certificate: +```sh +pki "example.com" cert "/etc/ssl/certs/example.com.pem" +pki "example.com" key "/etc/ssl/private/example.com.key" +``` +And tell OpenSMTPD which keys to use for the [Sender Rewriting Scheme](https://en.wikipedia.org/wiki/Sender_Rewriting_Scheme), +which prevents forwarded emails from breaking SPF and looking like spam: +```sh +srs key "" +srs key backup "" # optional, read below +``` +It's recommended to change the key every year or so, +but in that case you need to ensure that emails from the last month +can still be verified using the old one, so if you change it, +simply move it to the backup slot for a month. + +It's very important that the secrets can't be guessed, +otherwise *anyone* can send mail through your server, +so I recommend generating these keys randomly as follows: +```sh +$ head -c 30 /dev/urandom | base64 +``` +Next, define the spam filters as follows. +For the last line to work, you'll need to [download](https://github.com/poolpOrg/filter-rspamd) +and build the `filter-rspamd` adapter binary for yourself, +created by OpenSMTPD's official maintainer. +We'll set up the Rspamd service in the next section, +which will be responsible for spam filtering and DKIM signing: +```sh +filter "rdns" phase connect match !rdns disconnect "550 DNS error" +filter "fcrdns" phase connect match !fcrdns disconnect "550 DNS error" +filter "rspamd" proc-exec "/etc/smtpd/filter-rspamd" +``` +I cannot overstate the importance of the first two lines: +these will block hundreds of spam attempts, and have, +at least for me, never blocked anything legitimate so far. + +The only thing left to do here is to tell OpenSMTPD which ports to listen on +and what to do with the incoming traffic. +This comes in two parts: first, we listen on port 25 for incoming +messages for your domain coming from other email servers: +```sh +# Inbound +listen on eth0 port 25 tls pki "example.com" filter { "rdns", "fcrdns", "rspamd" } +action "RECV" lmtp "/var/run/dovecot/lmtp" rcpt-to virtual +match from any for domain "example.com" action "RECV" +``` +Line 1 says to listen on `port 25` of interface `eth0`, +providing *optional* `tls` using the certificate for `example.com`, +and passing everything through the three filters definer earlier. +Making TLS mandatory is a **bad** idea here, because not all servers can use TLS. + +Line 2 defines an action called `RECV`, +which relays an email to Dovecot's LMTP socket, +with the `rcpt-to` and `virtual` making sure +that Dovecot will actually accept the message. + +Line 3 then simply says that any incoming mail +for your domain should have the action `RECV` applied to it, +unless the spam filters rejected it. + +Secondly, we listen on port 465 (SMTPS) +and/or port 587 (STARTTLS) for messages getting sent +from your email client to the rest of the world, +so we require user authentication: +```sh +# Outbound +listen on eth0 port 465 smtps pki "example.com" auth filter "rspamd" +listen on eth0 port 587 tls-require pki "example.com" auth filter "rspamd" +action "SEND" relay srs +match from any auth for any action "SEND" +``` +The only difference between lines 1 and 2 is the port and the protocol. +Both demand a mandatory TLS connection, and that users authenticate +themselves according to the password file created earlier. +The `filter` at the end is for DKIM signing of outgoing mail. + +Lines 3 and 4 are only triggered after successful `auth`entication, +and, unsurprisingly, `relay` the email to its destination. +The `srs` at the end enables using the SRS settings from earlier. + +OpenSMTPD is the only program in this setup that needs to handle +untrusted connections, when other MTAs send you messages. +Since, like most server software, it may have [vulnerabilities](https://opensmtpd.org/security.html), +it is **very** important that you keep it as up-to-date as possible. + + + +## Spam filter: Rspamd + +[Rspamd](https://www.rspamd.com/) is a modern spam filtering solution, +with many features that you could spend hours tweaking. +In this guide, however, we'll keep it short, +because we're only really interested in its ability add +our server's DKIM signature to outgoing messages. + +Besides, according to my limited experience, +for small-scale servers spam filtering isn't essential, +as long as you tell OpenSMTPD to check reverse DNS as described above. +However, there are some much more experienced people who disagree with me. + +Basically, for our purposes, don't touch any of Rspamd's default +configuration except for creating/editing the file +`/etc/rspamd/local.d/dkim_signing.conf` with the following contents, +where `` is the DKIM selector you chose in the DNS record: +```sh +allow_username_mismatch = true; + +domain { + example.com { + path = "/path/to/dkim/private.key"; + selector = ""; + } +} +``` +Make sure that the DKIM `private.key` file is +readable (and *only* readable) by `rspamd:rspamd`. +Allowing username mismatches is necessary, +because OpenSMTPD will only tell Rspamd about `username` +while the DKIM signer actually expects `username@example.com`. + +And... that's it! Of course, don't forget to start all the necessary daemons. + + + +## Testing + +Everything is set up now, so it's time to test. Fingers crossed! + +As mentioned earlier, you can check the validity +of your DNS using the [MX Lookup](https://mxtoolbox.com/MXLookup.aspx) tool. +You can use the same website to test OpenSMTPD +with the [SMTP Diagnostics](https://mxtoolbox.com/diagnostic.aspx) tool. + +All decent email clients include an option to set the server for an email account. +This guide doesn't include instructions for that, +because it will vary a lot from client to client. +If asked for your login type, choose plain/normal/unencrypted passwords. +Don't worry, the client-server connection is TLS-encrypted, +so nobody will be able to steal it. + +Next, to verify that you can send and receive messages, +and that your SPF/DKIM/DMARC is working, +the following websites will be useful: + +* [~~Is my email working?~~](http://ismyemailworking.com/) + UPDATE: no longer available. +* [DKIM validator](https://dkimvalidator.com/): + by sending an email, checks that SPF and DKIM are set up correctly. +* [Email deliverability tool](https://mxtoolbox.com/deliverability): + tests basically everything. + As of writing, there's a bug that causes DKIM signatures to fail verification + even if you did it right; just ignore that. +* [Learn and test DMARC](https://www.learndmarc.com/): + checks SPF, DKIM and DMARC in detail. + +If everything is good so far, congratulations! + +Now comes the big scary final test: +sending an email to one of the "big guys", like Google, Yahoo or Microsoft. +Their spam filters are very strict, so if you get through, great! +Try to write your test message so that it doesn't look like spam, +otherwise there's no point to this exercise. + +If something failed, then you have some investigating to do. +Either one of the daemons is misconfigured, or there's a problem +with your domain name and/or DNS records. +Do some research, and you'll get there, don't give up. + +But if everything works, congratulations! +You're now the proud administrator of a private email server. +Have fun with it, and don't forget to update your +TLS certificate and DKIM and SRS keys. + +PS: and please don't spam; you'll ruin it for everyone else. + diff --git a/source/blog/2022/email-server-revisited/index.md b/source/blog/2022/email-server-revisited/index.md new file mode 100644 index 0000000..4519bae --- /dev/null +++ b/source/blog/2022/email-server-revisited/index.md @@ -0,0 +1,302 @@ +--- +title: "Revisiting my email server in 2022" +date: 2022-09-12 +layout: "blog" +toc: true +--- + +More than two years have passed since my first post about +[setting up an email server in 2020 with OpenSMTPD and Dovecot](/blog/2020/email-server/) +and [its sequel](/blog/2020/email-server-extras/). +Since then, my server been going strong, with a few minor hiccups along the way. +In this post, I'll explain some of the changes I made. + +This assumes you've followed the preceding guides: +the configuration snippets given here +should be interpreted as modifications to those guides, +*not* as complete setups! + +Last updated on 2022-09-12. + + + +## More about DMARC + +When I wrote my original guide, +I didn't properly understand how DMARC works: +I misinterpreted it as an optional wrapper around SPF and DKIM. +But oh God, I was wrong. +Simon Andrews' article "[I figured out how DMARC works, and it almost broke me](https://simonandrews.ca/articles/how-to-set-up-spf-dkim-dmarc)" +showed me the ugly truth, +and I highly recommend reading it. +Briefly, DMARC does two arguably unrelated things. + +Firstly, DMARC provides a way to diagnose issues with your SPF and DKIM configurations, +in the form of reports that get sent to the `ruf=` and/or `rua=` email address(es) +you put in the DNS record. +Without this, there's no way of knowing why +your emails are getting marked as spam. + +Secondly, it improves the trustworthiness of SPF and DKIM by enforcing *alignment*. +This means something slightly different for SPF and DKIM, +and boils down to fixing a glaring issue: + +* For some reason, in vanilla SMTP, it turns out that the email's `From:` header + doesn't need to agree with the address in the SMTP `MAIL FROM` command; + in other words, the server can claim a different sender than what's written in the message's header. + SPF only verifies the former (i.e. it takes the domain in `MAIL FROM`), + so one SMTP server can impersonate another. +* An email's DKIM signature header states the domain of the signing server with the `d=` tag, + but, once again, that doesn't need to agree with the `From:` header's domain. + DKIM doesn't look at the latter, so an SMTP server can validly sign impersonated messages. + +DMARC's alignment refers to checking whether the domains match up for SPF and DKIM, +thus ensuring that an SMTP server can't pretend to be someone else. +It sounds obvious, but nope, apparently it wasn't before DMARC was made. + + + +## DKIMproxy + +To add DKIM signatures to my messages, +I switched from Rspamd to [DKIMproxy](http://dkimproxy.sourceforge.net/). + + + +### Motivation + +To sign outgoing emails for DKIM, my original guide used Rspamd --- +an unusual choice, since it's a spam filter designed to act on *incoming* messages. +Later, in the [sequel](/blog/2020/email-server-extras/) to that guide, +I needed some ugly workarounds to compensate for Rspamd's "smartness" +when I tried to play around with authentication schemes in OpenSMTPD. +Clearly, this wasn't ideal. + +However, the reason I cut my losses and switched to another DKIM signer +was actually a bug in +MXToolBox' [deliverability tool](https://mxtoolbox.com/deliverability). +It appears that no matter what you do, +this tool claims that your email's signature fails validation. +I'm not the first to notice this issue: +see e.g. [this question](https://serverfault.com/questions/1005818/dkim-validating-but-mxtoolbox-reports-as-dkim-signature-not-verified) on Server Fault. +Other tools like the [DKIM validator](https://dkimvalidator.com/) +say that my DKIM signatures are correct. + +There aren't many open-source alternatives out there for DKIM signing: +the only ones I know of are [OpenDKIM](http://www.opendkim.org/) +and DKIMproxy. +The former is a so-called "milter", +meaning it can only interact with MTAs via the milter API, +which is only supported by [Sendmail](https://www.proofpoint.com/us/products/email-protection/open-source-email-solution) +and [Postfix](https://www.postfix.org/). +Since we're using OpenSMTPD, +our only option is DKIMproxy, +which consists of two daemons: +`dkimproxy.out` to sign outgoing mail, +and `dkimproxy.in` to verify incoming mail. +We just need the former; +Rspamd is still convenient for handling the latter's functionality. + + + +### DKIM settings + +Let's start by disabling Rspamd's DKIM signer +in `/etc/rspamd/local.d/dkim_signing.conf`: +```sh +enabled = false; +``` +Then configure `dkimproxy.out` as follows +in `/etc/dkimproxy/dkimproxy_out.conf`. +If you placed your DKIM public key +in a TXT DNS record for `._domainkey.example.com.`, +and stored your private key in `/path/to/dkim/private.key`, +then: +```sh +# Receive emails on 10027, sign them, and forward them to 10028 +listen 127.0.0.1:10027 +relay 127.0.0.1:10028 + +# Settings for email signing +domain example.com +signature dkim(c=relaxed/relaxed,a=rsa-sha256) +keyfile /path/to/dkim/private.key +selector +``` +Here, `rsa-sha256` is the signature algorithm +(this is the best available, because DKIM is ancient), +and `relaxed/relaxed` is the so-called *canonicalization* method, +which is applied before signing and verification, +to prevents failures if e.g. the email's whitespace gets changed in transit. + + + +### OpenSMTPD settings + +OpenSMTPD needs to send all outbound mail through `dkimproxy.out`. +In `/etc/smtpd/smtpd.conf`, we tell it that all emails coming from the MUA +must be relayed through `localhost:10027`, and then, after DKIM signing, +picked up again on `localhost:10028`: +```sh +# Outbound +listen on eth0 port 465 smtps pki "example.com" auth tag "TRUSTED" +listen on eth0 port 587 tls-require pki "example.com" auth tag "TRUSTED" +action "SIGN" relay host "localhost:10027" +match from any tag "TRUSTED" for any action "SIGN" + +listen on lo port 10028 tag "SIGNED" +action "SEND" relay srs +match from any tag "SIGNED" for any action "SEND" +``` +The tag name `TRUSTED` reflects that only messages from trusted +(i.e. authenticated) MUAs should be signed. +After signing, emails get the tag `SIGNED`, +and are sent to their destination as usual. + + + +## SMTP relay + +Instead of sending my emails directly to their destinations, +I now send them to an SMTP relay server, +which then passes them on to their actual destinations. + + + +### Motivation + +Large email providers such as Google, Microsoft and Yahoo +manage many user accounts, so for them it makes sense +to keep track of IP-based sender reputations. +For example, if a number of low-quality emails are sent from a single IP +to many of the accounts they manage, +it's cheaper to simply blacklist that IP entirely at the MTA level, +rather than passing each message through a computationally-intensive spam filter. + +But, as usual, Microsoft has to ruin everything with their draconic policies. +In a stroke of genius, +someone there decided to blindly ban IPs, +seemingly in blocks belonging to VPS providers. +One day, I tried to send an email to an Outlook-based account, +and OpenSMTPD reported it had been unable to make the delivery, +because Microsoft had thrown an error: + + + + + +To their credit, they seem to be offering a way out. +This approach is reasonable: preventively ban high-risk IP ranges, +and allow "trustworthy" servers at the owner's request. +I got error 5.7.511, asking me to send an email to a support address. +If you're lucky, you may have a different error, +and get the opportunity to use the slick [delist portal](https://sender.office.com/) instead. +The URL in the bounce message links to [this list](https://go.microsoft.com/fwlink/?LinkId=526653) of error codes. + +I confess, I never actually bothered to forward the message to the provided address: +my initial email was time-sensitive, +so I couldn't afford to wait for Microsoft's response. +Also, their customer support's stellar reputation precedes them, +so I chose to use my time more wisely. +Even if they would've resolved it nicely, +there's nothing preventing Microsoft (or any other provider) +from breaking my deliverability again in the future. +Instead, I opted for a compromise. + +As a result of providers' IP reputation systems, +a whole new business has appeared: SMTP relays. +They offer to take the issue out of your hands: +you send your emails through their servers, +and they do their best to deliver them to large providers. +SMTP relays are mostly used for sending marketing emails in bulk, +but are also useful to avoid small-scale problems as described above. +There are many SMTP relay services to choose from, at various prices. + +Using an SMTP relay results in more reliable delivery of your messages to large providers, +but an obvious concern is privacy: +the relay server can read all your outgoing emails (but not incoming), +so you'll have to trust the service you choose. +But it's no worse than using a major provider, +and if you're sending sensitive material, why use email in the first place? +Personally, I use [SMTP2GO](https://www.smtp2go.com/), +but I can't say how good they are; do your own research. + + + +### OpenSMTPD settings + +Once you've chosen an SMTP relay service, let's say `relay.com`, +and set up your account, +they'll let you create credentials to use their SMTP servers. +Suppose these credentials are `` and ``, +create a file `/etc/smtpd/relaypw` with contents: + +```sh +