+++ title = "How to set up an email server in 2019" +++ # How to set up an email server in 2019 This is a guide to set up your own email server in 2019. This is a messy topic, and the available documentation is even messier, so I compiled this guide in an attempt to make it all more accessible. So why run your own mail server? There are many reasons, ranging from privacy concerns about popular providers, to simply wanting to learn about this stuff for fun. If you've arrived here, then I assume you've already found a reason. But beware that this is not for the faint of heart: email is a horribly designed system, and even if you manage to set up your own server, you may find that GMail flags all your messages as spam, with little you can do about it. This guide tries to implement all the security measures that GMail seems to like, but *your mileage may vary*. ## How email works The program on your device that you use to read and send emails contains both a Mail User Agent (MUA) and a Mail Retrieval Agent (MRA), although in practice the MRA is also often referred to as an MUA. These mail agents can be in a dedicated app like [Thunderbird](https://www.thunderbird.net/en-US/) or Microsoft Outlook, or it can simply be a web interface. When you check your inbox, your MRA asks for any new emails from the server, which then sends them via the [IMAP](https://en.wikipedia.org/wiki/Internet_Message_Access_Protocol) protocol (we ignore [POP3](https://en.wikipedia.org/wiki/Post_Office_Protocol), since it is rather [outdated](https://www.pop2imap.com/)). The server-side program here is simply called the IMAP server, which watches over the server's copy of your inbox. When you send an email, your MUA passes it along to the server's Mail Submission Agent (MSA), which then immediately hands it over to the Mail Transfer Agent (MTA) running on that same server. In practice, the MSA's functionality is often regarded as part of the MTA. The MTA then works out where your email needs to go, and sends it over a protocol called [SMTP](https://en.wikipedia.org/wiki/Simple_Mail_Transfer_Protocol) to the MTA of the server responsible for the recipient of your message. In complex networks there may be multiple MTAs in a chain. The final receiving MTA then hands over your email to the Mail Delivery Agent (MDA) running on that same server. The MDA checks whether it's spam, and then writes it to the server's storage, which is being watched over by an IMAP server, thus completing the cycle. Most MTA programs also have MDA features for simplicity, although in this guide we'll be using our IMAP server's MDA helper. Long story short, the server we want to set up must at least be running both an MTA for you to send emails and an IMAP server for you to check your inbox, one of which should also include MDA functionality. ## Email security This messy base email system is horribly insecure on its own, so we still need to tack on some more messy features. In this context, "security" refers to both spam and privacy protection. This guide covers all techniques mentioned here. Spam protection also means two things: defending yourself against spammers, and preventing that your emails' recipients' filters don't flag *you* as a spammer. The former is optional, but the latter is not. Let's start with how to come across as a trustworthy mail server: The first feature is [Sender Policy Framework](https://en.wikipedia.org/wiki/Sender_Policy_Framework) (SPF), which stops a spammer from filling in your email address in the "sender" field of their message. You do this by publishing a policy in a DNS record for how your domain should relate to the IP address of your email server. A filter enforcing SPF will use that to check that your server is authorized to send messages with your email address. Then there's [DomainKeys Identified Mail](https://en.wikipedia.org/wiki/DomainKeys_Identified_Mail) (DKIM), which is a more comprehensive form of SPF. It adds a cryptographic signature to all your emails, which the receiver's spam filter will check against the contents and a public key you need to publish in a DNS record. You should implement *both* SPF and DKIM, despite their overlap. Lastly, we have [Domain-based Message Authentication, Reporting and Conformance](https://en.wikipedia.org/wiki/DMARC). which uses another DNS record to specify whether you're using SPF and/or DKIM, and whether super- and subdomains should be regarded as valid for authentication. It also gives a contact address to which the receiver should send failure reports. I highly recommend implementing this too. To defend yourself against spammers your server should enforce the SPF, DKIM and DMARC policies of senders, in addition to some other checks. All of that will be handled by your spam filter. There's one final problem to bring up: your emails will be sent across the Internet unencrypted. For client-server communication, this is effectively solved by IMAPS and SMTPS, which wrap IMAP and SMTP in [TLS](https://en.wikipedia.org/wiki/Transport_Layer_Security). To be clear, SMTPS only encrypts email submission by an MUA to an MTA (MSA). For server-server relaying of messages over SMTP, we have STARTTLS, which is a system for opportunistic TLS, meaning that communication is only encrypted if both parties agree after a short unencrypted discussion. That last part is vulnerable to [MitM](https://en.wikipedia.org/wiki/Man-in-the-middle_attack) attacks, where anyone along the path of the email servers' discussion can alter the exchange to block the use of encryption, which sometimes actually [happens](https://www.eff.org/deeplinks/2014/11/starttls-downgrade-attacks) in practice. The only way to make sure that STARTTLS is used is to refuse any emails unless the TLS handshake succeeds. That's a risky approach, because fewer mail servers support STARTTLS than you might expect or hope: I've had airline booking confirmations, full of personal details, and made with multi-billion-dollar companies, sent across the Internet without any protection. I'll show you how, but do it at your own risk. ## Preparation Are you still here? Good, let's get started. Here's what you'll need: * SSH root access to a 24/7-accessible server running Linux or *BSD * an associated domain name for which you can create or edit DNS records * a TLS certificate for encryption The TLS certificate is technically optional, but you *really* should use one; it's 2019 for goodness' sake. You may get away with using a self-signed certificate, but instead I highly recommend getting a proper one from [Let's Encrypt](https://letsencrypt.org/) (it's free!). In the rest of this guide I'll assume that you have your public TLS certificate at `/etc/ssl/certs/example.com.pem`, and your private encryption key at `/etc/ssl/private/example.com.pem`. You should also create a file with [DH parameters](https://weakdh.org/sysadmin.html) using the `openssl` tool, included in OpenSSL and LibreSSL: ```sh openssl dhparam -out /etc/ssl/dhparam.pem 4096 ``` This will take a while, because it's looking for sources of randomness in your (very non-random) computer. Just keep doing stuff with it running in the background. The resulting file is public and expendable. There exist many options for most of the software roles described earlier. This guide covers multiple options for most of these, given here from most to least recommended by me: * IMAP server + MDA: [Dovecot](https://www.dovecot.org/). * MTA: [OpenSMTPD](https://opensmtpd.org/), [Postfix](http://www.postfix.org/), or [Exim](https://exim.org/). * Spam filtering: [Rspamd](https://www.rspamd.com/), or [SpamAssassin](https://spamassassin.apache.org/). * DKIM signing: [OpenDKIM](http://www.opendkim.org/), or [DKIMproxy](http://dkimproxy.sourceforge.net/). The MTA feeds arriving emails through the spam filter before giving them to the MDA, which puts them in an inbox folder. Likewise, it feeds outgoing emails through the DKIM signer. The IMAP server then delivers your inbox contents to your MUA. All these MTAs include an MDA, but nevertheless I recommend using Dovecot's one because it gives the most control. This is also important because of how spam filtering works: according to the email specification, an MTA assumes full responsibility over any emails it accepts for delivery or relaying, and promises that it will do its best to get it to its destination. The takeaway from this is that an MTA must not magically swallow any emails it doesn't like, so if it uses a spam filter, that filter may tag emails as "spam" or "not spam", but the MTA is not allowed to act based on that verdict. The MDA does not have this restriction, so only there can we make the decision what to do with spam, and Dovecot's MDA is by far the best equipped for that. When setting up an email server, you have the choice between attaching email addresses to system users or virtual users. With system users, if an email arrives for `foo@example.com`, then MTA, MDA and IMAP server will all expect that there exists a `foo` Unix user on the server to deliver it to. With virtual users, this is not the case. This guide only covers virtual users, which I recommend because it's more flexible, and unless you're using some ancient mainframe there is no advantage to using system users. Therefore, create a global email user and group, which are traditionally both called `vmail`. Do *not* pass the "system user/group" flags, because that will get you into trouble with Dovecot: ```sh # GNU CoreUtils: groupadd vmail useradd -M -g vmail vmail # BusyBox: addgroup vmail adduser -HD -G vmail vmail # *BSD: no clue, you can work it out ``` ## DNS records First things first, you should create an RSA keypair for DKIM, which you do using the `openssl` utility again. The minimum size is 1024 bits, but I recommend using 2048 bits. Bigger is better, but because DNS is involved we can't stretch it to 4096 bits without risking causing discomfort to [some](https://serverfault.com/questions/747185/dkim-can-i-use-a-rsa-key-larger-than-2048bit-i-e-4096) servers. ```sh $ openssl genrsa -out /TODO/private.key 2048 $ openssl rsa -in /path/private.key -out /path/public.key ``` Now we must set up all the necessary DNS records. It may take up to a day for these to propagate over the Internet, so I recommend doing this section now and the rest tomorrow. Now, this might sound obvious, but you must have an A and/or AAAA record to associate your domain `example.com` with the server's IP address. For email you also must have [reverse DNS](https://en.wikipedia.org/wiki/Reverse_DNS_lookup) set up correctly. If you're renting your server remotely, you can often do this from the provider's configuration tool, or you can just create a PTR-type DNS record. To inform the rest of the Internet that your server is an email server, create an MX (Mail eXchanger) DNS record for your domain. Note the dot at the end of the domain name: ``` MX 42 example.com. ``` When a message is sent to an email address ending in `@example.com`, the sending server will query DNS for any MX records for `example.com`. There it will find a domain name (in this case `example.com` again), for which it will look up the IP address, and send the email to. The domain name in the record must *not* have an associated CNAME record; it must be directly translatable to an IP address. You may have multiple MX records, which must contain different domain names, each with an associated preference number (`42` here). The sending server will use all MX records with lower numbers first, and if those servers are all unavailable, it will try a higher number. If you have multiple mail servers (which is good for availability), you can therefore declare those as follows: ``` MX 13 mx1.example.com. MX 42 mx2.example.com. ``` Here, a server sending an email to your domain `example.com` will try to send it to the IP address of `mx1.example.com` first, and if that fails, it will move on to the higher numbered `mx2.example.com`. If both `mx1` and `mx2` have the same number, then the sender will randomly choose one, which is useful for load balancing. Next, publish an SPF policy record so you don't look like a spammer. This is a TXT (*not* SPF!) record with the following contents: ``` TXT "v=spf1 mx -all" ``` Everything after the version `v=spf1` is a list of *mechanisms* for a spam filter to try out in the given order. The `-all` at the end says to reject your email if all of the previous mechanisms fail verification. I recommend only using the `mx` mechanism, which tells the verifier to look at the A/AAAA addresses of the domains in your MX records. This allows you to add, remove, or change your servers without needing to update this record. Your DKIM policy must also be published in a TXT record as follows, where `` is the public RSA key `MI...AB` stored in `/TODO/public.key`, with the newlines removed: ``` TXT "v=DKIM1; t=s; h=sha256; p=" ``` Here, `v=DKIM1` is the version and must be the first tag. The flag `t=s` enables strict mode, as recommended by the [DKIM spec](https://tools.ietf.org/html/rfc6376#page-29), meaning emails sent from subdomains immediately fail verification. The optional tag `h=sha256` blocks the use of the old SHA1 algorithm. Finally, there is the DMARC policy, which, unlike the others, will need to be updated at the end of this guide. For now, create another TXT record with these contents, where `` is an email address of your choosing, which may belong to the domain you're setting up for: ``` TXT "v=DMARC1; aspf=s; adkim=s; p=none; sp=reject; fo=1; ruf=mailto:" ``` The version tag `v=DMARC1` is required and must come first. Next, `aspf=s` and `adkim=s` enable strict mode for both SPF and DKIM, which once again blocks subdomains from passing the test. Then `p=none` and `sp=reject` control what to do to failed messages coming from the main domain and subdomains, respectively. Unsurprisingly, `reject` means that delivery should be refused, `none` asks to let it through anyway, and `quarantine` tells the filter to take a closer look at the email or to put it in a spam folder. Finally, `fo=1` asks the filter to create a forensic report if any type of verification fails, and `ruf=` gives an address to send it to. ## IMAP server: Dovecot [Dovecot](https://www.dovecot.org/) is a very popular IMAP server, focused on being lightweight, simple, and secure. Its documentation, while extensive and accurate, is an absolute mess: the information you need is usually either spread across multiple pages, or buried somewhere two internal links deep. If you installed Dovecot via your package manager, you'll probably have a lot of configuration files in the `/etc/dovecot` directory and its subdirectories. I want you to delete all of them. Yes, `rm -rf` that crap. Dovecot is simple to configure, and doesn't care where you put its settings, so having all that chaos in `/etc/dovecot` just makes things unnecessarily confusing. Create a new blank configuration file `/etc/dovecot/dovecot.conf`, and start by filling in the details about your TLS certificate, and making clear that unencrypted connections are unacceptable: ```sh ssl = required ssl_cert = /`. The `user` name may contain the ending `@example.com`, but doesn't need to. Create the password hash to put in the `password` field as follows: ```sh # If your server is beefy and has lots of RAM: doveadm pw -s ARGON2ID-CRYPT # If you're using an old potato: doveadm pw -s SHA512-CRYPT ``` After you've entered your password, simply copy-paste the entire hash string outputted by the program into the `password` field. Dovecot needs a file describing user accounts on two occasions: * To check whether a user logging into the IMAP server is valid and has given the right password. This is handled by the `passdb` block(s) in the configuration. * To know which email addresses the server is responsible for. This is given by the `userdb` block(s) in the configuration. These functions aren't necessarily fulfilled by the same file: you can map multiple email addresses to one user acccount, or multiple user accounts to one email address. For simplicity, though, we'll use the same file: ```sh passdb { driver = passwd-file args = scheme=ARGON2ID-CRYPT username_format=%n /etc/dovecot/users #args = scheme=SHA512-CRYPT username_format=%n /etc/dovecot/users } userdb { driver = passwd-file args = username_format=%n /etc/dovecot/users # Use vmail for all users, and if all users' inboxes have the same form: override_fields = uid=vmail gid=vmail home=/var/vmail/%n } ``` The `driver` option sets the kind of table Dovecot should expect. We tell it to use a file in the `passwd`-like format described above, but other common choices include SQL or the actual `/etc/passwd` file. The options available in `args` depend on the chosen `driver`. The password hashing algorithm is given by `scheme`, while `username_format` desribes the format of user names in the table: if Dovecot receives an email for `someone@example.com`, then for `%n` it will expect `someone` in `/etc/dovecot/users`, but if you use `%u` it will want the full email address instead. In the `userdb` block we force the use of `vmail:vmail` for all users, and tell Dovecot to put their mail in `/var/vmail/%n`, where `%n` means the same as before. You can also use `%u` for the full address again, or `%d` for the domain name. Optionally, you can create a catch-all inbox that will accept all emails sent to your domain that don't match anyone in `/etc/dovecot/users`. Just add this second `userdb` block after the first: ```sh userdb { driver = static args = uid=vmail gid=vmail home=/var/vmail/catchall allow_all_users=yes } ``` The `static` driver means there is no table file: all configuration is directly within this `userdb` block. If we didn't specify `allow_all_users=yes`, then Dovecot would check whether users existed using the `passdb` table. Finally, now that we've told Dovecot where the virtual mail users' `home` folders are, we need to specify how to store the emails within them: ```sh mail_location = maildir:~:LAYOUT=fs ``` The two standard mailbox formats to choose from are `maildir` and `mbox`. I highly recommend `maildir`; it's faster and more flexible than `mbox` because it uses folders, whereas the older `mbox` uses a single database file. The `~` tells Dovecot to directly put all mails in the account's home folder, and `LAYOUT=fs` to user filesystem directories for you email folders. TODO? ## MTA: Postfix ## MTA: Exim ## MTA: OpenSMTPD OpenSMTPD is an MTA by the [OpenBSD](https://www.openbsd.org/) project, who are known for their focus on security and minimalism. I really like OpenSMTPD's flexible configuration format, and its simple and intuitive way of passing around emails. It doesn't implement milter or any other filtering interface, so DKIM signing and spam checking must be shoehorned in. As such, only DKIMproxy can be used for DKIM signing. To begin, write your email domain on a single line in `/etc/smtpd/mailname`: ``` example.com ``` Next, you must fill in the `/etc/smtpd/aliases` file which maps recipient addresses to system users: ``` mailuser@example.com systemuser ``` You can create a wildcard (catch-all) address for your domain by omitting `mailuser`, such that any unknown recipients end up there. You may also specify multiple system users as `systemuser1,systemuser2`. The first part of the actual config `/etc/smtpd/smtpd.conf` is as follows, with DKIMproxy listening on port 19999 and outputting to 20000: ```sh # Register TLS certificate to encrypt email transport: pki "example.com" certificate "/etc/ssl/certs/example.com.pem" pki "example.com" key "/etc/ssl/private/example.com.key" pki "example.com" dhe auto table users "/etc/smtpd/aliases" # Relay emails submitted by client to DKIM signer on port 19999: listen on eth0 port 465 smtps pki "example.com" tag SEND listen on eth0 port 587 tls-require pki "example.com" tag SEND accept tagged SEND from any for any relay via smtp://127.0.0.1:19999 # Relay outgoing DKIM-signed emails to their intended destination: listen on lo port 20000 tag DKIM accept tagged DKIM from local for any relay ``` The lines starting with `pki` enable [DHE](https://en.wikipedia.org/wiki/Diffie%E2%80%93Hellman_key_exchange) and tell where the public certificate and private key are for the domain. We listen for client emails on both ports 465 (SMTPS) and 587 (SMTP), although for the latter we refuse the submission if STARTTLS fails. If no spam filtering is desired, let OpenSMTPD deliver the email by appending the following to the configuration file: ```sh # Receive incoming emails and write them to the inbox: listen on eth0 port 25 tls pki "example.com" accept from any for domain "example.com" virtual deliver to lmtp rcpt-to as vmail ``` Here, we look up which system user corresponds to the recipient in ``, and deliver it to their maildir `mail` in their home folder. ### With SpamAssassin UNTESTED For SpamAssassin, we'll need to use [spampd](https://github.com/mpaperno/spampd) as an SMTP shim, which we set to listen on port 29999 and relay to port 30000. Add the `--nodetach` option in an init script or systemd service, but leave it out when starting `spampd` by hand for testing: ```sh $ spampd --host=127.0.0.1:29999 --relayhost=127.0.0.1:30000 [--nodetach] ``` The rest of `smtpd`'s configuration is then as follows: ```sh # Receive incoming emails and send them to spampd on port 29999: listen on eth0 port 25 tls pki "example.com" tag SCAN accept tagged SCAN from any for domain "example.com" relay via smtp://127.0.0.1:29999 # Receive filtered incoming emails and write them to the inbox: listen on lo port 30000 tag DONE accept tagged DONE from local for any virtual deliver to maildir "~/mail" ``` TODO ### With Rspamd ## DKIM signer: DKIMproxy [DKIMproxy](http://dkimproxy.sourceforge.net/) is a mature and simple SMTP proxy that signs or verifies all emails passed through it using the Perl Mail::DKIM module. In this guide we only use its signing feature, which is controlled by `/etc/dkimproxy/dkimproxy_out.conf`. This configuration file is pretty self-explanatory: ```sh listen 127.0.0.1:19999 relay 127.0.0.1:20000 domain example.com selector mail keyfile /etc/dkim/private.key signature dkim(c=relaxed/simple,a=rsa-sha256) ``` This last line clearly tells it to create a DKIM signature, because it also supports DKIM's predecessor, Yahoo! DomainKey. The tag `a=rsa-sha256` specifies the signing algorithm, while `c=relaxed/simple` gives the [canonicalization](https://tools.ietf.org/html/rfc6376#section-3.4) method. Canonicalization is a way to prevent the signature from becoming invalid as the email might get reformatted by the MTAs on its journey. Our choice `relaxed/simple` allows the header to be reformatted (`relaxed`), but not the actual body of the message (`simple`). ## DKIM signer: OpenDKIM [OpenDKIM](http://www.opendkim.org/) is a milter program that can both create and verify DKIM signatures. ## Spam filter: Rspamd ## Spam filter: SpamAssassin