diff options
48 files changed, 3770 insertions, 0 deletions
diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..6a2fe92 --- /dev/null +++ b/.gitignore @@ -0,0 +1 @@ +/public/* diff --git a/Makefile b/Makefile new file mode 100644 index 0000000..104b8ff --- /dev/null +++ b/Makefile @@ -0,0 +1,10 @@ +FINDARGS= \( -name "*.html" -o -name "*.xml" -o -name "*.css" -o -name "*.js" \) + +all: + rm -rf public/* + hugo + for f in `find public/ -type f $(FINDARGS)`; do \ + echo "Compressing $$f..." ; \ + gzip --keep --force -9 $$f ; \ + brotli --keep --force -Z $$f ; \ + done diff --git a/archetypes/default.md b/archetypes/default.md new file mode 100644 index 0000000..555c1bd --- /dev/null +++ b/archetypes/default.md @@ -0,0 +1,7 @@ +--- +title: "{{ replace .Name "-" " " | humanize }}" +firstLetter: "{{ slicestr .Name 0 1 | upper }}" +date: {{ .Date }} +draft: false +--- + diff --git a/archetypes/default.pdc b/archetypes/default.pdc new file mode 100644 index 0000000..031a41e --- /dev/null +++ b/archetypes/default.pdc @@ -0,0 +1,9 @@ +--- +title: "{{ replace .Name "-" " " | humanize }}" +firstLetter: "{{ slicestr .Name 0 1 | upper }}" +publishDate: nil +date: {{ .Date }} +draft: false +markup: pandoc +--- + diff --git a/config.toml b/config.toml new file mode 100644 index 0000000..d5a8d6d --- /dev/null +++ b/config.toml @@ -0,0 +1,5 @@ +baseURL = "" +languageCode = "en-us" +title = "Prefetch" + +disableKinds = ["taxonomy", "term"] diff --git a/content/_index.md b/content/_index.md new file mode 100644 index 0000000..e9e5dcc --- /dev/null +++ b/content/_index.md @@ -0,0 +1,17 @@ +--- +title: "Home" +date: 2021-02-22T17:15:50+01:00 +draft: false +--- + +Welcome to my website. + +Once in a blue moon, I'll post something here +related to my areas of interest: +programming, optimization, +mathematics, physics and even linguistics. + +This site is served to you by [nginx](https://nginx.org/), +and statically generated by [Hugo](https://gohugo.io). +I intend to keep it forever free of advertising and trackers, +and to maintain my A+ score for [TLS quality](https://www.ssllabs.com/ssltest/analyze.html?d=prefetch.eu). diff --git a/content/blog/2020/email-server-extras.md b/content/blog/2020/email-server-extras.md new file mode 100644 index 0000000..72299c9 --- /dev/null +++ b/content/blog/2020/email-server-extras.md @@ -0,0 +1,414 @@ +--- +title: "Setting up an email server in 2020 with OpenSMTPD and Dovecot: extras" +publishDate: 2020-04-27 +date: 2021-02-22T17:19:49+01:00 +draft: false +--- + +# + +This sequel to my earlier [guide](/blog/2020/email-server/) discusses +extra tips and tricks to extend your email setup. +This page will be updated continuously as I come up with ideas. + +Last updated 2020-04-29. + + +## General + +### Multiple domains + +You can generalize your setup to handle multiple domains +with very little effort. In the following, +I'll assume that your two domains are called `foo.com` and `bar.com`. + + +#### DNS records + +There should be MX, SPF, DKIM and DMARC records for both domains, +as explained in the previous guide. Fortunately, these records +can have identical contents for both domains! + +However, it remains essential that the mail server's mailname +and reverse DNS domain name match up exactly, +so you should create MX records with that in mind. +Therefore, if the email server for both domains has `mx1.foo.com` +as reverse DNS name, the MX records should look like this: +```sh +foo.com. MX 42 mx1.foo.com. +bar.com. MX 42 mx1.foo.com. +``` +This is perfectly valid: the only thing that matters is that +what your SMTP server calls itself agrees with what reverse DNS +says that the server is actually called. + + +#### Dovecot + +To make Dovecot aware of multiple domains, +you only need to update the `/etc/dovecot/users` file +to add accounts for both domains. +However, in the original guide, I said to only write `user` +in the file, without the `@foo.com`, for an address `user@foo.com`. +Unsurprisingly, that isn't an option for multiple domains, +so you must put the full address in `/etc/dovecot/users`. + +Then update `/etc/dovecot/dovecot.conf` to reflect that, +by changing `%n` to `%u` in `username_format`: +```sh +userdb { + driver = passwd-file + args = username_format=%u /etc/dovecot/users + override_fields = uid=vmail gid=vmail home=/home/vmail/%d/%n +} +``` +Also note the change in the `home` setting: +the inbox of a user `user@foo.com` will now be stored +in `/home/vmail/foo.com/user`. +That's all you need to change. + + +#### OpenSMTPD + +To inform OpenSMTPD of all the domains, +create a new file `/etc/smtpd/domains`, +and in there put all desired names on their own line: +```sh +foo.com +bar.com +``` +And as I mentioned when discussing the DNS records, +you should check that `/etc/smtpd/mailname` agrees +with your server's reverse DNS. + +Then, in the main configuration file, tell OpenSMTPD to +use the new domains file when deciding whether to accept an message, +by declaring a new table and changing the `match` line for inbound mail: +```sh +table domains "/etc/smtpd/domains" +# ... +match from any for domain <domains> action "RECV" +``` + +#### Rspamd + +The last thing to do is to inform Rspamd of the multiple domains. +It's really easy: simply add multiple domain blocks: +```c +domain { + foo.com { + path = "/path/to/dkim/private.key"; + selector = "hello"; + } +} +domain { + bar.com { + path = "/path/to/dkim/private.key"; + selector = "world"; + } +} +``` + +### Advanced security + +SPF, DKIM and DMARC are email's traditional DNS-based security systems, +but in 2018 the IETF released [RFC 8460](https://tools.ietf.org/html/rfc8460) and [RFC 8461](https://www.rfc-editor.org/rfc/rfc8461.txt), +which respectively define TLSRPT and MTA-STS, +two fancy new systems focused on TLS-encrypted email transport. + +These security mechanisms are pretty new, +so you won't get a huge benefit from enabling them, +but big email providers' draconian spam filters might like it. + + +#### TLSRPT + +TLS reporting, or TLSRPT for short, is very simple: +all it does is provide a contact email address in case +somebody has trouble with the TLS configuration of your SMTP server. + +To enable it for your custom email domain `example.com`, +simply create a DNS TXT record for the `_smtp._tls` subdomain: +```sh +_smtp._tls.example.com. TXT "v=TLSRPTv1; rua=mailto:<contact>" +``` +Where `<contact>` is an email address of your choosing. +That's all! + + +#### MTA-STS + +MTA Strict Transport Security (MTA-STS) tells other servers +that you take TLS encryption of messages very seriously, +so they should avoid sending you unencrypted email, +and should only accept certain certificates from your side. + +Compared to the previously discussed DNS-based security extensions, +MTA-STS is a bit more work to set up, +because you'll also need an HTTP web server. + +The DNS part is still pretty simple: +create yet another DNS TXT record, +this time for the subdomain `_mta-sts`: +```sh +_mta-sts.example.com. TXT "v=STSv1; id=<id>" +``` +The `<id>` should identify the version of your policy, +so other servers can quickly see if something changed. +I recommend using today's date. + +For the next part, I'll assume that you already have +a web server running on a server with the IP address `1.2.3.4`. +I use [nginx](https://nginx.org/) for this, running +on the same server as OpenSMTPD and Dovecot, +but you don't have to do the same. + +Create an A record which binds your server +to the subdomain `mta-sts` (without underscore): +```sh +mta-sts.example.com. A 1.2.3.4 +``` +Set your web server to serve the file +`https://mta-sts.example.com/.well-known/mta-sts.txt` +(we'll discuss that file in a moment). +Note that this policy file **must** be served over HTTPS, +so you need a valid TLS certificate for that domain. + +The contents of the `mta-sts.txt` policy file are as follows, +where `mx1.example.com` and `mx2.example.com` are the hosts +mentioned in `example.com`'s DNS MX records: +```sh +version: STSv1 +mode: enforce +mx: mx1.example.com +mx: mx2.example.com +max_age: <age> +``` +All MX servers must be mentioned this way. +If you're feeling cautious, you may want to set +`mode` to `testing` in the beginning. +This policy is valid for `<age>` seconds, +which is recommended to be several weeks, +but to start with, I suggest using 86400 seconds (one day). +Finally, ensure that this file has CRLF Windows-style line endings. + +To correctly pass an MTA-STS test, the TLS certificate +presented by e.g. `mx1.example.com` should be valid for `mx1.exaple.com`. +To achieve this without needing to manage too many certificates, +you can specify multiple domains when requesting a certificate, +or you can use a wildcard domain (`*.example.com`). +Note, however, that MTA-STS testing tools don't like +the latter option, so I recommend the former. + +Once you're done, check your work by using either +[ESMTP](https://esmtp.email/tools/mta-sts/)'s or [Ayke](https://aykevl.nl/apps/mta-sts/)'s +online MTA-STS validation tools, +ignoring any warnings about DNSSEC or DANE. +If all is good, great! + +Even if you did everything correctly, +these tools will warn you that you're not using DNSSEC/DANE. +It might then be tempting to set that up for even more security, +but I recommend against that for private servers: take a look at [this](https://dane.sys4.de/common_mistakes). + + + +## OpenSMTPD + +### Client certificates (in addition to passwords) + +You can configure OpenSMTPD to request a client certificate +for sending emails, as a second factor for authentication. + + +#### Certificates + +We need to start with some cryptography to create and verify certificates. +I recommend that you do all of this on your trusted *client* device, +and only copy the necessary files to the server later. + +DISCLAIMER: +All the keys and certificates that we'll generate in this section are +for **private use** only, to handle a small number of trusted clients. +I'm not a cryptography expert, so you should **not** listen to me +for large-scale systems that may involve untrusted devices. + +The first step is to set up a private Certificate Authority (CA), +which issues the client certificates and can be used to verify them. +Start by generating an RSA private key, +which you should store in a safe place and not share with anyone: +```sh +$ openssl genrsa -out mailca.key 2048 +``` +Extract a public certificate from this key as follows. +Because we're lazy, we give it a lifetime of 36500 days: +```sh +$ openssl req -new -x509 -days 36500 -key mailca.key -out mailca.crt +``` +When running this command, OpenSSL will ask you some questions +about who this certificate is intended for. +Since this is for personal use, your answers don't matter, +so just use the defaults. +Some fields (I think only *Country Name* and *Organization Name*) +cannot be empty, but the others can. + +Moving on to the client, once again generate an RSA private key: +```sh +$ openssl genrsa -out mailclient.key 2048 +``` +From this private key, create a Certificate Signing Request (CSR) +as follows, where you'll be asked the same questions as before: +```sh +$ openssl req -new -key mailclient.key -out mailclient.csr +``` +By feeding this CSR to the CA, we can create a signed client certificate +that can be verified using the CA's public certificate. +```sh +$ openssl x509 -req -in mailclient.csr -out mailclient.crt \ + -days 36499 -CA mailca.crt -CAkey mailca.key +``` +If you want to multiple client certificates, +just repeat the last few steps for each one. + + +#### Server + +OpenSMTPD needs to verify the validity of client certificates +using the CA's public certificate, so you should copy that +to somewhere on the server, e.g. `/etc/smtpd/mailca.crt`, +and declare it to OpenSMTPD by adding this near +the top of `/etc/smtpd/smtpd.conf`: +```sh +ca "mailca" cert "/etc/smtpd/mailca.crt" +``` +Then replace the entire configuration for outbound mail as follows. +Note that this removes SMTPS support, leaving only STARTTLS: +```sh +# Outbound +listen on eth0 port 587 tls-require verify pki "example.com" ca "mailca" auth <passwds> filter "rspamd" +action "SEND" relay srs +match from any auth for any action "SEND" +``` +The magic word here is "`verify`", which tells OpenSMTPD +to ask for a client certificate and to verify it using the given CA. + + +#### Client + +Now you won't be able to send emails if your client doesn't +present its certificate to the server! +Unfortunately, not all mail clients support this; personally +I use [Thunderbird](https://www.thunderbird.net/) with success. +I won't include any client-specific configuration here, +but I will say this: + +For some clients (like Thunderbird), you'll have an easier time +importing your client certificate if you encode it in the +[PKCS #12](https://en.wikipedia.org/wiki/PKCS_12) storage format: +```sh +$ openssl pkcs12 -export -in mailclient.crt -inkey mailclient.key \ + -certfile mailca.crt -out mailclient.pfx +``` +OpenSSL will ask you to set a password, which you'll need to +enter again when importing the certificate into the client. + + + +### Client certificates (instead of passwords) + +If you really want to, you can use the client certificates +as a substitute for passwords. This is especially useful +if you set up a catchall inbox in Dovecot, +because this will allow you to send emails +from arbitrary addresses from your domain. + +To do this, follow the same procedure as in the previous section, +but with a slightly different OpenSMTPD configuration: +```sh +listen on eth0 port 587 tls-require verify pki "example.com" ca "mailca" filter "rspamd" tag "VALID" +action "SEND" relay srs +match from any tag "VALID" for any action "SEND" +``` +All incoming connections that present a good certificate +will be tagged as being `VALID`, and their mail will be relayed. + +Unfortunately, we're not quite done yet here, +because Rspamd is now very confused... + + +#### Rspamd + +When OpenSMTPD passes a message through Rspamd, it also includes +some metadata, most notably whether the sender has authenticated +successfully with OpenSMTPD... which is now no longer the case +for submissions, because we've removed the `auth` directive! + +Rspamd therefore starts regarding these outgoing emails +as *incoming* emails, because they don't seem +to come from a trusted user. So instead of signing them with DKIM +and handing them back to OpenSMTPD, it will do a full spam scan. +If they get a high spam score (which is likely for short test emails), +*your* spam filter, running on *your* server, +will be flagging *your* messages as spam! + +The solution is to whitelist your domain(s) in Rspamd, +so it won't scan them. To do this, create a new file +`/etc/rspamd/local.d/settings.conf` with these contents, +where `foo.com` and `bar.com` are the domains to whitelist: +```c +outbound { + priority = high; + from = "@foo.com"; + from = "@bar.com"; + apply { + actions { + add_header = 1000; + } + } +} +``` +Setting `priority` to `high` ensures that Rspamd checks +this rule before doing anything else. +You can add any number of `from` directives; +this rule will be applied if any of them match. +It only sets the threshold for the action `add_header` to `1000`. +That is, if the email doesn't get a spam score of at least 1000 +(the default is 6) Rspamd will not add any spam tags. + +Because Rspamd is still regarding your emails as inbound, +you also need to change the global settings of +the DKIM signer in `/etc/rspamd/local.d/dkim_signing.conf`, +such that they include the following: +```c +sign_inbound = true; +allow_hdrfrom_mismatch = true; +allow_username_mismatch = true; +``` +This tells Rspamd to add DKIM signatures to incoming emails, +which in this case includes yours. +Allowing these mismatches ensures that the messages still get signed, +even if you're sending from an arbitrary address. + + + +## Dovecot + +### Catchall inbox + +In Dovecot, you can create a catch-all inbox that will accept all +emails sent to your domain that don't match anyone in `/etc/dovecot/users`. +Just add another `userdb` block *after* the first: +```sh +userdb { + driver = static + args = uid=vmail gid=vmail home=/var/vmail/catchall allow_all_users=yes +} +``` +The `static` driver means there is no table file: +all configuration is directly within this `userdb` block. +If we don't specify `allow_all_users=yes`, then Dovecot +will check whether users exist using the `passdb` table, +and will conclude that the recipient is invalid. + + + diff --git a/content/blog/2020/email-server.md b/content/blog/2020/email-server.md new file mode 100644 index 0000000..3b7f439 --- /dev/null +++ b/content/blog/2020/email-server.md @@ -0,0 +1,718 @@ +--- +title: "Setting up an email server in 2020 with OpenSMTPD and Dovecot" +publishDate: 2020-04-27 +date: 2021-02-22T17:27:49+01:00 +draft: false +--- + +# + +So, you want to set up your own email server? In that case, welcome. + +There are many reasons to run a custom email server, +ranging from privacy concerns about providers like Google, +to just wanting to do it for fun and/or learning. +Since you're here, I assume you've already found a reason. + +Beware: this is a messy topic, and the available documentation +is even messier, so it could take a while before you get it to work properly. +I've compiled this guide according to my experiences +in an attempt to make this dark art more accessible, +but your mileage may vary considerably. I hope you find it useful. + +This guide is aimed at people who are comfortable with +the Linux/*BSD command line. + +When you're done, take a look at the +[sequel](/blog/2020/email-server-extras/) +for ideas to extend your setup. + +Last content update on 2020-04-29. Last correction on 2021-02-20. + + + +## Preparation + +Setting up email is relatively complex compared to e.g. a static website, +because you need to configure not one, but *two* server programs, +*and* you need to shoehorn modern security features into email's Stone-Age design. +I'll start by explaining the general structure of a mail server setup. + + +### How email works + +The programs involved in the exchange of emails are called [agents](https://en.wikipedia.org/wiki/Email_agent_(infrastructure)). +Officially, there are 5 different types of agent: MUA, MSA, MTA, MDA and MRA. +But fortunately, it's reasonable to treat the MRA and MSA +as being part of the MUA and MTA, respectively. + +The *Mail User Agent* (MUA) is simply the client on your device at home +that you use to send and receive emails, and this guide assumes +you already have a favourite program for this, e.g. [Thunderbird](https://www.thunderbird.net/en-US/). +Nowadays it's fashionable to use a web interface for emails, +but that's also beyond the scope of this guide. + +The *Mail Delivery Agent* (MDA) is a program that watches over +the server's copy of your mailbox: it manages your inbox, +remembers which messages you have or haven't read, +keeps a copy of your drafts, etc. +When you open your mailbox, your MUA will connect to +your server's MDA using the [IMAP](https://en.wikipedia.org/wiki/Internet_Message_Access_Protocol) protocol +(or [POP3](https://en.wikipedia.org/wiki/Post_Office_Protocol), but that one's [obsolete](https://pop2imap.com/)). + +The *Mail Transfer Agent* (MTA) is responsible for +making messages arrive at the right destination. +When you send an email, your MUA will pass it on to your server's MTA, +which will in turn pass it on to the recipient's mail server. +Likewise, when someone sends *you* an email from another server, +the MTA will receive it and hand it over to the MDA so you can read it later. +In both cases the MTA speaks the [SMTP](https://en.wikipedia.org/wiki/Simple_Mail_Transfer_Protocol) protocol. + +In this guide our MDA will be [Dovecot](https://dovecot.org/), +which is a very popular choice for that role. +As for the MTA, there exist several options, +the most popular being [Postfix](http://www.postfix.org/) and [Exim](https://exim.org/). +However, this guide uses the newer, lesser-known [OpenSMTPD](https://opensmtpd.org/), +which in my experience is *much* easier to set up: +Postfix and Exim have complex configurations and +are geared towards large-scale email providers, +whereas OpenSMTPD is more beginner-friendly. + + + +### Security + +The base email system is horribly insecure on its own, +so we still need to duct-tape on some security features. +In this context, "security" has two meanings: +spam protecion and privacy protection (encryption). + +Spam protection also means two things here: +defending yourself against spammers, and +preventing that *your* emails get flagged as spam. +The former is optional, but the latter is not: +big providers such as Google and Microsoft +use infamously strict spam filters, +and if they decide that your server is a spammer, +there's almost nothing you can do about it. +Spam protection techniques will be discussed +in more detail over the course of this guide. + +Privacy protection is important in the 21st century: +you don't want a random router in the Internet to read all your emails, +which may contain sensitive information such as +private conversations and account password reset links. +You should therefore try to make sure that emails are +transported over an encrypted channel. +To do this, you have two options for encryption: +*mandatory* and *opportunistic* encryption. + +Mandatory encryption is only practical for client-server +communication (not server-server), and is provided by IMAPS and SMTPS, +which wrap the IMAP and SMTP protocols in [TLS](https://en.wikipedia.org/wiki/Transport_Layer_Security), +in the same way that [HTTPS](https://en.wikipedia.org/wiki/HTTPS) does for [HTTP](https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol). + +For server-server communication, the only option is +opportunistic encryption in the form of [STARTTLS](https://en.wikipedia.org/wiki/STARTTLS), +where communication is only encrypted if both parties agree +after a short unencrypted discussion. +That last part is vulnerable to [MitM](https://en.wikipedia.org/wiki/Man-in-the-middle_attack) attacks, +where anyone along the path of the email servers' discussion +can alter the exchange to block the use of encryption, +which sometimes actually [happens](https://www.eff.org/deeplinks/2014/11/starttls-downgrade-attacks) in practice. + +The only way to make sure that STARTTLS is used in that case +is to refuse any exchange unless the servers agree to use encryption. +Unfortunately, that's a risky approach that I can't recommend, +because not all servers support encryption (unbelievable, right?). +For example, I've received airline booking confirmations, +full of personal details, and made with billion-dollar companies, +sent across the Internet without any protection. + +This guide includes intructions to enable encryption, +but assumes that you already have a TLS certificate for that. +If not, find a guide to get one from [Let's Encrypt](https://letsencrypt.org/) (it's free!), +and remember that you'll need to renew it every few months. +Using a self-signed certificate *may* work, but I don't recommend it. + +In the rest of this guide I'll assume that +you have a public full-chain TLS certificate at `/etc/ssl/certs/example.com.pem`, +and a private encryption key at `/etc/ssl/private/example.com.pem`. + + + +### Server + +Obviously, you'll need a server to run the MTA and MDA on. +You can host your own at home, but the more reliable option is +to rent one in a data center ([VPS](https://en.wikipedia.org/wiki/Virtual_private_server)). +This guide was written with a Linux server in mind, +but in theory it should also work on the BSDs +([OpenBSD](https://www.openbsd.org/), [FreeBSD](https://www.freebsd.org/), +[NetBSD](https://www.netbsd.org/), etc.) with minimal adaptation. + +The server must be online 24/7, you must have root SSH access, +it must have a static IP address, and TCP network port 25 must be open. +Especially check that last one: you may need to explicitly ask +your home ISP or the server provider to enable port 25, +because they often close it to prevent spam. +You can usually do this from their web interface. + +You also must have a domain name, which I'll call `example.com`. +This will be necessary for basically everything: +DNS records, TLS certificate, MTA network configuration, etc. +If you don't have one yet, you can choose between many registrars |