v0.50 (September 25, 2020)

--------------------------
 
 Setup:
 
 * When upgrading from versions before v0.40, setup will now warn that ownCloud/Nextcloud data cannot be migrated rather than failing the installation.
 
 Mail:
 
 * An MTA-STS policy for incoming mail is now published (in DNS and over HTTPS) when the primary hostname and email address domain both have a signed TLS certificate installed, allowing senders to know that an encrypted connection should be enforced.
 * The per-IP connection limit to the IMAP server has been doubled to allow more devices to connect at once, especially with multiple users behind a NAT.
 
 DNS:
 
 * autoconfig and autodiscover subdomains and CalDAV/CardDAV SRV records are no longer generated for domains that don't have user accounts since they are unnecessary.
 * IPv6 addresses can now be specified for secondary DNS nameservers in the control panel.
 
 TLS:
 
 * TLS certificates are now provisioned in groups by parent domain to limit easy domain enumeration and make provisioning more resilient to errors for particular domains.
 
 Control Panel:
 
 * The control panel API is now fully documented at https://mailinabox.email/api-docs.html.
 * User passwords can now have spaces.
 * Status checks for automatic subdomains have been moved into the section for the parent domain.
 * Typo fixed.
 
 Web:
 
 * The default web page served on fresh installations now adds the `noindex` meta tag.
 * The HSTS header is revised to also be sent on non-success responses.
 -----BEGIN PGP SIGNATURE-----
 
 iQFDBAABCgAtFiEEX0wOcxPM10RpOyrquSBB9MEL3YEFAl9t2AgPHGp0QG9jY2Ft
 cy5pbmZvAAoJELkgQfTBC92BZNkH/1jIGoWTz0xlS+e+TeXpHoCp/7zYAvQq/a/y
 vj9t1N1+bBg6Ywbd8UxyvOHwuL/UQU/5LTq6hk3gD+2ARfJUvDRbb047Xzlisg3N
 LhNoVhVbsxqKP1X2ZjeIBq9DgzMavuB64Bwd5UNdceM0Addi8KuCDOMF+FNY2t8k
 xytGjYdBi1/BG6SLBX+FAm5yrJghmkUJs2FnJjebSyyeV2HP3L1iBrk2N8UBd6PU
 fVjde534lgygFZK/8yXJpY2olfLMYJv7CaOMxvaW6RpbMI8VeLwDLfRt5LcrQZqq
 YXkuEnUI0eygbQYkeK/Vr1Vey6uQAWzIfbImEglHfvOXsZSYFXs=
 =SJNM
 -----END PGP SIGNATURE-----

Merge tag 'v0.50' of https://github.com/mail-in-a-box/mailinabox into master

v0.50 (September 25, 2020)
--------------------------

Setup:

* When upgrading from versions before v0.40, setup will now warn that ownCloud/Nextcloud data cannot be migrated rather than failing the installation.

Mail:

* An MTA-STS policy for incoming mail is now published (in DNS and over HTTPS) when the primary hostname and email address domain both have a signed TLS certificate installed, allowing senders to know that an encrypted connection should be enforced.
* The per-IP connection limit to the IMAP server has been doubled to allow more devices to connect at once, especially with multiple users behind a NAT.

DNS:

* autoconfig and autodiscover subdomains and CalDAV/CardDAV SRV records are no longer generated for domains that don't have user accounts since they are unnecessary.
* IPv6 addresses can now be specified for secondary DNS nameservers in the control panel.

TLS:

* TLS certificates are now provisioned in groups by parent domain to limit easy domain enumeration and make provisioning more resilient to errors for particular domains.

Control Panel:

* The control panel API is now fully documented at https://mailinabox.email/api-docs.html.
* User passwords can now have spaces.
* Status checks for automatic subdomains have been moved into the section for the parent domain.
* Typo fixed.

Web:

* The default web page served on fresh installations now adds the `noindex` meta tag.
* The HSTS header is revised to also be sent on non-success responses.

# gpg verification failed.

# Conflicts:
#	.gitignore
#	setup/bootstrap.sh
This commit is contained in:
John Supplee 2020-10-11 18:16:36 +02:00
commit 38ac127344
26 changed files with 2886 additions and 75 deletions

1
.gitignore vendored
View file

@ -6,3 +6,4 @@ externals/
.env
.vagrant
.idea/
api/docs/api-docs.html

View file

@ -1,6 +1,39 @@
CHANGELOG
=========
v0.50 (September 25, 2020)
--------------------------
Setup:
* When upgrading from versions before v0.40, setup will now warn that ownCloud/Nextcloud data cannot be migrated rather than failing the installation.
Mail:
* An MTA-STS policy for incoming mail is now published (in DNS and over HTTPS) when the primary hostname and email address domain both have a signed TLS certificate installed, allowing senders to know that an encrypted connection should be enforced.
* The per-IP connection limit to the IMAP server has been doubled to allow more devices to connect at once, especially with multiple users behind a NAT.
DNS:
* autoconfig and autodiscover subdomains and CalDAV/CardDAV SRV records are no longer generated for domains that don't have user accounts since they are unnecessary.
* IPv6 addresses can now be specified for secondary DNS nameservers in the control panel.
TLS:
* TLS certificates are now provisioned in groups by parent domain to limit easy domain enumeration and make provisioning more resilient to errors for particular domains.
Control Panel:
* The control panel API is now fully documented at https://mailinabox.email/api-docs.html.
* User passwords can now have spaces.
* Status checks for automatic subdomains have been moved into the section for the parent domain.
* Typo fixed.
Web:
* The default web page served on fresh installations now adds the `noindex` meta tag.
* The HSTS header is revised to also be sent on non-success responses.
v0.48 (August 26, 2020)
-----------------------

View file

@ -69,6 +69,10 @@ Issues
Changes
-------
### v0.50-quota-0.22-beta
* Update to v0.50 of Mail-in-a-Box
### v0.48-quota-0.22-beta
* Update to v0.48 of Mail-in-a-Box
@ -199,27 +203,34 @@ Our goals are to:
Additionally, this project has a [Code of Conduct](CODE_OF_CONDUCT.md), which supersedes the goals above. Please review it when joining our community.
The Box
-------
In The Box
----------
Mail-in-a-Box turns a fresh Ubuntu 18.04 LTS 64-bit machine into a working mail server by installing and configuring various components.
It is a one-click email appliance. There are no user-configurable setup options. It "just works".
It is a one-click email appliance. There are no user-configurable setup options. It "just works."
The components installed are:
* SMTP ([postfix](http://www.postfix.org/)), IMAP ([dovecot](http://dovecot.org/)), CardDAV/CalDAV ([Nextcloud](https://nextcloud.com/)), Exchange ActiveSync ([z-push](http://z-push.org/))
* Webmail ([Roundcube](http://roundcube.net/)), static website hosting ([nginx](http://nginx.org/))
* Spam filtering ([spamassassin](https://spamassassin.apache.org/)), greylisting ([postgrey](http://postgrey.schweikert.ch/))
* DNS ([nsd4](https://www.nlnetlabs.nl/projects/nsd/)) with [SPF](https://en.wikipedia.org/wiki/Sender_Policy_Framework), DKIM ([OpenDKIM](http://www.opendkim.org/)), [DMARC](https://en.wikipedia.org/wiki/DMARC), [DNSSEC](https://en.wikipedia.org/wiki/DNSSEC), [DANE TLSA](https://en.wikipedia.org/wiki/DNS-based_Authentication_of_Named_Entities), and [SSHFP](https://tools.ietf.org/html/rfc4255) records automatically set
* Backups ([duplicity](http://duplicity.nongnu.org/)), firewall ([ufw](https://launchpad.net/ufw)), intrusion protection ([fail2ban](http://www.fail2ban.org/wiki/index.php/Main_Page)), system monitoring ([munin](http://munin-monitoring.org/))
* SMTP ([postfix](http://www.postfix.org/)), IMAP ([Dovecot](http://dovecot.org/)), CardDAV/CalDAV ([Nextcloud](https://nextcloud.com/)), and Exchange ActiveSync ([z-push](http://z-push.org/)) servers
* Webmail ([Roundcube](http://roundcube.net/)), mail filter rules (thanks to Roundcube and Dovecot), and email client autoconfig settings (served by [nginx](http://nginx.org/))
* Spam filtering ([spamassassin](https://spamassassin.apache.org/)) and greylisting ([postgrey](http://postgrey.schweikert.ch/))
* DNS ([nsd4](https://www.nlnetlabs.nl/projects/nsd/)) with [SPF](https://en.wikipedia.org/wiki/Sender_Policy_Framework), DKIM ([OpenDKIM](http://www.opendkim.org/)), [DMARC](https://en.wikipedia.org/wiki/DMARC), [DNSSEC](https://en.wikipedia.org/wiki/DNSSEC), [DANE TLSA](https://en.wikipedia.org/wiki/DNS-based_Authentication_of_Named_Entities), [MTA-STS](https://tools.ietf.org/html/rfc8461), and [SSHFP](https://tools.ietf.org/html/rfc4255) policy records automatically set
* TLS certificates are automatically provisioned using [Let's Encrypt](https://letsencrypt.org/) for protecting https and all of the other services on the box
* Backups ([duplicity](http://duplicity.nongnu.org/)), firewall ([ufw](https://launchpad.net/ufw)), intrusion protection ([fail2ban](http://www.fail2ban.org/wiki/index.php/Main_Page)), and basic system monitoring ([munin](http://munin-monitoring.org/))
It also includes:
It also includes system management tools:
* A control panel and API for adding/removing mail users, aliases, custom DNS records, etc. and detailed system monitoring.
* Comprehensive health monitoring that checks each day that services are running, ports are open, TLS certificates are valid, and DNS records are correct
* A control panel for adding/removing mail users, aliases, custom DNS records, configuring backups, etc.
* An API for all of the actions on the control panel
It also supports static website hosting since the box is serving HTTPS anyway. (To serve a website for your domains elsewhere, just add a custom DNS "A" record in you Mail-in-a-Box's control panel to point domains to another server.)
For more information on how Mail-in-a-Box handles your privacy, see the [security details page](security.md).
Installation
------------
@ -238,7 +249,7 @@ by him:
$ curl -s https://keybase.io/joshdata/key.asc | gpg --import
gpg: key C10BDD81: public key "Joshua Tauberer <jt@occams.info>" imported
$ git verify-tag v0.48
$ git verify-tag v0.50
gpg: Signature made ..... using RSA key ID C10BDD81
gpg: Good signature from "Joshua Tauberer <jt@occams.info>"
gpg: WARNING: This key is not certified with a trusted signature!
@ -251,7 +262,7 @@ and on his [personal homepage](https://razor.occams.info/). (Of course, if this
Checkout the tag corresponding to the most recent release:
$ git checkout v0.48
$ git checkout v0.50
Begin the installation.
@ -261,6 +272,9 @@ For help, DO NOT contact Josh directly --- I don't do tech support by email or t
Post your question on the [discussion forum](https://discourse.mailinabox.email/) instead, where maintainers and Mail-in-a-Box users may be able to help you.
Note that while we want everything to "just work," we can't control the rest of the Internet. Other mail services might block or spam-filter email sent from your Mail-in-a-Box.
This is a challenge faced by everyone who runs their own mail server, with or without Mail-in-a-Box. See our discussion forum for tips about that.
Contributing and Development
----------------------------
@ -274,6 +288,7 @@ This project was inspired in part by the ["NSA-proof your email in 2 hours"](htt
Mail-in-a-Box is similar to [iRedMail](http://www.iredmail.org/) and [Modoboa](https://github.com/tonioo/modoboa).
The History
-----------

23
api/docs/generate-docs.sh Executable file
View file

@ -0,0 +1,23 @@
#!/usr/bin/env sh
# Requirements:
# - Node.js
# - redoc-cli (`npm install redoc-cli -g`)
redoc-cli bundle ../mailinabox.yml \
-t template.hbs \
-o api-docs.html \
--templateOptions.metaDescription="Mail-in-a-Box HTTP API" \
--title="Mail-in-a-Box HTTP API" \
--options.expandSingleSchemaField \
--options.hideSingleRequestSampleTab \
--options.jsonSampleExpandLevel=10 \
--options.hideDownloadButton \
--options.theme.logo.maxHeight=180px \
--options.theme.logo.maxWidth=180px \
--options.theme.colors.primary.main="#C52" \
--options.theme.typography.fontSize=16px \
--options.theme.typography.fontFamily="Raleway, sans-serif" \
--options.theme.typography.headings.fontFamily="Ubuntu, Arial, sans-serif" \
--options.theme.typography.code.fontSize=15px \
--options.theme.typography.code.fontFamily='"Source Code Pro", monospace'

31
api/docs/template.hbs Normal file
View file

@ -0,0 +1,31 @@
<!DOCTYPE html>
<html>
<head>
<meta charset="utf8" />
<title>{{title}}</title>
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="description" content="{{templateOptions.metaDescription}}" />
<link rel="icon" type="image/png" href="https://mailinabox.email/static/logo_small.png">
<link rel="apple-touch-icon" type="image/png" href="https://mailinabox.email/static/logo_small.png">
<link href="https://fonts.googleapis.com/css?family=Raleway:400,700" rel="stylesheet" />
<link href="https://fonts.googleapis.com/css?family=Ubuntu:300" rel="stylesheet" />
<link href="https://fonts.googleapis.com/css?family=Source+Code+Pro:500" rel="stylesheet" />
<style>
body {
margin: 0;
padding: 0;
}
h1 {
color: #000 !important;
}
</style>
{{{redocHead}}}
</head>
<body>
{{{redocHTML}}}
</body>
</html>

2531
api/mailinabox.yml Normal file

File diff suppressed because it is too large Load diff

4
conf/mta-sts.txt Normal file
View file

@ -0,0 +1,4 @@
version: STSv1
mode: MODE
mx: PRIMARY_HOSTNAME
max_age: 86400

View file

@ -21,6 +21,9 @@
location = /mail/config-v1.1.xml {
alias /var/lib/mailinabox/mozilla-autoconfig.xml;
}
location = /.well-known/mta-sts.txt {
alias /var/lib/mailinabox/mta-sts.txt;
}
# Roundcube Webmail configuration.
rewrite ^/mail$ /mail/ redirect;

View file

@ -1,6 +1,7 @@
<html>
<head>
<title>this is a mail-in-a-box</title>
<meta name="robots" content="noindex">
</head>
<body>
<h1>this is a mail-in-a-box</h1>

View file

@ -460,9 +460,8 @@ def system_status():
self.items[-1]["extra"].append({ "text": message, "monospace": monospace })
output = WebOutput()
# Create a temporary pool of processes for the status checks
pool = multiprocessing.pool.Pool(processes=5)
run_checks(False, env, output, pool)
pool.terminate()
with multiprocessing.pool.Pool(processes=5) as pool:
run_checks(False, env, output, pool)
return json_response(output.items)
@app.route('/system/updates')

View file

@ -16,10 +16,10 @@ if [ `date "+%u"` -eq 1 ]; then
fi
# Take a backup.
management/backup.py | management/email_administrator.py "Backup Status"
management/backup.py 2>&1 | management/email_administrator.py "Backup Status"
# Provision any new certificates for new domains or domains with expiring certificates.
management/ssl_certificates.py -q | management/email_administrator.py "TLS Certificate Provisioning Result"
management/ssl_certificates.py -q 2>&1 | management/email_administrator.py "TLS Certificate Provisioning Result"
# Run status checks and email the administrator if anything changed.
management/status_checks.py --show-changes | management/email_administrator.py "Status Checks Change Notice"
management/status_checks.py --show-changes 2>&1 | management/email_administrator.py "Status Checks Change Notice"

View file

@ -9,8 +9,9 @@ import ipaddress
import rtyaml
import dns.resolver
from mailconfig import get_mail_domains
from mailconfig import get_mail_domains, get_mail_aliases
from utils import shell, load_env_vars_from_file, safe_domain_name, sort_domains
from ssl_certificates import get_ssl_certificates, check_certificate
# From https://stackoverflow.com/questions/3026957/how-to-validate-a-domain-name-using-regex-php/16491074#16491074
# This regular expression matches domain names according to RFCs, it also accepts fqdn with an leading dot,
@ -280,25 +281,81 @@ def build_zone(domain, all_domains, additional_records, www_redirect_domains, en
if not has_rec(dmarc_qname, "TXT", prefix="v=DMARC1; "):
records.append((dmarc_qname, "TXT", 'v=DMARC1; p=reject', "Recommended. Prevents use of this domain name for outbound mail by specifying that the SPF rule should be honoured for mail from @%s." % (qname + "." + domain)))
# Add CardDAV/CalDAV SRV records on the non-primary hostname that points to the primary hostname.
# Add CardDAV/CalDAV SRV records on the non-primary hostname that points to the primary hostname
# for autoconfiguration of mail clients (so only domains hosting user accounts need it).
# The SRV record format is priority (0, whatever), weight (0, whatever), port, service provider hostname (w/ trailing dot).
if domain != env["PRIMARY_HOSTNAME"]:
if domain != env["PRIMARY_HOSTNAME"] and domain in get_mail_domains(env, users_only=True):
for dav in ("card", "cal"):
qname = "_" + dav + "davs._tcp"
if not has_rec(qname, "SRV"):
records.append((qname, "SRV", "0 0 443 " + env["PRIMARY_HOSTNAME"] + ".", "Recommended. Specifies the hostname of the server that handles CardDAV/CalDAV services for email addresses on this domain."))
# Adds autoconfiguration A records for all domains.
# Adds autoconfiguration A records for all domains that there are user accounts at.
# This allows the following clients to automatically configure email addresses in the respective applications.
# autodiscover.* - Z-Push ActiveSync Autodiscover
# autoconfig.* - Thunderbird Autoconfig
autodiscover_records = [
("autodiscover", "A", env["PUBLIC_IP"], "Provides email configuration autodiscovery support for Z-Push ActiveSync Autodiscover."),
("autodiscover", "AAAA", env["PUBLIC_IPV6"], "Provides email configuration autodiscovery support for Z-Push ActiveSync Autodiscover."),
("autoconfig", "A", env["PUBLIC_IP"], "Provides email configuration autodiscovery support for Thunderbird Autoconfig."),
("autoconfig", "AAAA", env["PUBLIC_IPV6"], "Provides email configuration autodiscovery support for Thunderbird Autoconfig.")
if domain in get_mail_domains(env, users_only=True):
autodiscover_records = [
("autodiscover", "A", env["PUBLIC_IP"], "Provides email configuration autodiscovery support for Z-Push ActiveSync Autodiscover."),
("autodiscover", "AAAA", env["PUBLIC_IPV6"], "Provides email configuration autodiscovery support for Z-Push ActiveSync Autodiscover."),
("autoconfig", "A", env["PUBLIC_IP"], "Provides email configuration autodiscovery support for Thunderbird Autoconfig."),
("autoconfig", "AAAA", env["PUBLIC_IPV6"], "Provides email configuration autodiscovery support for Thunderbird Autoconfig.")
]
for qname, rtype, value, explanation in autodiscover_records:
if value is None or value.strip() == "": continue # skip IPV6 if not set
if not has_rec(qname, rtype):
records.append((qname, rtype, value, explanation))
# If this is a domain name that there are email addresses configured for, i.e. "something@"
# this domain name, then the domain name is a MTA-STS (https://tools.ietf.org/html/rfc8461)
# Policy Domain.
#
# A "_mta-sts" TXT record signals the presence of a MTA-STS policy. The id field helps clients
# cache the policy. It should be stable so we don't update DNS unnecessarily but change when
# the policy changes. It must be at most 32 letters and numbers, so we compute a hash of the
# policy file.
#
# The policy itself is served at the "mta-sts" (no underscore) subdomain over HTTPS. Therefore
# the TLS certificate used by Postfix for STARTTLS must be a valid certificate for the MX
# domain name (PRIMARY_HOSTNAME) *and* the TLS certificate used by nginx for HTTPS on the mta-sts
# subdomain must be valid certificate for that domain. Do not set an MTA-STS policy if either
# certificate in use is not valid (e.g. because it is self-signed and a valid certificate has not
# yet been provisioned). Since we cannot provision a certificate without A/AAAA records, we
# always set them --- only the TXT records depend on there being valid certificates.
mta_sts_enabled = False
mta_sts_records = [
("mta-sts", "A", env["PUBLIC_IP"], "Optional. MTA-STS Policy Host serving /.well-known/mta-sts.txt."),
("mta-sts", "AAAA", env.get('PUBLIC_IPV6'), "Optional. MTA-STS Policy Host serving /.well-known/mta-sts.txt."),
]
for qname, rtype, value, explanation in autodiscover_records:
if domain in get_mail_domains(env):
# Check that PRIMARY_HOSTNAME and the mta_sts domain both have valid certificates.
for d in (env['PRIMARY_HOSTNAME'], "mta-sts." + domain):
cert = get_ssl_certificates(env).get(d)
if not cert:
break # no certificate provisioned for this domain
cert_status = check_certificate(d, cert['certificate'], cert['private-key'])
if cert_status[0] != 'OK':
break # certificate is not valid
else:
# 'break' was not encountered above, so both domains are good
mta_sts_enabled = True
if mta_sts_enabled:
# Compute an up-to-32-character hash of the policy file. We'll take a SHA-1 hash of the policy
# file (20 bytes) and encode it as base-64 (28 bytes, using alphanumeric alternate characters
# instead of '+' and '/' which are not allowed in an MTA-STS policy id) but then just take its
# first 20 characters, which is more than sufficient to change whenever the policy file changes
# (and ensures any '=' padding at the end of the base64 encoding is dropped).
with open("/var/lib/mailinabox/mta-sts.txt", "rb") as f:
mta_sts_policy_id = base64.b64encode(hashlib.sha1(f.read()).digest(), altchars=b"AA").decode("ascii")[0:20]
mta_sts_records.extend([
("_mta-sts", "TXT", "v=STSv1; id=" + mta_sts_policy_id, "Optional. Part of the MTA-STS policy for incoming mail. If set, a MTA-STS policy must also be published.")
])
# Enable SMTP TLS reporting (https://tools.ietf.org/html/rfc8460) if the user has set a config option.
# Skip if the rules below if the user has set a custom _smtp._tls record.
if env.get("MTA_STS_TLSRPT_RUA") and not has_rec("_smtp._tls", "TXT", prefix="v=TLSRPTv1;"):
mta_sts_records.append(("_smtp._tls", "TXT", "v=TLSRPTv1; rua=" + env["MTA_STS_TLSRPT_RUA"], "Optional. Enables MTA-STS reporting."))
for qname, rtype, value, explanation in mta_sts_records:
if value is None or value.strip() == "": continue # skip IPV6 if not set
if not has_rec(qname, rtype):
records.append((qname, rtype, value, explanation))
@ -906,18 +963,19 @@ def set_secondary_dns(hostnames, env):
try:
response = resolver.query(item, "A")
except (dns.resolver.NoNameservers, dns.resolver.NXDOMAIN, dns.resolver.NoAnswer):
raise ValueError("Could not resolve the IP address of %s." % item)
try:
response = resolver.query(item, "AAAA")
except (dns.resolver.NoNameservers, dns.resolver.NXDOMAIN, dns.resolver.NoAnswer):
raise ValueError("Could not resolve the IP address of %s." % item)
else:
# Validate IP address.
try:
if "/" in item[4:]:
v = ipaddress.ip_network(item[4:]) # raises a ValueError if there's a problem
if not isinstance(v, ipaddress.IPv4Network): raise ValueError("That's an IPv6 subnet.")
else:
v = ipaddress.ip_address(item[4:]) # raises a ValueError if there's a problem
if not isinstance(v, ipaddress.IPv4Address): raise ValueError("That's an IPv6 address.")
except ValueError:
raise ValueError("'%s' is not an IPv4 address or subnet." % item[4:])
raise ValueError("'%s' is not an IPv4 or IPv6 address or subnet." % item[4:])
# Set.
set_custom_dns_record("_secondary_nameserver", "A", " ".join(hostnames), "set", env)

View file

@ -307,13 +307,15 @@ def get_domain(emailaddr, as_unicode=True):
pass
return ret
def get_mail_domains(env, filter_aliases=lambda alias : True):
def get_mail_domains(env, filter_aliases=lambda alias : True, users_only=False):
# Returns the domain names (IDNA-encoded) of all of the email addresses
# configured on the system.
return set(
[get_domain(login, as_unicode=False) for login in get_mail_users(env)]
+ [get_domain(address, as_unicode=False) for address, *_ in get_mail_aliases(env) if filter_aliases(address) ]
)
# configured on the system. If users_only is True, only return domains
# with email addresses that correspond to user accounts.
domains = []
domains.extend([get_domain(login, as_unicode=False) for login in get_mail_users(env)])
if not users_only:
domains.extend([get_domain(address, as_unicode=False) for address, *_ in get_mail_aliases(env) if filter_aliases(address) ])
return set(domains)
def add_mail_user(email, pw, privs, quota, env):
# validate email
@ -715,8 +717,6 @@ def validate_password(pw):
# validate password
if pw.strip() == "":
raise ValueError("No password provided.")
if re.search(r"[\s]", pw):
raise ValueError("Passwords cannot contain spaces.")
if len(pw) < 8:
raise ValueError("Passwords must be at least eight characters.")

View file

@ -180,7 +180,7 @@ def get_certificates_to_provision(env, limit_domains=None, show_valid_certs=True
# for and subtract:
# * domains not in limit_domains if limit_domains is not empty
# * domains with custom "A" records, i.e. they are hosted elsewhere
# * domains with actual "A" records that point elsewhere
# * domains with actual "A" records that point elsewhere (misconfiguration)
# * domains that already have certificates that will be valid for a while
from web_update import get_web_domains
@ -256,15 +256,41 @@ def provision_certificates(env, limit_domains):
"result": "skipped",
})
# Break into groups by DNS zone: Group every domain with its parent domain, if
# its parent domain is in the list of domains to request a certificate for.
# Start with the zones so that if the zone doesn't need a certificate itself,
# its children will still be grouped together. Sort the provision domains to
# put parents ahead of children.
# Since Let's Encrypt requests are limited to 100 domains at a time,
# we'll create a list of lists of domains where the inner lists have
# at most 100 items. By sorting we also get the DNS zone domain as the first
# entry in each list (unless we overflow beyond 100) which ends up as the
# primary domain listed in each certificate.
from dns_update import get_dns_zones
certs = { }
for zone, zonefile in get_dns_zones(env):
certs[zone] = [[]]
for domain in sort_domains(domains, env):
# Does the domain end with any domain we've seen so far.
for parent in certs.keys():
if domain.endswith("." + parent):
# Add this to the parent's list of domains.
# Start a new group if the list already has
# 100 items.
if len(certs[parent][-1]) == 100:
certs[parent].append([])
certs[parent][-1].append(domain)
break
else:
# This domain is not a child of any domain we've seen yet, so
# start a new group. This shouldn't happen since every zone
# was already added.
certs[domain] = [[domain]]
# Break into groups of up to 100 certificates at a time, which is Let's Encrypt's
# limit for a single certificate. We'll sort to put related domains together.
max_domains_per_group = 100
domains = sort_domains(domains, env)
certs = []
while len(domains) > 0:
certs.append( domains[:max_domains_per_group] )
domains = domains[max_domains_per_group:]
# Flatten to a list of lists of domains (from a mapping). Remove empty
# lists (zones with no domains that need certs).
certs = sum(certs.values(), [])
certs = [_ for _ in certs if len(_) > 0]
# Prepare to provision.

View file

@ -5,11 +5,13 @@
# what to do next.
import sys, os, os.path, re, subprocess, datetime, multiprocessing.pool
import asyncio
import dns.reversename, dns.resolver
import dateutil.parser, dateutil.tz
import idna
import psutil
import postfix_mta_sts_resolver.resolver
from dns_update import get_dns_zones, build_tlsa_record, get_custom_dns_config, get_secondary_dns, get_custom_dns_records
from web_update import get_web_domains, get_domains_with_a_records
@ -309,6 +311,17 @@ def run_domain_checks(rounded_time, env, output, pool):
domains_to_check = mail_domains | dns_domains | web_domains
# Remove "www", "autoconfig", "autodiscover", and "mta-sts" subdomains, which we group with their parent,
# if their parent is in the domains to check list.
domains_to_check = [
d for d in domains_to_check
if not (
d.split(".", 1)[0] in ("www", "autoconfig", "autodiscover", "mta-sts")
and len(d.split(".", 1)) == 2
and d.split(".", 1)[1] in domains_to_check
)
]
# Get the list of domains that we don't serve web for because of a custom CNAME/A record.
domains_with_a_records = get_domains_with_a_records(env)
@ -327,6 +340,11 @@ def run_domain_checks(rounded_time, env, output, pool):
def run_domain_checks_on_domain(domain, rounded_time, env, dns_domains, dns_zonefiles, mail_domains, web_domains, domains_with_a_records):
output = BufferedOutput()
# When running inside Flask, the worker threads don't get a thread pool automatically.
# Also this method is called in a forked worker pool, so creating a new loop is probably
# a good idea.
asyncio.set_event_loop(asyncio.new_event_loop())
# we'd move this up, but this returns non-pickleable values
ssl_certificates = get_ssl_certificates(env)
@ -354,6 +372,26 @@ def run_domain_checks_on_domain(domain, rounded_time, env, dns_domains, dns_zone
if domain in dns_domains:
check_dns_zone_suggestions(domain, env, output, dns_zonefiles, domains_with_a_records)
# Check auto-configured subdomains. See run_domain_checks.
# Skip mta-sts because we check the policy directly.
for label in ("www", "autoconfig", "autodiscover"):
subdomain = label + "." + domain
if subdomain in web_domains or subdomain in mail_domains:
# Run checks.
subdomain_output = run_domain_checks_on_domain(subdomain, rounded_time, env, dns_domains, dns_zonefiles, mail_domains, web_domains, domains_with_a_records)
# Prepend the domain name to the start of each check line, and then add to the
# checks for this domain.
for attr, args, kwargs in subdomain_output[1].buf:
if attr == "add_heading":
# Drop the heading, but use its text as the subdomain name in
# each line since it is in Unicode form.
subdomain = args[0]
continue
if len(args) == 1 and isinstance(args[0], str):
args = [ subdomain + ": " + args[0] ]
getattr(output, attr)(*args, **kwargs)
return (domain, output)
def check_primary_hostname_dns(domain, env, output, dns_domains, dns_zonefiles):
@ -611,6 +649,19 @@ def check_mail_domain(domain, env, output):
if mx != recommended_mx:
good_news += " This configuration is non-standard. The recommended configuration is '%s'." % (recommended_mx,)
output.print_ok(good_news)
# Check MTA-STS policy.
loop = asyncio.get_event_loop()
sts_resolver = postfix_mta_sts_resolver.resolver.STSResolver(loop=loop)
valid, policy = loop.run_until_complete(sts_resolver.resolve(domain))
if valid == postfix_mta_sts_resolver.resolver.STSFetchResult.VALID:
if policy[1].get("mx") == [env['PRIMARY_HOSTNAME']] and policy[1].get("mode") == "enforce": # policy[0] is the policyid
output.print_ok("MTA-STS policy is present.")
else:
output.print_error("MTA-STS policy is present but has unexpected settings. [{}]".format(policy[1]))
else:
output.print_error("MTA-STS policy is missing: {}".format(valid))
else:
output.print_error("""This domain's DNS MX record is incorrect. It is currently set to '%s' but should be '%s'. Mail will not
be delivered to this box. It may take several hours for public DNS to update after a change. This problem may result from
@ -970,13 +1021,14 @@ if __name__ == "__main__":
from utils import load_environment
env = load_environment()
pool = multiprocessing.pool.Pool(processes=10)
if len(sys.argv) == 1:
run_checks(False, env, ConsoleOutput(), pool)
with multiprocessing.pool.Pool(processes=10) as pool:
run_checks(False, env, ConsoleOutput(), pool)
elif sys.argv[1] == "--show-changes":
run_and_output_changes(env, pool)
with multiprocessing.pool.Pool(processes=10) as pool:
run_and_output_changes(env, pool)
elif sys.argv[1] == "--check-primary-hostname":
# See if the primary hostname appears resolvable and has a signed certificate.

View file

@ -288,7 +288,7 @@ function aliases_remove(elem) {
},
function(r) {
// Responses are multiple lines of pre-formatted text.
show_modal_error("Remove User", $("<pre/>").text(r));
show_modal_error("Remove Alias", $("<pre/>").text(r));
show_aliases();
});
});

View file

@ -90,7 +90,7 @@
<div class="col-sm-offset-1 col-sm-11">
<p class="small">
Multiple secondary servers can be separated with commas or spaces (i.e., <code>ns2.hostingcompany.com ns3.hostingcompany.com</code>).
To enable zone transfers to additional servers without listing them as secondary nameservers, add an IP address or subnet using <code>xfr:10.20.30.40</code> or <code>xfr:10.20.30.40/24</code>.
To enable zone transfers to additional servers without listing them as secondary nameservers, add an IP address or subnet using <code>xfr:10.20.30.40</code> or <code>xfr:10.0.0.0/8</code>.
</p>
<p id="secondarydns-clear-instructions" style="display: none" class="small">
Clear the input field above and click Update to use this machine itself as secondary DNS, which is the default/normal setup.

View file

@ -24,11 +24,14 @@ def get_web_domains(env, include_www_redirects=True, exclude_dns_elsewhere=True)
# the topmost of each domain we serve.
domains |= set('www.' + zone for zone, zonefile in get_dns_zones(env))
# Add Autoconfiguration domains, allowing us to serve correct SSL certs.
# Add Autoconfiguration domains for domains that there are user accounts at:
# 'autoconfig.' for Mozilla Thunderbird auto setup.
# 'autodiscover.' for Activesync autodiscovery.
domains |= set('autoconfig.' + maildomain for maildomain in get_mail_domains(env))
domains |= set('autodiscover.' + maildomain for maildomain in get_mail_domains(env))
domains |= set('autoconfig.' + maildomain for maildomain in get_mail_domains(env, users_only=True))
domains |= set('autodiscover.' + maildomain for maildomain in get_mail_domains(env, users_only=True))
# 'mta-sts.' for MTA-STS support for all domains that have email addresses.
domains |= set('mta-sts.' + maildomain for maildomain in get_mail_domains(env))
if exclude_dns_elsewhere:
# ...Unless the domain has an A/AAAA record that maps it to a different
@ -155,9 +158,23 @@ def make_domain_config(domain, templates, ssl_certificates, env):
# any proxy or redirect here?
for path, url in yaml.get("proxies", {}).items():
# Parse some flags in the fragment of the URL.
pass_http_host_header = False
m = re.search("#(.*)$", url)
if m:
for flag in m.group(1).split(","):
if flag == "pass-http-host":
pass_http_host_header = True
url = re.sub("#(.*)$", "", url)
nginx_conf_extra += "\tlocation %s {" % path
nginx_conf_extra += "\n\t\tproxy_pass %s;" % url
if pass_http_host_header:
nginx_conf_extra += "\n\t\tproxy_set_header Host $http_host;"
nginx_conf_extra += "\n\t\tproxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;"
nginx_conf_extra += "\n\t\tproxy_set_header X-Forwarded-Host $http_host;"
nginx_conf_extra += "\n\t\tproxy_set_header X-Forwarded-Proto $scheme;"
nginx_conf_extra += "\n\t\tproxy_set_header X-Real-IP $remote_addr;"
nginx_conf_extra += "\n\t}\n"
for path, alias in yaml.get("aliases", {}).items():
nginx_conf_extra += "\tlocation %s {" % path
@ -171,9 +188,9 @@ def make_domain_config(domain, templates, ssl_certificates, env):
# Add the HSTS header.
if hsts == "yes":
nginx_conf_extra += "add_header Strict-Transport-Security max-age=15768000;\n"
nginx_conf_extra += "add_header Strict-Transport-Security \"max-age=15768000\" always;\n"
elif hsts == "preload":
nginx_conf_extra += "add_header Strict-Transport-Security \"max-age=15768000; includeSubDomains; preload\";\n"
nginx_conf_extra += "add_header Strict-Transport-Security \"max-age=15768000; includeSubDomains; preload\" always;\n"
# Add in any user customizations in the includes/ folder.
nginx_conf_custom_include = os.path.join(env["STORAGE_ROOT"], "www", safe_domain_name(domain) + ".conf")

View file

@ -101,9 +101,13 @@ The box restricts the envelope sender address (also called the return path or MA
Incoming Mail
-------------
### Encryption
### Encryption Settings
As discussed above, there is no way to require on-the-wire encryption of mail. When the box receives an incoming email (SMTP on port 25), it offers encryption (STARTTLS) but cannot require that senders use it because some senders may not support STARTTLS at all and other senders may support STARTTLS but not with the latest protocols/ciphers. To give senders the best chance at making use of encryption, the box offers protocols back to TLSv1 and ciphers with key lengths as low as 112 bits. Modern clients (senders) will make use of the 256-bit ciphers and Diffie-Hellman ciphers with a 2048-bit key for perfect forward secrecy, however. ([source](setup/mail-postfix.sh))
As with outbound email, there is no way to require on-the-wire encryption of incoming mail from all senders. When the box receives an incoming email (SMTP on port 25), it offers encryption (STARTTLS) but cannot require that senders use it because some senders may not support STARTTLS at all and other senders may support STARTTLS but not with the latest protocols/ciphers. To give senders the best chance at making use of encryption, the box offers protocols back to TLSv1 and ciphers with key lengths as low as 112 bits. Modern clients (senders) will make use of the 256-bit ciphers and Diffie-Hellman ciphers with a 2048-bit key for perfect forward secrecy, however. ([source](setup/mail-postfix.sh))
### MTA-STS
The box publishes a SMTP MTA Strict Transport Security ([SMTP MTA-STS](https://en.wikipedia.org/wiki/Simple_Mail_Transfer_Protocol#SMTP_MTA_Strict_Transport_Security)) policy (via DNS and HTTPS) in "enforce" mode. Senders that support MTA-STS will use a secure SMTP connection. (MTA-STS tells senders to connect and expect a signed TLS certificate for the "MX" domain without permitting a fallback to an unencrypted connection.)
### DANE

View file

@ -20,7 +20,7 @@ if [ -z "$TAG" ]; then
# want to display in status checks.
if [ "`lsb_release -d | sed 's/.*:\s*//' | sed 's/18\.04\.[0-9]/18.04/' `" == "Ubuntu 18.04 LTS" ]; then
# This machine is running Ubuntu 18.04.
TAG=v0.48-quota-0.22-beta
TAG=v0.50-quota-0.22-beta
elif [ "`lsb_release -d | sed 's/.*:\s*//' | sed 's/14\.04\.[0-9]/14.04/' `" == "Ubuntu 14.04 LTS" ]; then
# This machine is running Ubuntu 14.04.

View file

@ -146,7 +146,7 @@ service imap-login {
}
}
protocol imap {
mail_max_userip_connections = 20
mail_max_userip_connections = 40
}
EOF

View file

@ -50,7 +50,7 @@ hide_output $venv/bin/pip install --upgrade pip
hide_output $venv/bin/pip install --upgrade \
rtyaml "email_validator>=1.0.0" "exclusiveprocess" \
flask dnspython python-dateutil \
"idna>=2.0.0" "cryptography==2.2.2" boto psutil
"idna>=2.0.0" "cryptography==2.2.2" boto psutil postfix-mta-sts-resolver
# CONFIGURATION

View file

@ -134,11 +134,11 @@ if [ ! -d /usr/local/lib/owncloud/ ] || [[ ! ${CURRENT_NEXTCLOUD_VER} =~ ^$nextc
# Database migrations from ownCloud are no longer possible because ownCloud cannot be run under
# PHP 7.
if [[ ${CURRENT_NEXTCLOUD_VER} =~ ^[89] ]]; then
echo "Upgrades from Mail-in-a-Box prior to v0.28 (dated July 30, 2018) with Nextcloud < 13.0.6 (you have ownCloud 8 or 9) are not supported. Upgrade to Mail-in-a-Box version v0.30 first. Setup aborting."
exit 1
echo "Upgrades from Mail-in-a-Box prior to v0.28 (dated July 30, 2018) with Nextcloud < 13.0.6 (you have ownCloud 8 or 9) are not supported. Upgrade to Mail-in-a-Box version v0.30 first. Setup will continue, but skip the Nextcloud migration."
return 0
elif [[ ${CURRENT_NEXTCLOUD_VER} =~ ^1[012] ]]; then
echo "Upgrades from Mail-in-a-Box prior to v0.28 (dated July 30, 2018) with Nextcloud < 13.0.6 (you have ownCloud 10, 11 or 12) are not supported. Upgrade to Mail-in-a-Box version v0.30 first. Setup aborting."
exit 1
echo "Upgrades from Mail-in-a-Box prior to v0.28 (dated July 30, 2018) with Nextcloud < 13.0.6 (you have ownCloud 10, 11 or 12) are not supported. Upgrade to Mail-in-a-Box version v0.30 first. Setup will continue, but skip the Nextcloud migration."
return 0
elif [[ ${CURRENT_NEXTCLOUD_VER} =~ ^13 ]]; then
# If we are running Nextcloud 13, upgrade to Nextcloud 14
InstallNextcloud 14.0.6 4e43a57340f04c2da306c8eea98e30040399ae5a

View file

@ -82,9 +82,10 @@ if [ ! -f $STORAGE_ROOT/mailinabox.version ]; then
chown $STORAGE_USER.$STORAGE_USER $STORAGE_ROOT/mailinabox.version
fi
# Save the global options in /etc/mailinabox.conf so that standalone
# tools know where to look for data.
# tools know where to look for data. The default MTA_STS_MODE setting
# is blank unless set by an environment variable, but see web.sh for
# how that is interpreted.
cat > /etc/mailinabox.conf << EOF;
STORAGE_USER=$STORAGE_USER
STORAGE_ROOT=$STORAGE_ROOT
@ -93,6 +94,7 @@ PUBLIC_IP=$PUBLIC_IP
PUBLIC_IPV6=$PUBLIC_IPV6
PRIVATE_IP=$PRIVATE_IP
PRIVATE_IPV6=$PRIVATE_IPV6
MTA_STS_MODE=${MTA_STS_MODE-}
EOF
# Start service configuration.

View file

@ -19,7 +19,7 @@ fi
echo "Installing Nginx (web server)..."
apt_install nginx php-cli php-fpm
apt_install nginx php-cli php-fpm idn2
rm -f /etc/nginx/sites-enabled/default
@ -122,6 +122,21 @@ cat conf/mozilla-autoconfig.xml \
> /var/lib/mailinabox/mozilla-autoconfig.xml
chmod a+r /var/lib/mailinabox/mozilla-autoconfig.xml
# Create a generic mta-sts.txt file which is exposed via the
# nginx configuration at /.well-known/mta-sts.txt
# more documentation is available on:
# https://www.uriports.com/blog/mta-sts-explained/
# default mode is "enforce". Change to "testing" which means
# "Messages will be delivered as though there was no failure
# but a report will be sent if TLS-RPT is configured" if you
# are not sure you want this yet. Or "none".
PUNY_PRIMARY_HOSTNAME=$(echo "$PRIMARY_HOSTNAME" | idn2)
cat conf/mta-sts.txt \
| sed "s/MODE/${MTA_STS_MODE:-enforce}/" \
| sed "s/PRIMARY_HOSTNAME/$PUNY_PRIMARY_HOSTNAME/" \
> /var/lib/mailinabox/mta-sts.txt
chmod a+r /var/lib/mailinabox/mta-sts.txt
# make a default homepage
if [ -d $STORAGE_ROOT/www/static ]; then mv $STORAGE_ROOT/www/static $STORAGE_ROOT/www/default; fi # migration #NODOC
mkdir -p $STORAGE_ROOT/www/default
@ -137,4 +152,3 @@ restart_service php7.2-fpm
# Open ports.
ufw_allow http
ufw_allow https

View file

@ -33,9 +33,6 @@ def read_password():
if len(first) < 8:
print("Passwords must be at least eight characters.")
continue
if re.search(r'[\s]', first):
print("Passwords cannot contain spaces.")
continue
second = getpass.getpass(' (again): ')
if first != second:
print("Passwords not the same. Try again.")