Merge tag 'v0.52' of https://github.com/mail-in-a-box/mailinabox

v0.52 (January 31, 2021)
------------------------

Software updates:

* Upgraded Roundcube to version 1.4.10.
* Upgraded zpush to 2.6.1.

Mail:

* Incoming emails with SPF/DKIM/DMARC failures now get a higher spam score, and these messages are more likely to appear in the junk folder, since they are often spam/phishing.
* Fixed the MTA-STS policy file's line endings.

Control panel:

* A new Download button in the control panel's External DNS page can be used to download the required DNS records in zonefile format.
* Fixed the problem when the control panel would report DNS entries as Not Set by increasing a bind query limit.
* Fixed a control panel startup bug on some systems.
* Improved an error message on a DNS lookup timeout.
* A typo was fixed.

DNS:

* The TTL for NS records has been increased to 1 day to comply with some registrar requirements.

System:

* Nextcloud's photos, dashboard, and activity apps are disabled since we only support contacts and calendar.

# Conflicts:
#	setup/bootstrap.sh

CHANGELOG.md

@@ -1,6 +1,35 @@
 CHANGELOG
 =========
 
+v0.52 (January 31, 2021)
+------------------------
+
+Software updates:
+
+* Upgraded Roundcube to version 1.4.10.
+* Upgraded zpush to 2.6.1.
+
+Mail:
+
+* Incoming emails with SPF/DKIM/DMARC failures now get a higher spam score, and these messages are more likely to appear in the junk folder, since they are often spam/phishing.
+* Fixed the MTA-STS policy file's line endings.
+
+Control panel:
+
+* A new Download button in the control panel's External DNS page can be used to download the required DNS records in zonefile format.
+* Fixed the problem when the control panel would report DNS entries as Not Set by increasing a bind query limit.
+* Fixed a control panel startup bug on some systems.
+* Improved an error message on a DNS lookup timeout.
+* A typo was fixed.
+
+DNS:
+
+* The TTL for NS records has been increased to 1 day to comply with some registrar requirements.
+
+System:
+
+* Nextcloud's photos, dashboard, and activity apps are disabled since we only support contacts and calendar.
+
 v0.51 (November 14, 2020)
 -------------------------
 
@@ -13,7 +42,7 @@ Mail:
 
 * The MTA-STA max_age value was increased to the normal one week.
 
-Control Panel:
+Control panel:
 
 * Two-factor authentication can now be enabled for logins to the control panel. However, keep in mind that many online services (including domain name registrars, cloud server providers, and TLS certificate providers) may allow an attacker to take over your account or issue a fraudulent TLS certificate with only access to your email address, and this new two-factor authentication does not protect access to your inbox. It therefore remains very important that user accounts with administrative email addresses have strong passwords.
 * TLS certificate expiry dates are now shown in ISO8601 format for clarity.
@@ -39,7 +68,7 @@ TLS:
 
 * TLS certificates are now provisioned in groups by parent domain to limit easy domain enumeration and make provisioning more resilient to errors for particular domains.
 
-Control Panel:
+Control panel:
 
 * The control panel API is now fully documented at https://mailinabox.email/api-docs.html.
 * User passwords can now have spaces.

README.md

@@ -71,6 +71,10 @@ Issues
 Changes
 -------
 
+### v0.52-quota-0.22-beta
+
+* Update to v0.52 of Mail-in-a-Box
+
 ### v0.51-quota-0.22-beta
 
 * Update to v0.51 of Mail-in-a-Box
@@ -244,36 +248,18 @@ See the [setup guide](https://mailinabox.email/guide.html) for detailed, user-fr
 
 For experts, start with a completely fresh (really, I mean it) Ubuntu 18.04 LTS 64-bit machine. On the machine...
 
-Clone this repository:
+Clone this repository and checkout the tag corresponding to the most recent release:
 
 	$ git clone https://github.com/mail-in-a-box/mailinabox
 	$ cd mailinabox
-
+	$ git checkout v0.52
-_Optional:_ Download Josh's PGP key and then verify that the sources were signed
-by him:
-
-	$ curl -s https://keybase.io/joshdata/key.asc | gpg --import
-	gpg: key C10BDD81: public key "Joshua Tauberer <jt@occams.info>" imported
-
-	$ git verify-tag v0.51
-	gpg: Signature made ..... using RSA key ID C10BDD81
-	gpg: Good signature from "Joshua Tauberer <jt@occams.info>"
-	gpg: WARNING: This key is not certified with a trusted signature!
-	gpg:          There is no indication that the signature belongs to the owner.
-	Primary key fingerprint: 5F4C 0E73 13CC D744 693B  2AEA B920 41F4 C10B DD81
-
-You'll get a lot of warnings, but that's OK. Check that the primary key fingerprint matches the
-fingerprint in the key details at [https://keybase.io/joshdata](https://keybase.io/joshdata)
-and on his [personal homepage](https://razor.occams.info/). (Of course, if this repository has been compromised you can't trust these instructions.)
-
-Checkout the tag corresponding to the most recent release:
-
-	$ git checkout v0.51
 
 Begin the installation.
 
 	$ sudo setup/start.sh
 
+The installation will install, uninstall, and configure packages to turn the machine into a working, good mail server.
+
 For help, DO NOT contact Josh directly --- I don't do tech support by email or tweet (no exceptions).
 
 Post your question on the [discussion forum](https://discourse.mailinabox.email/) instead, where maintainers and Mail-in-a-Box users may be able to help you.
@@ -281,6 +267,7 @@ Post your question on the [discussion forum](https://discourse.mailinabox.email/
 Note that while we want everything to "just work," we can't control the rest of the Internet. Other mail services might block or spam-filter email sent from your Mail-in-a-Box.
 This is a challenge faced by everyone who runs their own mail server, with or without Mail-in-a-Box. See our discussion forum for tips about that.
 
+
 Contributing and Development
 ----------------------------
 

api/mailinabox.yml

@@ -15,7 +15,7 @@ info:
   license:
     name: CC0 1.0 Universal
     url: https://creativecommons.org/publicdomain/zero/1.0/legalcode
-  version: 0.47.0
+  version: 0.51.0
   x-logo:
     url: https://mailinabox.email/static/logo.png
     altText: Mail-in-a-Box logo
@@ -743,6 +743,38 @@ paths:
             text/html:
               schema:
                 type: string
+  /dns/zonefile/{zone}:
+    parameters:
+      - in: path
+        name: zone
+        schema:
+          $ref: '#/components/schemas/Hostname'
+        required: true
+        description: Hostname
+    get:
+      tags:
+        - DNS
+      summary: Get DNS zonefile
+      description: Returns a DNS zone file for a hostname.
+      operationId: getDnsZonefile
+      x-codeSamples:
+        - lang: curl
+          source: |
+            curl -X GET "https://{host}/admin/dns/zonefile/<zone>" \
+              -u "<email>:<password>"
+      responses:
+        200:
+          description: Successful operation
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/DNSZonefileResponse'
+        403:
+          description: Forbidden
+          content:
+            text/html:
+              schema:
+                type: string
   /dns/update:
     post:
       tags:
@@ -1781,7 +1813,7 @@ components:
         text/plain:
           schema:
             type: string
-            example: 1.2.3.4
+            example: '1.2.3.4'
             description: The value of the DNS record.
           example: '1.2.3.4'
   schemas:
@@ -2050,6 +2082,8 @@ components:
       items:
         $ref: '#/components/schemas/Hostname'
       description: DNS zones response.
+    DNSZonefileResponse:
+      type: string
     DNSSecondaryNameserverResponse:
       type: object
       required:
@@ -2663,13 +2697,6 @@ components:
           type: string
     MfaEnableSuccessResponse:
       type: string
-    MfaEnableBadRequestResponse:
-      type: object
-      required:
-        - error
-      properties:
-        error:
-          type: string
     MfaDisableRequest:
       type: object
       properties:

conf/mta-sts.txt

@@ -1,4 +1,4 @@
-version: STSv1
+version: STSv1
-mode: MODE
+mode: MODE
-mx: PRIMARY_HOSTNAME
+mx: PRIMARY_HOSTNAME
-max_age: 604800
+max_age: 604800

management/backup.py

@@ -456,6 +456,23 @@ def list_target_files(config):
 			raise ValueError(e.reason)
 
 		return [(key.name[len(path):], key.size) for key in bucket.list(prefix=path)]
+	elif target.scheme == 'b2':
+		from b2sdk.v1 import InMemoryAccountInfo, B2Api
+		from b2sdk.v1.exception import NonExistentBucket
+		info = InMemoryAccountInfo()
+		b2_api = B2Api(info)
+		
+		# Extract information from target
+		b2_application_keyid = target.netloc[:target.netloc.index(':')]
+		b2_application_key = target.netloc[target.netloc.index(':')+1:target.netloc.index('@')]
+		b2_bucket = target.netloc[target.netloc.index('@')+1:]
+
+		try:
+			b2_api.authorize_account("production", b2_application_keyid, b2_application_key)
+			bucket = b2_api.get_bucket_by_name(b2_bucket)
+		except NonExistentBucket as e:
+			raise ValueError("B2 Bucket does not exist. Please double check your information!")
+		return [(key.file_name, key.size) for key, _ in bucket.ls()]
 
 	else:
 		raise ValueError(config["target"])

management/daemon.py

@@ -1,3 +1,12 @@
+#!/usr/local/lib/mailinabox/env/bin/python3
+#
+# During development, you can start the Mail-in-a-Box control panel
+# by running this script, e.g.:
+#
+# service mailinabox stop # stop the system process
+# DEBUG=1 management/daemon.py
+# service mailinabox start # when done debugging, start it up again
+
 import os, os.path, re, json, time
 import multiprocessing.pool, subprocess
 
@@ -362,6 +371,12 @@ def dns_get_dump():
 	from dns_update import build_recommended_dns
 	return json_response(build_recommended_dns(env))
 
+@app.route('/dns/zonefile/<zone>')
+@authorized_personnel_only
+def dns_get_zonefile(zone):
+	from dns_update import get_dns_zonefile
+	return Response(get_dns_zonefile(zone, env), status=200, mimetype='text/plain')
+
 # SSL
 
 @app.route('/ssl/status')
@@ -719,7 +734,22 @@ def log_failed_login(request):
 # APP
 
 if __name__ == '__main__':
-	if "DEBUG" in os.environ: app.debug = True
+	if "DEBUG" in os.environ:
+		# Turn on Flask debugging.
+		app.debug = True
+
+		# Use a stable-ish master API key so that login sessions don't restart on each run.
+		# Use /etc/machine-id to seed the key with a stable secret, but add something
+		# and hash it to prevent possibly exposing the machine id, using the time so that
+		# the key is not valid indefinitely.
+		import hashlib
+		with open("/etc/machine-id") as f:
+			api_key = f.read()
+		api_key += "|" + str(int(time.time() / (60*60*2)))
+		hasher = hashlib.sha1()
+		hasher.update(api_key.encode("ascii"))
+		auth_service.key = hasher.hexdigest()
+
 	if "APIKEY" in os.environ: auth_service.key = os.environ["APIKEY"]
 
 	if not app.debug:

management/dns_update.py

@@ -470,14 +470,14 @@ def write_nsd_zone(domain, zonefile, records, env, force):
 
 	zone = """
 $ORIGIN {domain}.
-$TTL 1800           ; default time to live
+$TTL 86400          ; default time to live
 
 @ IN SOA ns1.{primary_domain}. hostmaster.{primary_domain}. (
            __SERIAL__     ; serial number
            7200     ; Refresh (secondary nameserver update interval)
-           1800     ; Retry (when refresh fails, how often to try again)
+           86400    ; Retry (when refresh fails, how often to try again)
            1209600  ; Expire (when refresh fails, how long secondary nameserver will keep records around anyway)
-           1800     ; Negative TTL (how long negative responses are cached)
+           86400    ; Negative TTL (how long negative responses are cached)
            )
 """
 
@@ -564,6 +564,17 @@ $TTL 1800           ; default time to live
 
 	return True # file is updated
 
+def get_dns_zonefile(zone, env):
+	for domain, fn in get_dns_zones(env):
+		if zone == domain:
+			break
+	else:
+		raise ValueError("%s is not a domain name that corresponds to a zone." % zone)
+
+	nsd_zonefile = "/etc/nsd/zones/" + fn
+	with open(nsd_zonefile, "r") as f:
+		return f.read()
+
 ########################################################################
 
 def write_nsd_conf(zonefiles, additional_records, env):

management/status_checks.py

@@ -293,6 +293,8 @@ def run_network_checks(env, output):
 	zen = query_dns(rev_ip4+'.zen.spamhaus.org', 'A', nxdomain=None)
 	if zen is None:
 		output.print_ok("IP address is not blacklisted by zen.spamhaus.org.")
+	elif zen == "[timeout]":
+		output.print_warning("Connection to zen.spamhaus.org timed out. We could not determine whether your server's IP address is blacklisted. Please try again later.")
 	else:
 		output.print_error("""The IP address of this machine %s is listed in the Spamhaus Block List (code %s),
 			which may prevent recipients from receiving your email. See http://www.spamhaus.org/query/ip/%s."""
@@ -678,6 +680,8 @@ def check_mail_domain(domain, env, output):
 	dbl = query_dns(domain+'.dbl.spamhaus.org', "A", nxdomain=None)
 	if dbl is None:
 		output.print_ok("Domain is not blacklisted by dbl.spamhaus.org.")
+	elif dbl == "[timeout]":
+		output.print_warning("Connection to dbl.spamhaus.org timed out. We could not determine whether the domain {} is blacklisted. Please try again later.".format(domain))
 	else:
 		output.print_error("""This domain is listed in the Spamhaus Domain Block List (code %s),
 			which may prevent recipients from receiving your mail.

management/templates/custom-dns.html

@@ -89,7 +89,7 @@
   <div class="form-group">
     <div class="col-sm-offset-1 col-sm-11">
       <p class="small">
-        Multiple secondary servers can be separated with commas or spaces (i.e., <code>ns2.hostingcompany.com ns3.hostingcompany.com</code>). 
+        Multiple secondary servers can be separated with commas or spaces (i.e., <code>ns2.hostingcompany.com ns3.hostingcompany.com</code>).
         To enable zone transfers to additional servers without listing them as secondary nameservers, add an IP address or subnet using <code>xfr:10.20.30.40</code> or <code>xfr:10.0.0.0/8</code>.
       </p>
       <p id="secondarydns-clear-instructions" style="display: none" class="small">

management/templates/external-dns.html

@@ -42,6 +42,19 @@
 	You may need to adopt this technique when adding DomainKeys. Use a tool like <code>named-checkzone</code> to validate your zone file.
 </p>
 
+<h3>Download zonefile</h3>
+<p>You can download your zonefiles here or use the table of records below.</p>
+<form class="form-inline" role="form" onsubmit="do_download_zonefile(); return false;">
+    <div class="form-group">
+        <div class="form-group">
+            <label for="downloadZonefile" class="control-label sr-only">Zone</label>
+            <select id="downloadZonefile" class="form-control" style="width: auto"> </select>
+        </div>
+        <button type="submit" class="btn btn-primary">Download</button>
+    </div>
+</form>
+
+<h3>Records</h3>
 
 <table id="external_dns_settings" class="table">
 	<thead>
@@ -57,6 +70,18 @@
 
 <script>
 function show_external_dns() {
+  api(
+    "/dns/zones",
+    "GET",
+    { },
+    function(data) {
+        var zones = $('#downloadZonefile');
+        zones.text('');
+        for (var j = 0; j < data.length; j++) {
+            zones.append($('<option/>').text(data[j]));
+        }
+    });
+
   $('#external_dns_settings tbody').html("<tr><td colspan='2' class='text-muted'>Loading...</td></tr>")
   api(
     "/dns/dump",
@@ -84,4 +109,19 @@ function show_external_dns() {
       }
     })
 }
+
+function do_download_zonefile() {
+    var zone = $('#downloadZonefile').val();
+
+    api(
+        "/dns/zonefile/"+ zone,
+        "GET",
+        {},
+        function(data) {
+            show_modal_error("Download Zonefile", $("<pre/>").text(data));
+        },
+        function(err) {
+            show_modal_error("Download Zonefile (Error)", $("<pre/>").text(err));
+        });
+}
 </script>

management/templates/system-backup.html

@@ -111,6 +111,31 @@
       <input type="text" class="form-control" rows="1" id="backup-target-pass">
     </div>
   </div>
+  <!-- Backblaze -->
+  <div class="form-group backup-target-b2">
+      <div class="col-sm-10 col-sm-offset-2">
+        <p>Backups are stored in a <a href="https://www.backblaze.com/" target="_blank" rel="noreferrer">Backblaze</a> B2 bucket. You must have a Backblaze account already.</p>
+        <p>You MUST manually copy the encryption password from <tt class="backup-encpassword-file"></tt> to a safe and secure location. You will need this file to decrypt backup files. It is NOT stored in your Backblaze B2 bucket.</p>
+      </div>
+  </div>
+  <div class="form-group backup-target-b2">
+      <label for="backup-target-b2-user" class="col-sm-2 control-label">B2 Application KeyID</label>
+      <div class="col-sm-8">
+        <input type="text" class="form-control" rows="1" id="backup-target-b2-user">
+      </div>
+  </div>
+  <div class="form-group backup-target-b2">
+      <label for="backup-target-b2-pass" class="col-sm-2 control-label">B2 Application Key</label>
+      <div class="col-sm-8">
+        <input type="text" class="form-control" rows="1" id="backup-target-b2-pass">
+      </div>
+  </div>
+  <div class="form-group backup-target-b2">
+      <label for="backup-target-b2-bucket" class="col-sm-2 control-label">B2 Bucket</label>
+      <div class="col-sm-8">
+        <input type="text" class="form-control" rows="1" id="backup-target-b2-bucket">
+      </div>
+  </div>
   <!-- Common -->
   <div class="form-group backup-target-local backup-target-rsync backup-target-s3">
     <label for="min-age" class="col-sm-2 control-label">Retention Days:</label>
@@ -144,7 +169,7 @@
 
 function toggle_form() {
   var target_type = $("#backup-target-type").val();
-  $(".backup-target-local, .backup-target-rsync, .backup-target-s3").hide();
+  $(".backup-target-local, .backup-target-rsync, .backup-target-s3, .backup-target-b2").hide();
   $(".backup-target-" + target_type).show();
 
   init_inputs(target_type);
@@ -215,7 +240,7 @@ function show_system_backup() {
 }
 
 function show_custom_backup() {
-    $(".backup-target-local, .backup-target-rsync, .backup-target-s3").hide();
+    $(".backup-target-local, .backup-target-rsync, .backup-target-s3, .backup-target-b2").hide();
     api(
       "/system/backup/config",
       "GET",
@@ -245,6 +270,15 @@ function show_custom_backup() {
           var host = hostpath.shift();
           $("#backup-target-s3-host").val(host);
           $("#backup-target-s3-path").val(hostpath.join('/'));
+        } else if (r.target.substring(0, 5) == "b2://") {
+          $("#backup-target-type").val("b2");
+          var targetPath = r.target.substring(5);
+          var b2_application_keyid = targetPath.split(':')[0];
+          var b2_applicationkey = targetPath.split(':')[1].split('@')[0];
+          var b2_bucket = targetPath.split('@')[1];
+          $("#backup-target-b2-user").val(b2_application_keyid);
+          $("#backup-target-b2-pass").val(b2_applicationkey);
+          $("#backup-target-b2-bucket").val(b2_bucket);
         }
         toggle_form()
       })
@@ -264,6 +298,11 @@ function set_custom_backup() {
     target = "rsync://" + $("#backup-target-rsync-user").val() + "@" + $("#backup-target-rsync-host").val()
                                                                 + "/" + $("#backup-target-rsync-path").val();
     target_user = '';
+  } else if (target_type == "b2") {
+    target = 'b2://' + $('#backup-target-b2-user').val() + ':' + $('#backup-target-b2-pass').val()
+        + '@' + $('#backup-target-b2-bucket').val()
+    target_user = '';
+    target_pass = '';
   }
 
 

management/templates/users.html

@@ -36,7 +36,7 @@
   <button type="submit" class="btn btn-primary">Add User</button>
 </form>
 <ul style="margin-top: 1em; padding-left: 1.5em; font-size: 90%;">
-  <li>Passwords must be at least eight characters consisting of English lettters and numbers only. For best results, <a href="#" onclick="return generate_random_password()">generate a random password</a>.</li>
+  <li>Passwords must be at least eight characters consisting of English letters and numbers only. For best results, <a href="#" onclick="return generate_random_password()">generate a random password</a>.</li>
   <li>Use <a href="#" onclick="return show_panel('aliases')">aliases</a> to create email addresses that forward to existing accounts.</li>
   <li>Administrators get access to this control panel.</li>
   <li>User accounts cannot contain any international (non-ASCII) characters, but <a href="#" onclick="return show_panel('aliases');">aliases</a> can.</li>

setup/bootstrap.sh

@@ -20,7 +20,7 @@ if [ -z "$TAG" ]; then
 	# want to display in status checks.
 	if [ "`lsb_release -d | sed 's/.*:\s*//' | sed 's/18\.04\.[0-9]/18.04/' `" == "Ubuntu 18.04 LTS" ]; then
 		# This machine is running Ubuntu 18.04.
-		TAG=v0.51-quota-0.22-beta
+		TAG=v0.52-quota-0.22-beta
 
 	elif [ "`lsb_release -d | sed 's/.*:\s*//' | sed 's/14\.04\.[0-9]/14.04/' `" == "Ubuntu 14.04 LTS" ]; then
 		# This machine is running Ubuntu 14.04.

setup/dkim.sh

@@ -64,6 +64,32 @@ tools/editconf.py /etc/opendmarc.conf -s \
 	"Syslog=true" \
 	"Socket=inet:8893@[127.0.0.1]"
 
+# SPFIgnoreResults causes the filter to ignore any SPF results in the header
+# of the message. This is useful if you want the filter to perfrom SPF checks
+# itself, or because you don't trust the arriving header. This added header is
+# used by spamassassin to evaluate the mail for spamminess.
+
+tools/editconf.py /etc/opendmarc.conf -s \
+        "SPFIgnoreResults=true"
+
+# SPFSelfValidate causes the filter to perform a fallback SPF check itself
+# when it can find no SPF results in the message header. If SPFIgnoreResults
+# is also set, it never looks for SPF results in headers and always performs
+# the SPF check itself when this is set. This added header is used by
+# spamassassin to evaluate the mail for spamminess.
+
+tools/editconf.py /etc/opendmarc.conf -s \
+        "SPFSelfValidate=true"
+
+# AlwaysAddARHeader Adds an "Authentication-Results:" header field even to
+# unsigned messages from domains with no "signs all" policy. The reported DKIM
+# result will be  "none" in such cases. Normally unsigned mail from non-strict
+# domains does not cause the results header field to be added. This added header
+# is used by spamassassin to evaluate the mail for spamminess.
+
+tools/editconf.py /etc/opendkim.conf -s \
+        "AlwaysAddARHeader=true"
+
 # Add OpenDKIM and OpenDMARC as milters to postfix, which is how OpenDKIM
 # intercepts outgoing mail to perform the signing (by adding a mail header)
 # and how they both intercept incoming mail to add Authentication-Results

setup/management.sh

@@ -18,11 +18,7 @@ while [ -d /usr/local/lib/python3.4/dist-packages/acme ]; do
 	pip3 uninstall -y acme;
 done
 
-# duplicity is used to make backups of user data. It uses boto
+# duplicity is used to make backups of user data.
-# (via Python 2) to do backups to AWS S3. boto from the Ubuntu
-# package manager is too out-of-date -- it doesn't support the newer
-# S3 api used in some regions, which breaks backups to those regions.
-# See #627, #653.
 #
 # virtualenv is used to isolate the Python 3 packages we
 # install via pip from the system-installed packages.
@@ -30,7 +26,10 @@ done
 # certbot installs EFF's certbot which we use to
 # provision free TLS certificates.
 apt_install duplicity python-pip virtualenv certbot
-hide_output pip2 install --upgrade boto
+
+# boto is used for amazon aws backups.
+# Both are installed outside the pipenv, so they can be used by duplicity
+hide_output pip3 install --upgrade boto
 
 # Create a virtualenv for the installation of Python 3 packages
 # used by the management daemon.
@@ -50,7 +49,7 @@ hide_output $venv/bin/pip install --upgrade pip
 hide_output $venv/bin/pip install --upgrade \
 	rtyaml "email_validator>=1.0.0" "exclusiveprocess" \
 	flask dnspython python-dateutil \
-    qrcode[pil] pyotp \
+  qrcode[pil] pyotp \
 	"idna>=2.0.0" "cryptography==2.2.2" boto psutil postfix-mta-sts-resolver
 
 # CONFIGURATION
@@ -90,6 +89,12 @@ rm -f /tmp/bootstrap.zip
 # running after a reboot.
 cat > $inst_dir/start <<EOF;
 #!/bin/bash
+# Set character encoding flags to ensure that any non-ASCII don't cause problems.
+export LANGUAGE=en_US.UTF-8
+export LC_ALL=en_US.UTF-8
+export LANG=en_US.UTF-8
+export LC_TYPE=en_US.UTF-8
+
 source $venv/bin/activate
 exec python `pwd`/management/daemon.py
 EOF

setup/nextcloud.sh

@@ -311,6 +311,9 @@ hide_output sudo -u www-data php /usr/local/lib/owncloud/console.php app:enable
 sudo -u www-data php /usr/local/lib/owncloud/occ upgrade
 if [ \( $? -ne 0 \) -a \( $? -ne 3 \) ]; then exit 1; fi
 
+# Disable default apps that we don't support
+sudo -u www-data php /usr/local/lib/owncloud/occ app:disable photos dashboard activity
+
 # Set PHP FPM values to support large file uploads
 # (semicolon is the comment character in this file, hashes produce deprecation warnings)
 tools/editconf.py /etc/php/7.2/fpm/php.ini -c ';' \

setup/spamassassin.sh

@@ -67,6 +67,56 @@ tools/editconf.py /etc/spamassassin/local.cf -s \
 	"add_header all Report"=_REPORT_ \
 	"add_header all Score"=_SCORE_
 
+
+# Authentication-Results SPF/Dmarc checks
+# ---------------------------------------
+# OpenDKIM and OpenDMARC are configured to validate and add "Authentication-Results: ..."
+# headers by checking the sender's SPF & DMARC policies. Instead of blocking mail that fails
+# these checks, we can use these headers to evaluate the mail as spam.
+#
+# Our custom rules are added to their own file so that an update to the deb package config
+# does not remove our changes.
+#
+# We need to escape period's in $PRIMARY_HOSTNAME since spamassassin config uses regex.
+
+escapedprimaryhostname="${PRIMARY_HOSTNAME//./\\.}"
+
+cat > /etc/spamassassin/miab_spf_dmarc.cf << EOF
+# Evaluate DMARC Authentication-Results
+header DMARC_PASS Authentication-Results =~ /$escapedprimaryhostname; dmarc=pass/
+describe DMARC_PASS DMARC check passed
+score DMARC_PASS -0.1
+
+header DMARC_NONE Authentication-Results =~ /$escapedprimaryhostname; dmarc=none/
+describe DMARC_NONE DMARC record not found
+score DMARC_NONE 0.1
+
+header DMARC_FAIL_NONE Authentication-Results =~ /$escapedprimaryhostname; dmarc=fail \(p=none/
+describe DMARC_FAIL_NONE DMARC check failed (p=none)
+score DMARC_FAIL_NONE 2.0
+
+header DMARC_FAIL_QUARANTINE Authentication-Results =~ /$escapedprimaryhostname; dmarc=fail \(p=quarantine/
+describe DMARC_FAIL_QUARANTINE DMARC check failed (p=quarantine)
+score DMARC_FAIL_QUARANTINE 5.0
+
+header DMARC_FAIL_REJECT Authentication-Results =~ /$escapedprimaryhostname; dmarc=fail \(p=reject/
+describe DMARC_FAIL_REJECT DMARC check failed (p=reject)
+score DMARC_FAIL_REJECT 10.0
+
+# Evaluate SPF Authentication-Results
+header SPF_PASS Authentication-Results =~ /$escapedprimaryhostname; spf=pass/
+describe SPF_PASS SPF check passed
+score SPF_PASS -0.1
+
+header SPF_NONE Authentication-Results =~ /$escapedprimaryhostname; spf=none/
+describe SPF_NONE SPF record not found
+score SPF_NONE 2.0
+
+header SPF_FAIL Authentication-Results =~ /$escapedprimaryhostname; spf=fail/
+describe SPF_FAIL SPF check failed
+score SPF_FAIL 5.0
+EOF
+
 # Bayesean learning
 # -----------------
 #

setup/system.sh

@@ -93,6 +93,9 @@ hide_output add-apt-repository -y universe
 # Install the certbot PPA.
 hide_output add-apt-repository -y ppa:certbot/certbot
 
+# Install the duplicity PPA.
+hide_output add-apt-repository -y ppa:duplicity-team/duplicity-release-git
+
 # ### Update Packages
 
 # Update system packages to make sure we have the latest upstream versions
@@ -317,6 +320,9 @@ fi #NODOC
 #   name server, on IPV6.
 # * The listen-on directive in named.conf.options restricts `bind9` to
 #   binding to the loopback interface instead of all interfaces.
+# * The max-recursion-queries directive increases the maximum number of iterative queries.
+#  	If more queries than specified are sent, bind9 returns SERVFAIL. After flushing the cache during system checks,
+#	we ran into the limit thus we are increasing it from 75 (default value) to 100.
 apt_install bind9
 tools/editconf.py /etc/default/bind9 \
 	"OPTIONS=\"-u bind -4\""
@@ -324,6 +330,10 @@ if ! grep -q "listen-on " /etc/bind/named.conf.options; then
 	# Add a listen-on directive if it doesn't exist inside the options block.
 	sed -i "s/^}/\n\tlisten-on { 127.0.0.1; };\n}/" /etc/bind/named.conf.options
 fi
+if ! grep -q "max-recursion-queries " /etc/bind/named.conf.options; then
+	# Add a max-recursion-queries directive if it doesn't exist inside the options block.
+	sed -i "s/^}/\n\tmax-recursion-queries 100;\n}/" /etc/bind/named.conf.options
+fi
 
 # First we'll disable systemd-resolved's management of resolv.conf and its stub server.
 # Breaking the symlink to /run/systemd/resolve/stub-resolv.conf means

setup/webmail.sh

@@ -28,8 +28,8 @@ apt_install \
 # Install Roundcube from source if it is not already present or if it is out of date.
 # Combine the Roundcube version number with the commit hash of plugins to track
 # whether we have the latest version of everything.
-VERSION=1.4.9
+VERSION=1.4.10
-HASH=df650f4d3eae9eaae2d5a5f06d68665691daf57d
+HASH=36b2351030e1ebddb8e39190d7b0ba82b1bbec1b
 PERSISTENT_LOGIN_VERSION=6b3fc450cae23ccb2f393d0ef67aa319e877e435
 HTML5_NOTIFIER_VERSION=4b370e3cd60dabd2f428a26f45b677ad1b7118d5
 CARDDAV_VERSION=3.0.3

setup/zpush.sh

@@ -22,8 +22,8 @@ apt_install \
 phpenmod -v php imap
 
 # Copy Z-Push into place.
-VERSION=2.5.2
+VERSION=2.6.1
-TARGETHASH=2dc3dbd791b96b0ba2638df0d3d1e03c7e1cbab2
+TARGETHASH=a4415f0dc0ed884acc8ad5c506944fc7e6d68eeb
 needs_update=0 #NODOC
 if [ ! -f /usr/local/lib/z-push/version ]; then
 	needs_update=1 #NODOC