feat(docs): restructure in proper Sphinx project

While increasing the comprehensibility of our docs, this should
also make the problem reported in readthedocs/sphinx_rtd_theme#834
go away.

docs/.gitignore

@@ -1 +1 @@
-*.html
+_build/*

docs/Makefile

@@ -0,0 +1,20 @@
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line, and also
+# from the environment for the first two.
+SPHINXOPTS    ?=
+SPHINXBUILD   ?= sphinx-build
+SOURCEDIR     = .
+BUILDDIR      = _build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

docs/authentication.rst → docs/auth/account.rst

@@ -1,5 +1,4 @@
-User Registration and Management
---------------------------------
+.. _manage-account:
 
 Manage Account
 ~~~~~~~~~~~~~~
@@ -8,6 +7,9 @@ Access to the domain management API is granted to registered and logged in
 users only. Users can register an account free of charge through the API as
 described below.
 
+
+.. _obtain-a-captcha:
+
 Obtain a Captcha
 ````````````````
 
@@ -30,6 +32,8 @@ a registration request. This means that if you send an incorrect solution,
 you will have to obtain a fresh captcha and try again.
 
 
+.. _register-account:
+
 Register Account
 ````````````````
 
@@ -118,6 +122,8 @@ be created (for example, because the domain name is unavailable), your account
 will be deleted, and you can start over with a fresh registration.
 
 
+.. _log-in:
+
 Log In
 ``````
 
@@ -262,7 +268,7 @@ Delete Account
 ``````````````
 
 Before you can delete your account, it is required to first delete all your
-domains from deSEC (see `Deleting a Domain`_).
+domains from deSEC (see :ref:`deleting-a-domain`).
 
 To delete your (empty) account, send a ``POST`` request with your email
 address and password to the ``/auth/account/delete/`` endpoint::
@@ -284,6 +290,8 @@ If your account still contains domains, the server will respond with ``409
 Conflict`` and not delete your account.
 
 
+.. _log-out:
+
 Log Out
 ```````
 
@@ -293,7 +301,7 @@ the log out endpoint::
     curl -X POST https://desec.io/api/v1/auth/logout/ \
         --header "Authorization: Token i-T3b1h_OI-H9ab8tRS98stGtURe"
 
-To delete other tokens based on their ID, see `Delete Tokens`_.
+To delete other tokens based on their ID, see :ref:`delete-tokens`.
 
 
 Security Considerations
@@ -367,109 +375,3 @@ Email verification
 Password Security
     Password information is stored using `Django's default method, PBKDF2
     <https://docs.djangoproject.com/en/2.1/topics/auth/passwords/>`_.
-
-
-Manage Tokens
-~~~~~~~~~~~~~
-
-To make authentication more flexible, the API can provide you with multiple
-authentication tokens. To that end, we provide a set of token management
-endpoints that are separate from the above-mentioned log in and log out
-endpoints. The most notable difference is that the log in endpoint needs
-authentication with email address and password, whereas the token management
-endpoint is authenticated using already issued tokens.
-
-
-Retrieving All Current Tokens
-`````````````````````````````
-
-To retrieve a list of currently valid tokens, issue a ``GET`` request::
-
-    curl -X GET https://desec.io/api/v1/auth/tokens/ \
-        --header "Authorization: Token mu4W4MHuSc0HyrGD1h/dnKuZBond"
-
-The server will respond with a list of token objects, each containing a
-timestamp when the token was created (note the ``Z`` indicating the UTC
-timezone) and a UUID to identify that token. Furthermore, each token can
-carry a name that is of no operational relevance to the API (it is meant
-for user reference only). Certain API operations (such as login) will
-automatically populate the ``name`` field with values such as "login" or
-"dyndns".
-
-::
-
-    [
-        {
-            "created": "2018-09-06T07:05:54.080564Z",
-            "id": "3159e485-5499-46c0-ae2b-aeb84d627a8e",
-            "name": "login"
-        },
-        {
-            "created": "2018-09-06T08:53:26.428396Z",
-            "id": "76d6e39d-65bc-4ab2-a1b7-6e94eee0a534",
-            "name": ""
-        }
-    ]
-
-You can also retrieve an individual token by appending ``:id/`` to the URL,
-for example in order to look up a token's name or creation timestamp.
-
-
-Create Additional Tokens
-````````````````````````
-
-To create another token using the token management interface, issue a
-``POST`` request to the same endpoint::
-
-    curl -X POST https://desec.io/api/v1/auth/tokens/ \
-        --header "Authorization: Token mu4W4MHuSc0HyrGD1h/dnKuZBond" \
-        --header "Content-Type: application/json" --data @- <<< \
-        '{"name": "my new token"}'
-
-Note that the name is optional and will be empty if not specified. The server
-will reply with ``201 Created`` and the created token in the response body::
-
-    {
-        "created": "2018-09-06T09:08:43.762697Z",
-        "id": "3a6b94b5-d20e-40bd-a7cc-521f5c79fab3",
-        "token": "4pnk7u+NHvrEkFzrhFDRTjGFyX+S",
-        "name": "my new token"
-    }
-
-
-Delete Tokens
-`````````````
-
-To delete an existing token by its ID via the token management endpoints, issue a
-``DELETE`` request on the token's endpoint, replacing ``:id`` with the
-token ``id`` value::
-
-    curl -X DELETE https://desec.io/api/v1/auth/tokens/:id/ \
-        --header "Authorization: Token mu4W4MHuSc0HyrGD1h/dnKuZBond"
-
-The server will reply with ``204 No Content``, even if the token was not found.
-
-If you do not have the token UUID, but you do have the token value itself, you
-can use the `Log Out`_ endpoint to delete it.
-
-Note that, for now, all tokens have equal power -- every token can authorize
-any action. We are planning to implement scoped tokens in the future.
-
-
-Security Considerations
-```````````````````````
-
-This section is for information only. Token length and encoding may change in
-the future.
-
-Any token is generated from 168 bits of randomness at the server and stored in
-hashed format (PBKDF2-HMAC-SHA256). Guessing the token correctly or reversing
-the hash is hence practically impossible.
-
-The token value is represented by 28 characters using a URL-safe variant of
-base64 encoding. It comprises only the characters ``A-Z``, ``a-z``, ``0-9``, ``-``,
-and ``_``. (Base64 padding is not needed as the string length is a multiple of 4.)
-
-Old versions of the API encoded 20-byte tokens in 40 characters with hexadecimal
-representation. Such tokens will not be issued anymore, but remain valid until
-invalidated by the user.

docs/auth/tokens.rst

@@ -0,0 +1,108 @@
+.. _manage-tokens:
+
+Manage Tokens
+~~~~~~~~~~~~~
+
+To make authentication more flexible, the API can provide you with multiple
+authentication tokens. To that end, we provide a set of token management
+endpoints that are separate from the above-mentioned log in and log out
+endpoints. The most notable difference is that the log in endpoint needs
+authentication with email address and password, whereas the token management
+endpoint is authenticated using already issued tokens.
+
+
+Retrieving All Current Tokens
+`````````````````````````````
+
+To retrieve a list of currently valid tokens, issue a ``GET`` request::
+
+    curl -X GET https://desec.io/api/v1/auth/tokens/ \
+        --header "Authorization: Token mu4W4MHuSc0HyrGD1h/dnKuZBond"
+
+The server will respond with a list of token objects, each containing a
+timestamp when the token was created (note the ``Z`` indicating the UTC
+timezone) and a UUID to identify that token. Furthermore, each token can
+carry a name that is of no operational relevance to the API (it is meant
+for user reference only). Certain API operations (such as login) will
+automatically populate the ``name`` field with values such as "login" or
+"dyndns".
+
+::
+
+    [
+        {
+            "created": "2018-09-06T07:05:54.080564Z",
+            "id": "3159e485-5499-46c0-ae2b-aeb84d627a8e",
+            "name": "login"
+        },
+        {
+            "created": "2018-09-06T08:53:26.428396Z",
+            "id": "76d6e39d-65bc-4ab2-a1b7-6e94eee0a534",
+            "name": ""
+        }
+    ]
+
+You can also retrieve an individual token by appending ``:id/`` to the URL,
+for example in order to look up a token's name or creation timestamp.
+
+
+Create Additional Tokens
+````````````````````````
+
+To create another token using the token management interface, issue a
+``POST`` request to the same endpoint::
+
+    curl -X POST https://desec.io/api/v1/auth/tokens/ \
+        --header "Authorization: Token mu4W4MHuSc0HyrGD1h/dnKuZBond" \
+        --header "Content-Type: application/json" --data @- <<< \
+        '{"name": "my new token"}'
+
+Note that the name is optional and will be empty if not specified. The server
+will reply with ``201 Created`` and the created token in the response body::
+
+    {
+        "created": "2018-09-06T09:08:43.762697Z",
+        "id": "3a6b94b5-d20e-40bd-a7cc-521f5c79fab3",
+        "token": "4pnk7u+NHvrEkFzrhFDRTjGFyX+S",
+        "name": "my new token"
+    }
+
+
+.. _delete-tokens:
+
+Delete Tokens
+`````````````
+
+To delete an existing token by its ID via the token management endpoints, issue a
+``DELETE`` request on the token's endpoint, replacing ``:id`` with the
+token ``id`` value::
+
+    curl -X DELETE https://desec.io/api/v1/auth/tokens/:id/ \
+        --header "Authorization: Token mu4W4MHuSc0HyrGD1h/dnKuZBond"
+
+The server will reply with ``204 No Content``, even if the token was not found.
+
+If you do not have the token UUID, but you do have the token value itself, you
+can use the :ref:`log-out` endpoint to delete it.
+
+Note that, for now, all tokens have equal power -- every token can authorize
+any action. We are planning to implement scoped tokens in the future.
+
+
+Security Considerations
+```````````````````````
+
+This section is for information only. Token length and encoding may change in
+the future.
+
+Any token is generated from 168 bits of randomness at the server and stored in
+hashed format (PBKDF2-HMAC-SHA256). Guessing the token correctly or reversing
+the hash is hence practically impossible.
+
+The token value is represented by 28 characters using a URL-safe variant of
+base64 encoding. It comprises only the characters ``A-Z``, ``a-z``, ``0-9``, ``-``,
+and ``_``. (Base64 padding is not needed as the string length is a multiple of 4.)
+
+Old versions of the API encoded 20-byte tokens in 40 characters with hexadecimal
+representation. Such tokens will not be issued anymore, but remain valid until
+invalidated by the user.

docs/compile.sh

@@ -1,7 +0,0 @@
-# To generate the documentation, run
-rst2html5 -d --stylesheet-path=minimal.css,plain.css,theme.css index.rst > index.html
-
-# Note that there are several different versions of rst2html5. (Notably, pip ships a version that behaves slightly
-# differently. We use the Ubuntu Bionic version provided by the `python3-docutils` package:
-#
-#    apt install python3-docutils

docs/conf.py

@@ -0,0 +1,83 @@
+try:
+    import sphinx_rtd_theme
+except ImportError:
+    sphinx_rtd_theme = None
+
+# Configuration file for the Sphinx documentation builder.
+#
+# This file only contains a selection of the most common options. For a full
+# list see the documentation:
+# https://www.sphinx-doc.org/en/master/usage/configuration.html
+
+# -- Path setup --------------------------------------------------------------
+
+# If extensions (or modules to document with autodoc) are in another directory,
+# add these directories to sys.path here. If the directory is relative to the
+# documentation root, use os.path.abspath to make it absolute, like shown here.
+#
+# import os
+# import sys
+# sys.path.insert(0, os.path.abspath('.'))
+
+
+# -- Project information -----------------------------------------------------
+
+project = 'deSEC DNS API'
+copyright = '2019, deSEC e.V., Individual Contributors'
+author = 'deSEC e.V., Individual Contributors'
+
+
+# -- General configuration ---------------------------------------------------
+
+# Add any Sphinx extension module names here, as strings. They can be
+# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
+# ones.
+extensions = [
+]
+
+# Add any paths that contain templates here, relative to this directory.
+templates_path = ['_templates']
+
+# List of patterns, relative to source directory, that match files and
+# directories to ignore when looking for source files.
+# This pattern also affects html_static_path and html_extra_path.
+exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']
+
+
+# -- Options for HTML output -------------------------------------------------
+
+# The theme to use for HTML and HTML Help pages.  See the documentation for
+# a list of builtin themes.
+#
+if sphinx_rtd_theme:
+    html_theme = "sphinx_rtd_theme"
+    html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]
+else:
+    html_theme = "default"
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+html_static_path = ['_static']
+
+
+master_doc = 'index'
+
+if sphinx_rtd_theme:
+    html_theme = "sphinx_rtd_theme"
+    html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]
+else:
+    html_theme = "default"
+
+html_static_path = ['_static']
+
+# The name of the Pygments (syntax highlighting) style to use.
+pygments_style = 'sphinx'
+
+# Grouping the document tree into LaTeX files. List of tuples
+# (source start file, target name, title,
+#  author, documentclass [howto, manual, or own class]).
+latex_documents = [
+  (master_doc, 'deSEC.tex', u'deSEC DNS API Documentation',
+   u'deSEC e.V., Individual Contributors', 'manual'),
+]

docs/domains.rst → docs/dns/domains.rst

@@ -1,3 +1,5 @@
+.. _domain-management:
+
 Domain Management
 -----------------
 
@@ -63,12 +65,10 @@ Field details:
       We look at each active ``cryptokey_resource`` (``active`` is true) and
       then use the ``dnskey``, ``ds``, ``flags``, and ``keytype`` fields.
 
-.. _`minimum TTL`:
-
 ``minimum_ttl``
     :Access mode: read-only
 
-    Smallest TTL that can be used in an `RRset <RRset object_>`__. The value
+    Smallest TTL that can be used in an :ref:`RRset <RRset object>`. The value
     is set automatically by the server.
 
     If you would like to use lower TTL values, you can apply for an exception
@@ -146,7 +146,7 @@ if you do not own any domains; in this case, the response body will be an empty
 JSON array.
 
 Up to 500 items are returned at a time.  If you have a larger number of
-domains configured, the use of `pagination`_ is required.
+domains configured, the use of :ref:`pagination` is required.
 
 
 Retrieving a Specific Domain
@@ -165,6 +165,8 @@ returns the domain object in the reponse body.  Otherwise, the return status
 code is ``404 Not Found``.
 
 
+.. _deleting-a-domain:
+
 Deleting a Domain
 ~~~~~~~~~~~~~~~~~
 

docs/rrsets.rst → docs/dns/rrsets.rst

@@ -1,3 +1,5 @@
+.. _`manage-rrsets`:
+
 Retrieving and Manipulating DNS Information
 -------------------------------------------
 
@@ -21,7 +23,7 @@ The relevant endpoints all reside under ``/api/v1/domains/:name/rrsets/``,
 where ``:name`` is the name of a domain you own.  When operating on domains
 that don't exist or you don't own, the API responds with a ``404 Not Found``
 status code.  For a quick overview of the available endpoints, methods, and
-operations, see `Endpoint Reference`_.
+operations, see :ref:`endpoint-reference`.
 
 
 .. _`RRset object`:
@@ -100,7 +102,8 @@ Field details:
 
     TTL (time-to-live) value, which dictates for how long resolvers may cache
     this RRset, measured in seconds.  The smallest acceptable value is given by
-    the domain's `minimum TTL`_ setting.  The maximum value is 604800 (one week).
+    the domain's :ref:`minimum TTL <domain object>` setting.  The maximum value
+    is 604800 (one week).
 
 ``type``
     :Access mode: read, write-once (upon RRset creation)
@@ -212,6 +215,8 @@ The response status code in case of success is ``200 OK``.  This is true also
 if there are no RRsets in the zone; in this case, the response body will be an
 empty JSON array.
 
+.. _pagination:
+
 Pagination
 ``````````
 Up to 500 items are returned at a time.  If more than 500 items would match the

docs/dyndns.rst

@@ -1,251 +0,0 @@
-dynDNS Howto
-------------
-
-The following subsections contain information on the most common topics of
-interest in the context of our DNSSEC-secured dynDNS service at dedyn.io.
-
-
-Configuring your dynDNS Client
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Here's how to configure your client to send your IP address to our servers so
-that we can publish it in the DNS. Depending on your use case, one of the
-following options might be easier than the others.
-
-To update your dynDNS IP address, there are three options:
-
-Option 1: Use Your Router
-`````````````````````````
-For most folks, using the integrated dynDNS client of their router will be
-easiest. The configuration procedures vary for all routers which is why we
-can't provide a tutorial for all of them. However, most of the time it boils
-down to enter the following details in your router configuration:
-
-- Update Server ``update.dedyn.io``, or Update URL ``https://update.dedyn.io/``
-- Username (your dedyn.io hostname, e.g. yourname.dedyn.io)
-- Hostname (same as your username)
-- Password (as provided when you registered your domain with us)
-
-**Advanced API users only:** The dynDNS password technically is an API token.
-If you also use our REST API, make sure to use a token for this purpose. Do not
-enter your account password when setting up your domain!
-
-IPv6 Support
-************
-There is a chance that your router already properly supports pushing its IPv6
-address to us. If it does not, you can try to let our servers determine your
-IPv6 address by using IPv6 to connect. To see if this method works for you,
-modify the "Update Server" or "Update URL" setting in your router's
-configuration to ``update6.dedyn.io`` and ``https://update6.dedyn.io/``,
-respectively.
-
-Note that when using this update server, your IPv4 address will be deleted from
-the DNS, and your domain will operate in IPv6-only mode. (For an explanation
-why that is the case, see `Determine IP addresses`_.) It is **not** possible to
-set up IPv4 and IPv6 by using both update servers in an alternating fashion.
-
-To update both your IPv4 and IPv6 address at the same time, most routers need
-to be configured with an update URL that provides both IP addresses. For
-Fritz!Box devices, for example, the URL reads:
-``https://update.dedyn.io/?myipv4=<ipaddr>&myipv6=<ip6addr>`` (Note that the
-placeholders in this URL must remain unchanged; your router will substitute
-them automatically. To find out the placeholder names for your router, please
-refer to the manual of your device.)
-
-Option 2: Use ddclient
-``````````````````````
-
-Automatic configuration (Debian-/Ubuntu-based systems)
-******************************************************
-If you're on Debian, Ubuntu or any other Linux distribution that provides you
-with the ddclient package, you can use it to update your IP address with our
-servers. Note that depending on the ddclient version you are using, IPv6
-support may be limited.
-
-To install ddclient, run ``sudo apt-get install ddclient``. If a configuration
-dialog does not appear automatically, use ``sudo dpkg-reconfigure ddclient`` to
-start the configuration process.
-
-In the configuration process, select "other" dynamic DNS service provider, and
-enter ``update.dedyn.io`` as the dynamic DNS server. Next, tell ddclient to use
-the "dyndns2" protocol to perform updates. Afterwards, enter the username and
-password that you received during registration. Last, tell ddclient how to
-detect your IP address, your domain name and the update interval.
-
-**Note:** As of the time of this writing, ddclient does not use an encrypted
-HTTPS connection by default. To enable it, open ``/etc/ddclient.conf`` and add
-``ssl=yes`` above the ``server=`` statement. We **strongly recommend** doing
-so; otherwise, your credentials will be exposed during transmission.
-
-Manual configuration (other systems)
-************************************
-After installing ddclient, you can start with a ``ddclient.conf`` configuration
-file similar to this one, with the three placeholders replaced by your domain
-name and password::
-
-  protocol=dyndns2
-  # "use=cmd" and the curl command is one way of doing this; other ways exist
-  use=cmd, cmd='curl https://checkipv4.dedyn.io/'
-  ssl=yes
-  server=update.dedyn.io
-  login=[domain]
-  password='[password]'
-  [domain]
-
-For more information, check out `these
-<https://sourceforge.net/p/ddclient/wiki/routers/>`_ two `sections
-<https://sourceforge.net/p/ddclient/wiki/usage/>`_ of the ddclient
-documentation.
-
-**Hint:** We have been told that in newer versions of ddclient, IPv6 can be
-enabled by replacing ``use`` with ``usev6``, ``checkipv4.dedyn.io`` with
-``checkipv6.dedyn.io``, and ``update.dedyn.io`` with ``update6.dedyn.io``.
-Unfortunately, there seems to be no documentation of the ``usev6`` setting, so
-we don't know if it is reliable. If you know more about this, please open an
-issue or pull request at `<https://github.com/desec-io/desec-stack/>`_.
-
-To test your setup, run ``sudo ddclient -force`` and see if everything works as
-expected.
-
-
-TLS Certificate with Let's Encrypt
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-dynDNS by deSEC supports the DNS challenge protocol to make it easy for you to
-obtain certificates for your domain name easily from anywhere. All you need is
-`certbot <https://certbot.eff.org/>`_, your credentials and our certbot hook
-script. As always, we appreciate your feedback. Shoot us an email!
-
-To obtain a Let's Encrypt Certificate for your dedyn.io domain, follow these
-steps.
-
-#. **Install Certbot.** There are many ways to install certbot, depending on
-   your distribution and preference. Please follow the official instructions at
-   `<https://certbot.eff.org/>`_.
-
-#. **Install hook script.** To authenticate your dedyn.io domain against Let's
-   Encrypt using the DNS challenge mechanism, you will need to update your
-   domain according to instructions provided by Let's Encrypt. Our hook script
-   automatizes this process for you. To use it, download the following two
-   files and place them into a directory of your choice. Make sure to change
-   the owner/permissions of the file (``chown``/``chmod``), so that it is only
-   readable by your certbot user (usually ``root``). ::
-
-     wget https://raw.githubusercontent.com/desec-io/certbot-hook/master/hook.sh
-     wget https://raw.githubusercontent.com/desec-io/certbot-hook/master/.dedynauth
-
-#. **Configuration.** You need to provide your dedyn.io credentials to the hook
-   script, so that it can write the Let's Encrypt challenge to the DNS on your
-   behalf. To do so, edit the ``.dedynauth`` file to look something like::
-
-    DEDYN_TOKEN=your token / dynDNS password
-    DEDYN_NAME=yourdomain.dedyn.io
-
-#. **Run certbot.** To obtain your certificate, run certbot in manual mode as
-   follows. (For a detailed explanation, please refer to the certbot manual.)
-   Please notice that you need to insert your domain name one more time. (Also,
-   for users not familiar with shell commands, please note that you need to
-   remove the ``\`` if you reformat the command to fit on one line.) ::
-
-     certbot --manual --preferred-challenges dns --manual-auth-hook ./hook.sh \
-             -d "YOURDOMAINNAME.dedyn.io" certonly
-
-   Depending on how you installed certbot, you may need to replace ``certbot``
-   with ``./certbot-auto`` (assuming that the ``certbot-auto`` executable is
-   located in the current directory). Please also note that the hook script may
-   wait up to two minutes to be sure that the challenge was correctly
-   published.
-
-   **Note:** To include subdomains in your certificate, you can specify the
-   ``-d`` argument several times, e.g.
-   ``-d "YOURDOMAINNAME.dedyn.io" -d "www.YOURDOMAINNAME.dedyn.io"``.
-
-   If you would like to help improve this hook script, please check out our
-   open issues at `<https://github.com/desec-utils/certbot-hook/issues>`_. We'd
-   highly appreciate your help!
-
-
-IP Update API
-~~~~~~~~~~~~~
-
-In case you want to dig deeper, here are the details on how our IP update API
-works.  We provide this API to be compatible with
-most dynDNS clients. However, we also provide a RESTful API that is
-more powerful and always preferred over the legacy interface described here.
-
-Update Request
-``````````````
-An IP updates is performed by sending a GET request to ``update.dedyn.io`` via
-HTTP or HTTPS. The path component can be chosen freely as long as it does not
-end in ``.ico`` or ``.png``.
-
-You can connect via IPv4 or IPv6. To enforce IPv6, use ``update6.dedyn.io``.
-
-Please be aware that while we still accept unencrypted requests, we **urge**
-you to use HTTPS. For that reason, we also send an HSTS header on HTTPS
-connections.
-
-Authentication
-**************
-You can authenticate your client in several ways:
-
-- Preferred method: HTTP Basic Authentication. Encode your username and
-  password as provided upon registration in the ``Authorization: Basic ...``
-  header. This is the method virtually all dynDNS clients use out of the box.
-
-- REST API method: HTTP Token Authentication. Send an ``Authorization: Token
-  ...`` header along with your request, where ``...`` is an API token issued
-  for this purpose. This method is used by our REST API as well.
-
-- Set the ``username`` and ``password`` query string parameters (``GET
-  ?username=...&password=...``). We **strongly discourage** using this
-  method, but provide it as an emergency solution for situations where folks
-  need to deal with old and/or crappy clients.
-
-If we cannot authenticate you, the API will return a ``401 Unauthorized``
-status code.
-
-Determine Hostname
-******************
-To update your IP address in the DNS, our servers need to determine the
-hostname you want to update (it's possible to set up several domains). To
-determine the hostname, we try the following steps until there is a match:
-
-- ``hostname`` query string parameter, unless it is set to ``YES`` (this
-  sometimes happens with dynDNS update clients).
-
-- ``host_id`` query string parameter.
-
-- The username as provided in the HTTP Basic Authorization header.
-
-- The username as provided in the ``username`` query string parameter.
-
-- After successful authentication (no matter how), the only hostname that is
-  associated with your user account (if not ambiguous).
-
-If we cannot determine a hostname to update, the API will return a ``404 Not
-Found`` status code.
-
-Determine IP addresses
-**********************
-The last ingredient we need for a successful update of your DNS records is your
-IPv4 and/or IPv6 addresses, for storage in the ``A`` and ``AAAA`` records,
-respectively.
-
-For IPv4, we will use the first IPv4 address it can find in the query string
-parameters ``myip``, ``myipv4``, ``ip`` (in this order). If none of them is
-set, it will use the IP that connected to the API, if a IPv4 connection was
-made. If no address is found or if an empty value was provided instead of an IP
-address, the ``A`` record will be deleted from the DNS.
-
-For IPv6, the procedure is similar. We check ``myipv6``, ``ipv6``, ``myip``,
-``ip`` query string parameters (in this order) and the IP that was used to
-connect to the API for IPv6 addresses and use the first one found. If no
-address is found or an empty value provided instead, the ``AAAA`` record will
-be deleted.
-
-
-Update Response
-```````````````
-If successful, the server will return a response with status ``200 OK`` and
-``good`` as the body (as per the dyndns2 protocol specification). For error
-status codes, see above.

docs/dyndns/configure.rst

@@ -0,0 +1,101 @@
+Configuring your dynDNS Client
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Here's how to configure your client to send your IP address to our servers so
+that we can publish it in the DNS. Depending on your use case, one of the
+following options might be easier than the others.
+
+To update your dynDNS IP address, there are several options:
+
+Option 1: Use Your Router
+`````````````````````````
+For most folks, using the integrated dynDNS client of their router will be
+easiest. The configuration procedures vary for all routers which is why we
+can't provide a tutorial for all of them. However, most of the time it boils
+down to enter the following details in your router configuration:
+
+- Update Server ``update.dedyn.io``, or Update URL ``https://update.dedyn.io/``
+- Username (your dedyn.io hostname, e.g. yourname.dedyn.io)
+- Hostname (same as your username)
+- Password (as provided when you registered your domain with us)
+
+**Advanced API users only:** The dynDNS password technically is an API token.
+If you also use our REST API, make sure to use a token for this purpose. Do not
+enter your account password when setting up your domain!
+
+IPv6 Support
+************
+There is a chance that your router already properly supports pushing its IPv6
+address to us. If it does not, you can try to let our servers determine your
+IPv6 address by using IPv6 to connect. To see if this method works for you,
+modify the "Update Server" or "Update URL" setting in your router's
+configuration to ``update6.dedyn.io`` and ``https://update6.dedyn.io/``,
+respectively.
+
+Note that when using this update server, your IPv4 address will be deleted from
+the DNS, and your domain will operate in IPv6-only mode. (For an explanation
+why that is the case, see :ref:`determine-ip-addresses`.) It is **not** possible
+to set up IPv4 and IPv6 by using both update servers in an alternating fashion.
+
+To update both your IPv4 and IPv6 address at the same time, most routers need
+to be configured with an update URL that provides both IP addresses. For
+Fritz!Box devices, for example, the URL reads:
+``https://update.dedyn.io/?myipv4=<ipaddr>&myipv6=<ip6addr>`` (Note that the
+placeholders in this URL must remain unchanged; your router will substitute
+them automatically. To find out the placeholder names for your router, please
+refer to the manual of your device.)
+
+Option 2: Use ddclient
+``````````````````````
+
+Automatic configuration (Debian-/Ubuntu-based systems)
+******************************************************
+If you're on Debian, Ubuntu or any other Linux distribution that provides you
+with the ddclient package, you can use it to update your IP address with our
+servers. Note that depending on the ddclient version you are using, IPv6
+support may be limited.
+
+To install ddclient, run ``sudo apt-get install ddclient``. If a configuration
+dialog does not appear automatically, use ``sudo dpkg-reconfigure ddclient`` to
+start the configuration process.
+
+In the configuration process, select "other" dynamic DNS service provider, and
+enter ``update.dedyn.io`` as the dynamic DNS server. Next, tell ddclient to use
+the "dyndns2" protocol to perform updates. Afterwards, enter the username and
+password that you received during registration. Last, tell ddclient how to
+detect your IP address, your domain name and the update interval.
+
+**Note:** As of the time of this writing, ddclient does not use an encrypted
+HTTPS connection by default. To enable it, open ``/etc/ddclient.conf`` and add
+``ssl=yes`` above the ``server=`` statement. We **strongly recommend** doing
+so; otherwise, your credentials will be exposed during transmission.
+
+Manual configuration (other systems)
+************************************
+After installing ddclient, you can start with a ``ddclient.conf`` configuration
+file similar to this one, with the three placeholders replaced by your domain
+name and password::
+
+  protocol=dyndns2
+  # "use=cmd" and the curl command is one way of doing this; other ways exist
+  use=cmd, cmd='curl https://checkipv4.dedyn.io/'
+  ssl=yes
+  server=update.dedyn.io
+  login=[domain]
+  password='[password]'
+  [domain]
+
+For more information, check out `these
+<https://sourceforge.net/p/ddclient/wiki/routers/>`_ two `sections
+<https://sourceforge.net/p/ddclient/wiki/usage/>`_ of the ddclient
+documentation.
+
+**Hint:** We have been told that in newer versions of ddclient, IPv6 can be
+enabled by replacing ``use`` with ``usev6``, ``checkipv4.dedyn.io`` with
+``checkipv6.dedyn.io``, and ``update.dedyn.io`` with ``update6.dedyn.io``.
+Unfortunately, there seems to be no documentation of the ``usev6`` setting, so
+we don't know if it is reliable. If you know more about this, please open an
+issue or pull request at `<https://github.com/desec-io/desec-stack/>`_.
+
+To test your setup, run ``sudo ddclient -force`` and see if everything works as
+expected.

docs/dyndns/lets-encrypt.rst

@@ -0,0 +1,54 @@
+TLS Certificate with Let's Encrypt
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+dynDNS by deSEC supports the DNS challenge protocol to make it easy for you to
+obtain certificates for your domain name easily from anywhere. All you need is
+`certbot <https://certbot.eff.org/>`_, your credentials and our certbot hook
+script. As always, we appreciate your feedback. Shoot us an email!
+
+To obtain a Let's Encrypt Certificate for your dedyn.io domain, follow these
+steps.
+
+#. **Install Certbot.** There are many ways to install certbot, depending on
+   your distribution and preference. Please follow the official instructions at
+   `<https://certbot.eff.org/>`_.
+
+#. **Install hook script.** To authenticate your dedyn.io domain against Let's
+   Encrypt using the DNS challenge mechanism, you will need to update your
+   domain according to instructions provided by Let's Encrypt. Our hook script
+   automatizes this process for you. To use it, download the following two
+   files and place them into a directory of your choice. Make sure to change
+   the owner/permissions of the file (``chown``/``chmod``), so that it is only
+   readable by your certbot user (usually ``root``). ::
+
+     wget https://raw.githubusercontent.com/desec-io/certbot-hook/master/hook.sh
+     wget https://raw.githubusercontent.com/desec-io/certbot-hook/master/.dedynauth
+
+#. **Configuration.** You need to provide your dedyn.io credentials to the hook
+   script, so that it can write the Let's Encrypt challenge to the DNS on your
+   behalf. To do so, edit the ``.dedynauth`` file to look something like::
+
+    DEDYN_TOKEN=your token / dynDNS password
+    DEDYN_NAME=yourdomain.dedyn.io
+
+#. **Run certbot.** To obtain your certificate, run certbot in manual mode as
+   follows. (For a detailed explanation, please refer to the certbot manual.)
+   Please notice that you need to insert your domain name one more time. (Also,
+   for users not familiar with shell commands, please note that you need to
+   remove the ``\`` if you reformat the command to fit on one line.) ::
+
+     certbot --manual --preferred-challenges dns --manual-auth-hook ./hook.sh \
+             -d "YOURDOMAINNAME.dedyn.io" certonly
+
+   Depending on how you installed certbot, you may need to replace ``certbot``
+   with ``./certbot-auto`` (assuming that the ``certbot-auto`` executable is
+   located in the current directory). Please also note that the hook script may
+   wait up to two minutes to be sure that the challenge was correctly
+   published.
+
+   **Note:** To include subdomains in your certificate, you can specify the
+   ``-d`` argument several times, e.g.
+   ``-d "YOURDOMAINNAME.dedyn.io" -d "www.YOURDOMAINNAME.dedyn.io"``.
+
+   If you would like to help improve this hook script, please check out our
+   open issues at `<https://github.com/desec-utils/certbot-hook/issues>`_. We'd
+   highly appreciate your help!

docs/dyndns/update-api.rst

@@ -0,0 +1,87 @@
+IP Update API
+~~~~~~~~~~~~~
+
+In case you want to dig deeper, here are the details on how our IP update API
+works.  We provide this API to be compatible with
+most dynDNS clients. However, we also provide a RESTful API that is
+more powerful and always preferred over the legacy interface described here.
+
+Update Request
+``````````````
+An IP updates is performed by sending a GET request to ``update.dedyn.io`` via
+HTTP or HTTPS. The path component can be chosen freely as long as it does not
+end in ``.ico`` or ``.png``.
+
+You can connect via IPv4 or IPv6. To enforce IPv6, use ``update6.dedyn.io``.
+
+Please be aware that while we still accept unencrypted requests, we **urge**
+you to use HTTPS. For that reason, we also send an HSTS header on HTTPS
+connections.
+
+Authentication
+**************
+You can authenticate your client in several ways:
+
+- Preferred method: HTTP Basic Authentication. Encode your username and
+  password as provided upon registration in the ``Authorization: Basic ...``
+  header. This is the method virtually all dynDNS clients use out of the box.
+
+- REST API method: HTTP Token Authentication. Send an ``Authorization: Token
+  ...`` header along with your request, where ``...`` is an API token issued
+  for this purpose. This method is used by our REST API as well.
+
+- Set the ``username`` and ``password`` query string parameters (``GET
+  ?username=...&password=...``). We **strongly discourage** using this
+  method, but provide it as an emergency solution for situations where folks
+  need to deal with old and/or crappy clients.
+
+If we cannot authenticate you, the API will return a ``401 Unauthorized``
+status code.
+
+Determine Hostname
+******************
+To update your IP address in the DNS, our servers need to determine the
+hostname you want to update (it's possible to set up several domains). To
+determine the hostname, we try the following steps until there is a match:
+
+- ``hostname`` query string parameter, unless it is set to ``YES`` (this
+  sometimes happens with dynDNS update clients).
+
+- ``host_id`` query string parameter.
+
+- The username as provided in the HTTP Basic Authorization header.
+
+- The username as provided in the ``username`` query string parameter.
+
+- After successful authentication (no matter how), the only hostname that is
+  associated with your user account (if not ambiguous).
+
+If we cannot determine a hostname to update, the API will return a ``404 Not
+Found`` status code.
+
+.. _determine-ip-addresses:
+
+Determine IP addresses
+**********************
+The last ingredient we need for a successful update of your DNS records is your
+IPv4 and/or IPv6 addresses, for storage in the ``A`` and ``AAAA`` records,
+respectively.
+
+For IPv4, we will use the first IPv4 address it can find in the query string
+parameters ``myip``, ``myipv4``, ``ip`` (in this order). If none of them is
+set, it will use the IP that connected to the API, if a IPv4 connection was
+made. If no address is found or if an empty value was provided instead of an IP
+address, the ``A`` record will be deleted from the DNS.
+
+For IPv6, the procedure is similar. We check ``myipv6``, ``ipv6``, ``myip``,
+``ip`` query string parameters (in this order) and the IP that was used to
+connect to the API for IPv6 addresses and use the first one found. If no
+address is found or an empty value provided instead, the ``AAAA`` record will
+be deleted.
+
+
+Update Response
+```````````````
+If successful, the server will return a response with status ``200 OK`` and
+``good`` as the body (as per the dyndns2 protocol specification). For error
+status codes, see above.

docs/endpoint-reference.rst

@@ -1,8 +1,10 @@
+.. _endpoint-reference:
+
 Endpoint Reference
 ------------------
 
 The following table summarizes basic information about the deSEC API endpoints used
-for `User Registration and Management`_.
+for :ref:`managing users <manage-account>` and :ref:`tokens <manage-tokens>`.
 
 +------------------------------------------------+------------+---------------------------------------------+
 | Endpoint ``/api/v1``...                        | Methods    | Use case                                    |
@@ -41,7 +43,8 @@ for `User Registration and Management`_.
 +------------------------------------------------+------------+---------------------------------------------+
 
 The following table summarizes basic information about the deSEC API endpoints used
-for `Domain Management`_ and `Retrieving and Manipulating DNS Information`_.
+for :ref:`domain-management` and :ref:`Retrieving and Manipulating DNS
+Information <manage-rrsets>`.
 
 +------------------------------------------------+------------+---------------------------------------------+
 | Endpoint ``/api/v1/domains``...                | Methods    | Use case                                    |
@@ -65,13 +68,13 @@ for `Domain Management`_ and `Retrieving and Manipulating DNS Information`_.
 |                                                +------------+---------------------------------------------+
 |                                                | ``PUT``    | Create, modify or delete one or more RRsets |
 +------------------------------------------------+------------+---------------------------------------------+
+| ...\ ``/:name/rrsets/@/:type/``                |            | Access an RRset at the zone apex            |
++------------------------------------------------+------------+---------------------------------------------+
 | ...\ ``/:name/rrsets/:subname/:type/``         | ``GET``    | Retrieve a specific RRset                   |
-| ...\ ``/:name/rrsets/:subname.../:type/``      +------------+---------------------------------------------+
-|                                                | ``PATCH``  | Modify an RRset                             |
+|                                                +------------+---------------------------------------------+
+| ...\ ``/:name/rrsets/:subname.../:type/``      | ``PATCH``  | Modify an RRset                             |
 |                                                +------------+---------------------------------------------+
 |                                                | ``PUT``    | Replace an RRset                            |
 |                                                +------------+---------------------------------------------+
 |                                                | ``DELETE`` | Delete an RRset                             |
 +------------------------------------------------+------------+---------------------------------------------+
-| ...\ ``/:name/rrsets/@/:type/``                |            | Access an RRset at the zone apex            |
-+------------------------------------------------+------------+---------------------------------------------+

docs/index.rst

@@ -1,19 +1,61 @@
-.. include:: introduction.rst
-.. include:: quickstart.rst
-.. include:: dyndns.rst
-.. include:: authentication.rst
-.. include:: domains.rst
-.. include:: rrsets.rst
-.. include:: endpoint-reference.rst
-.. include:: lifecycle.rst
+Welcome to the deSEC DNS API
+============================
+
+The deSEC DNS API is a REST interface that allows easy management of DNS
+information.  The interface design aims for simplicity so that tasks such as
+creating domains and manipulating DNS records can be handled with ease and in
+an intuitive fashion.
+
+Server-side operations, such as creation of domains or DNS records, expect
+JSON-formatted user input in the body of the ``POST``, ``PATCH``, or ``PUT``
+request (see below).  The request is required to come with a ``Content-Type:
+application/json`` header field.
+
+API functionality is demonstrated using the command line tool ``curl``.  To
+pretty-print JSON output, process the data through ``jq``:  ``curl ... | jq .``.
+
+
+.. toctree::
+   :maxdepth: 2
+   :caption: User Management
+
+   quickstart
+   auth/account
+   auth/tokens
+
+
+.. toctree::
+   :maxdepth: 2
+   :caption: dynDNS
+
+   dyndns/configure
+   dyndns/lets-encrypt
+   dyndns/update-api
+
+
+.. toctree::
+   :maxdepth: 2
+   :caption: DNS Management
+
+   dns/domains
+   dns/rrsets
+
+
+.. toctree::
+   :maxdepth: 2
+   :caption: API Summary
+
+   endpoint-reference
+   lifecycle
+
 
 Getting Help
-------------
+============
 
 If you need help beyond this documentation, please do not hesitate and
 shoot us an email at support@desec.io.
 
 About this document
--------------------
+===================
 To add to our documentation or fix a mistake, please submit a Pull Request
 at https://github.com/desec-io/desec-stack.

docs/introduction.rst

@@ -1,15 +0,0 @@
-Introduction
-------------
-
-The deSEC DNS API is a REST interface that allows easy management of DNS
-information.  The interface design aims for simplicity so that tasks such as
-creating domains and manipulating DNS records can be handled with ease and in
-an intuitive fashion.
-
-Server-side operations, such as creation of domains or DNS records, expect
-JSON-formatted user input in the body of the ``POST``, ``PATCH``, or ``PUT``
-request (see below).  The request is required to come with a ``Content-Type:
-application/json`` header field.
-
-API functionality is demonstrated using the command line tool ``curl``.  To
-pretty-print JSON output, process the data through ``jq``:  ``curl ... | jq .``.

docs/lifecycle.rst

@@ -12,7 +12,7 @@ Check out the `current status of the API versions`_ to make sure you
 are using the latest stable API whenever using our service in
 production.
 
-.. _current status of the API versions: https://github.com/desec-io/desec-stack/#api-versions-and-road-map
+.. _current status of the API versions: https://github.com/desec-io/desec-stack/#api-versions-and-roadmap
 
 **Unstable API versions** are currently under development and may
 change without prior notice, but we promise to keep an eye on users

docs/make.bat

@@ -0,0 +1,35 @@
+@ECHO OFF
+
+pushd %~dp0
+
+REM Command file for Sphinx documentation
+
+if "%SPHINXBUILD%" == "" (
+	set SPHINXBUILD=sphinx-build
+)
+set SOURCEDIR=.
+set BUILDDIR=_build
+
+if "%1" == "" goto help
+
+%SPHINXBUILD% >NUL 2>NUL
+if errorlevel 9009 (
+	echo.
+	echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
+	echo.installed, then set the SPHINXBUILD environment variable to point
+	echo.to the full path of the 'sphinx-build' executable. Alternatively you
+	echo.may add the Sphinx directory to PATH.
+	echo.
+	echo.If you don't have Sphinx installed, grab it from
+	echo.http://sphinx-doc.org/
+	exit /b 1
+)
+
+%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+goto end
+
+:help
+%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+
+:end
+popd

docs/quickstart.rst

@@ -4,7 +4,7 @@ Quickstart
 To use our domain management API, you need to register an account with deSEC.
 Here's a quick intro how to get started:
 
-#. `Obtain a captcha`_ and solve it::
+#. :ref:`obtain-a-captcha` and solve it::
 
     curl -X POST https://desec.io/api/v1/captcha/
 
@@ -14,7 +14,7 @@ Here's a quick intro how to get started:
    ``data:image/png;base64,<challenge>``, after replacing ``<challenge>`` with
    the value of the ``challenge`` response field
 
-#. `Register account`_::
+#. :ref:`register-account`::
 
     curl -X POST https://desec.io/api/v1/auth/ \
         --header "Content-Type: application/json" --data @- <<EOF
@@ -34,7 +34,7 @@ Here's a quick intro how to get started:
    your address and activate your account, simply click the link.
 
 
-#. `Log in`_::
+#. :ref:`log-in`::
 
     curl -X POST https://desec.io/api/v1/auth/login/ \
         --header "Content-Type: application/json" --data @- <<< \
@@ -51,5 +51,5 @@ Here's a quick intro how to get started:
         --header "Content-Type: application/json" --data @- <<< \
         '{"name": "example.com"}'
 
-#. Yay! Keep browsing the `Domain Management`_ section of the docs to see how
+#. Yay! Keep browsing the :ref:`domain-management` section of the docs to see how
    to continue.

docs/theme.css

@@ -1,17 +0,0 @@
-body {
-    background: #eee;
-    font-family: sans-serif;
-}
-
-:not(.field-list) > dt {
-    font-weight: bold;
-}
-
-.literal, .literal-block {
-    background: #fff;
-    border: 1px solid #ddd;
-}
-
-td {
-    vertical-align: top;
-}

webapp/src/views/DynSetup.vue

@@ -79,7 +79,7 @@
                   <p>
                     Please only update your IP address when it has changed. If your client is
                     unable to determine when your address changes, please refer to our
-                    <a href="https://desec.readthedocs.io/en/latest/#configuring-your-dyndns-client">documentation</a>
+                    <a href="https://desec.readthedocs.io/en/latest/dyndns/configure.html">documentation</a>
                     for alternative IP update approaches.
                   </p>
                 </v-expansion-panel-content>
@@ -102,7 +102,7 @@
                   <p>
                     For alternative approaches to updating your IP address and for a
                     detailed explanation of the update protocol, please refer to our
-                    <a href="https://desec.readthedocs.io/en/latest/#dyndns-howto">documentation</a>.
+                    <a href="https://desec.readthedocs.io/en/latest/dyndns/update-api.html">documentation</a>.
                   </p>
                 </v-expansion-panel-content>
               </v-expansion-panel>