howto: add event manager

add groups section in the getting started guide.
Suggest to prefer configuration with env vars instead of modifying
the default configuration file

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>

docs/full-configuration.md

@@ -465,10 +465,11 @@ You can select `sha256-simd` setting the environment variable `SFTPGO_MINIO_SHA2
 
  `sha256-simd` is particularly useful if you have an Intel CPU with SHA extensions or an ARM CPU with Cryptography Extensions.
 
-The SFTPGo configuration file can change between different versions and merging your custom settings with the default config file may be time-consuming. For this reason we suggest to set your custom settings using environment variables. This eliminates the need to merge your changes with the default configuration file after each update, you have to just check that your configuration key still exists.
+The configuration file can change between different versions and merging your custom settings with the default configuration file, after updating SFTPGo, may be time-consuming. For this reason we suggest to set your custom settings using environment variables. This eliminates the need to merge your changes with the default configuration file after each update, you have to just check that your custom configuration keys still exists.
+
 Setting configuration options from environment variables is natural in Docker/Kubernetes.
-If you install SFTPGo on Linux using the official deb/rpm packages you can set your custom environment variables in the file `/etc/sftpgo/sftpgo.env` (create this file if it does not exist).
+If you install SFTPGo on Linux using the official deb/rpm packages you can set your custom environment variables in the file `/etc/sftpgo/sftpgo.env` (create this file if it does not exist, it is defined as `EnvironmentFile` in the SFTPGo systemd unit).
-SFTPGo also reads files inside the `env.d` directory relative to config dir and then export the valid variables into environment variables if they are not already set. With this method you can override any configuration options, set environment variables for SFTPGo plugins but you cannot set command flags because these files are read after that SFTPGo starts and the config dir must already be set.
+SFTPGo also reads files inside the `env.d` directory relative to config dir and then exports the valid variables into environment variables if they are not already set. With this method you can override any configuration options, set environment variables for SFTPGo plugins but you cannot set command flags because these files are read after that SFTPGo starts and the config dir must already be set.
 Of course you can also set environment variables with the method provided by the operating system of your choice.
 
 </details>

docs/groups.md

@@ -7,7 +7,7 @@ SFTPGo supports two types of groups:
 - primary groups
 - secondary groups
 
-A user can be a member of a primary group and many secondary groups. Depending on the group type, the settings are inherited differently.
+A user can be a member of a primary group and many secondary and membership groups. Depending on the group type, the settings are inherited differently.
 
 :warning: SFTPGo groups are completely unrelated to system groups. Therefore, it is not necessary to add Linux/Windows groups to use SFTPGo groups.
 
@@ -29,7 +29,7 @@ The following settings are inherited from the primary and secondary groups:
 - two factor auth protocols
 - web client/REST API permissions
 
-The settings from the primary group are always merged first.
+The settings from the primary group are always merged first. no setting is inherited from "membership" groups.
 
 The final settings are a combination of the user settings and the group ones.
 For example you can define the following groups:

docs/howto/README.md

@@ -5,6 +5,7 @@ Here we collect step-to-step tutorials. SFTPGo users are encouraged to contribut
 - [Getting Started](./getting-started.md)
 - [Securing SFTPGo with a free Let's Encrypt TLS Certificate](./lets-encrypt-certificate.md)
 - [Two-factor Authentication](./two-factor-authentication.md)
+- [Event Manager](./eventmanager.md)
 - [SFTPGo as OpenSSH's SFTP subsystem](./openssh-sftp-subsystem.md)
 - [SFTPGo with PostgreSQL data provider and S3 backend](./postgresql-s3.md)
 - [SFTPGo on Windows with Active Directory Integration + Caddy Static File Server](https://www.youtube.com/watch?v=M5UcJI8t4AI)

docs/howto/eventmanager.md

@@ -0,0 +1,96 @@
+# Event Manager
+
+The Event Manager allows an administrator to configure HTTP notifications, commands execution, email notifications and carry out certain server operations based on server events or schedules. More details [here](../eventmanager.md).
+
+Let's see some common use cases.
+
+- [Preliminary Note](#preliminary-note)
+- [Daily backups](#daily-backups)
+- [Automatically create a folder structure](#automatically-create-a-folder-structure)
+- [Upload notifications](#upload-notifications)
+
+## Preliminary Note
+
+We will use email actions in the following paragraphs, so let's assume you have a working SMTP configuration.
+You can adapt the following snippet to configure an SMTP server using environment variables.
+
+```shell
+SFTPGO_SMTP__HOST="your smtp server host"
+SFTPGO_SMTP__FROM="SFTPGo <sftpgo@example.com>"
+SFTPGO_SMTP__USER=sftpgo@example.com
+SFTPGO_SMTP__PASSWORD="your password"
+SFTPGO_SMTP__AUTH_TYPE=1 # change based on what your server supports
+SFTPGO_SMTP__ENCRYPTION=2 # change based on what your server supports
+```
+
+SFTPGo supports several placeholders for event actions. You can see all supported placeholders by clicking on the "info" icon at the top right of the add/update action page.
+
+## Daily backups
+
+You can schedule SFTPGo data backups (users, folders, groups, admins etc.) on a regular basis, such as daily.
+
+From the WebAdmin expand the `Event Manager` section, select `Event actions` and add a new action.
+Create an action named `backup` and set the type to `Backup`.
+
+![Backup action](./img/backup-action.png)
+
+Create another action named `backup notification`, set the type to `Email` and fill the recipient/s.
+As email subject set `Backup {{StatusString}}`. The `{{StatusString}}` placeholder will be expanded to `OK` or `KO`.
+As email body set `Backup done {{ErrorString}}`. The error string will be empty if no errors occur.
+
+![Backup notification action](./img/backup-notification-action.png)
+
+Now select `Event rules` and create a rule named `Daily backup`, select `Schedule` as trigger and schedule a backup at midnight UTC time.
+
+![Daily backup schedule](./img/daily-backup-schedule.png)
+
+As actions select `backup` and `backup notification`.
+
+![Daily backup actions](./img/daily-backup-actions.png)
+
+Done! SFTPGo will make a new backup every day and you will receive an email with the status of the backup. The backup will be saved on the server side in the configured backup directory. The backup files will have names like this `backup_<week day>_<hour>.json`.
+
+## Automatically create a folder structure
+
+Suppose you want to automatically create the folders `in` and `out` when you create new users.
+
+From the WebAdmin expand the `Event Manager` section, select `Event actions` and add a new action.
+Create an action named `create dirs`, with the settings you can see in the following screen.
+
+![Create dirs action](./img/create-dirs-action.png)
+
+Create another action named `create dirs failure notification`, set the type to `Email` and fill the recipient/s.
+As email subject set `Unable to create dirs for user {{ObjectName}}`.
+As email body set `Error: {{ErrorString}}`.
+
+![Create dirs notification](./img/create-dirs-failure-notification.png)
+
+Now select `Event rules` and create a rule named `Create dirs for users`, select `Provider event` as trigger, `add` as provider event and `user` as object filters.
+
+![Create dirs rule](./img/create-dirs-rule.png)
+
+As actions select `create dirs` and `create dirs failure notification`, check `Is failure action` for the notification action.
+This way you will only be notified by email if an error occurs.
+
+![Create dirs rule actions](./img/create-dirs-rule-actions.png)
+
+Done! Create a new user and check that the defined directories are automatically created.
+
+## Upload notifications
+
+Let's see how you can receive an email notification after each upload and, optionally, the uploaded file as well.
+
+From the WebAdmin expand the `Event Manager` section, select `Event actions` and add a new action.
+Create an action named `upload notification`, with the settings you can see in the following screen.
+
+![Upload notification action](./img/upload-notification.png)
+
+You can optionally add the uploaded file as an attachment but note that SFTPGo allows you to attach a maximum of 10MB. Then the action will fail for files bigger than 10MB.
+
+Now select `Event rules` and create a rule named `Upload rule`, select `Filesystem evens` as trigger and `upload` as filesystem event.
+You can also filters events based on protocol, user and group name, filepath shell-like patterns, file size. We omit these additional filters for simplicity.
+
+![Upload rule](./img/upload-rule.png)
+
+As actions, select `upload notification`.
+Done! Try uploading a new file and you will receive the configured email notification.

docs/howto/getting-started.md

@@ -14,6 +14,9 @@ In this tutorial we explore the main features and concepts using the built-in we
   - [Creating users with a local encrypted backend (Data At Rest Encryption)](#creating-users-with-a-local-encrypted-backend-data-at-rest-Encryption)
 - [Virtual permissions](#virtual-permissions)
 - [Virtual folders](#virtual-folders)
+- [Groups](#groups)
+  - [Usage example](#usage-example)
+  - [Simplify user page](#simplify-user-page)
 - [Configuration parameters](#configuration-parameters)
   - [Use PostgreSQL data provider](#use-postgresql-data-provider)
   - [Use MySQL/MariaDB data provider](#use-mysqlmariadb-data-provider)
@@ -237,6 +240,87 @@ sftp> quit
 
 The last upload failed since we exceeded the number of files quota limit.
 
+## Groups
+
+Using groups simplifies the administration of multiple SFTPGo users: you can assign settings once to a group, instead of multiple times to each individual user.
+
+SFTPGo supports the following types of groups:
+
+- primary groups
+- secondary groups
+- membership groups
+
+A user can be a member of a primary group and many secondary and membership groups. Depending on the group type, the settings are inherited differently, more details [here](../groups.md).
+
+:warning: SFTPGo groups are completely unrelated to system groups. Therefore, it is not necessary to add Linux/Windows groups to use SFTPGo groups.
+
+### Usage example
+
+Suppose you have the following requirements:
+
+- each user must be restricted to a local home directory containing the username as last element of the path, for example `/srv/sftpgo/data/<username>`
+- for each user, the maximum upload size for a single file must be limited to 1GB
+- each user must have an S3 virtual folder available in the path `/s3<username>` and each user can only access a specified "prefix" of the S3 bucket. It must not be able to access other users' files
+- each user must have an S3 virtual folder available in the path `/shared`. This is a folder shared with other users
+- a group of users can only download and list contents in the `/shared` path while another group of users have full access
+
+We can easily meet these requirements by defining two groups.
+
+From the SFTPGo WebAdmin UI, click on `Folders` and then on the `+` icon.
+
+Create a folder named `S3private`.
+Set the storage to `AWS S3 (Compatible)` and fill the required parameters:
+
+- bucket name
+- region
+- credentials: access key and access secret
+
+![S3Private folder](./img/s3-private-folder.png)
+
+The important part is the `Key Prefix`, set it to `users/%username%/`
+
+![S3Private Key Prefix](./img/s3-key-prefix.png)
+
+The placeholder `%username%` will be replaced with the associated username.
+
+Create another folder named `S3shared` with the same settings as `S3private` but this time set the `Key Prefix` to `shared/`.
+The `Key Prefix` has no placeholder, so the folder will operate on a static path that won't change based on the associated user.
+
+Now click on `Groups` and then on the `+` icon and add a group named `Primary`.
+
+Set the `Home Dir` to `/srv/sftpgo/data/%username%`.
+
+![Add group](./img/add-group.png)
+
+As before, the placeholder `%username%` will be replaced with the associated username.
+
+Add the two virtual folders to this group and set the `Max file upload size` to 1GB.
+
+![Primary group settings](./img/primary-group-settings.png)
+
+Add a new group and name it `SharedReadOnly`, in the ACLs section set the permission on the `/shared` path so that read only access is granted.
+
+![Read-only share](./img/read-only-share.png)
+
+The group setup is now complete. We can now create our users and set the primary group to `Primary`.
+For the users who need read-only access to the `/shared` path we also have to set `SharedReadOnly` as a secondary group.
+
+You can now login with any SFTP client like FileZilla, WinSCP etc. and verify that the requirements are met.
+
+### Simplify user page
+
+The add/update user page has many configuration options and can be intimidating for some administrators. We can hide most of the settings and automatically add groups to newly created users. This way the hidden settings are inherited from the automatically assigned groups and therefore administrators can add new users simply by setting the username and credentials.
+
+Click on `Admins` and then on the `+` icon and add an admin named `simply`.
+In the `Groups for users` section set `Primary` as primary group and `SharedReadOnly` as `seconday` group.
+In the `User page preferences` section hide all the sections.
+
+![Simplified admin](./img/simplified-admin.png)
+
+Log in using the newly created administrator and try to add a new user. The user page is simplified as you can see in the following screen.
+
+![Simplified user add](./img/add-user-simplified.png)
+
 ## Configuration parameters
 
 Until now we used the default configuration, to change the global service parameters you have to edit the configuration file, or set appropriate environment variables, and restart SFTPGo to apply the changes.
@@ -245,6 +329,11 @@ A full explanation of all configuration methods can be found [here](./../full-co
 
 The default configuration file is `sftpgo.json` and it can be found within the `/etc/sftpgo` directory if you installed from Linux distro packages. On Windows the configuration file can be found within the `{commonappdata}\SFTPGo` directory where `{commonappdata}` is typically `C:\ProgramData`. SFTPGo also supports reading from TOML and YAML configuration files.
 
+The configuration file can change between different versions and merging your custom settings with the default configuration file, after updating SFTPGo, may be time-consuming. For this reason we suggest to set your custom settings using environment variables.
+If you install SFTPGo on Linux using the official deb/rpm packages you can set your custom environment variables in the file `/etc/sftpgo/sftpgo.env`.
+SFTPGo also reads files inside the `env.d` directory relative to config dir (`/etc/sftpgo/env.d` on Linux and `{commonappdata}\SFTPGo\env.d` on Windows) and then exports the valid variables into environment variables if they are not already set.
+Of course you can also set environment variables with the method provided by the operating system of your choice.
+
 The following snippets assume your are running SFTPGo on Linux but they can be easily adapted for other operating systems.
 
 ### Use PostgreSQL data provider
@@ -259,7 +348,7 @@ grant all privileges on database "sftpgo" to "sftpgo";
 \q
 ```
 
-Open the SFTPGo configuration file, search for the `data_provider` section and change it as follow.
+You can open the SFTPGo configuration file, search for the `data_provider` section and change it as follow.
 
 ```json
   "data_provider": {
@@ -273,6 +362,17 @@ Open the SFTPGo configuration file, search for the `data_provider` section and c
 }
 ```
 
+Alternatively (recommended), you can use environment variables by creating the file `/etc/sftpgo/env.d/postgresql.env` with the following content.
+
+```shell
+SFTPGO_DATA_PROVIDER__DRIVER=postgresql
+SFTPGO_DATA_PROVIDER__NAME=sftpgo
+SFTPGO_DATA_PROVIDER__HOST=127.0.0.1
+SFTPGO_DATA_PROVIDER__PORT=5432
+SFTPGO_DATA_PROVIDER__USERNAME=sftpgo
+SFTPGO_DATA_PROVIDER__PASSWORD=your password here
+```
+
 Confirm that the database connection works by initializing the data provider.
 
 ```shell
@@ -313,7 +413,7 @@ MariaDB [(none)]> quit
 Bye
 ```
 
-Open the SFTPGo configuration file, search for the `data_provider` section and change it as follow.
+You can open the SFTPGo configuration file, search for the `data_provider` section and change it as follow.
 
 ```json
   "data_provider": {
@@ -327,6 +427,17 @@ Open the SFTPGo configuration file, search for the `data_provider` section and c
 }
 ```
 
+Alternatively (recommended), you can use environment variables by creating the file `/etc/sftpgo/env.d/mysql.env` with the following content.
+
+```shell
+SFTPGO_DATA_PROVIDER__DRIVER=mysql
+SFTPGO_DATA_PROVIDER__NAME=sftpgo
+SFTPGO_DATA_PROVIDER__HOST=127.0.0.1
+SFTPGO_DATA_PROVIDER__PORT=3306
+SFTPGO_DATA_PROVIDER__USERNAME=sftpgo
+SFTPGO_DATA_PROVIDER__PASSWORD=your password here
+```
+
 Confirm that the database connection works by initializing the data provider.
 
 ```shell
@@ -357,7 +468,7 @@ We suppose you have installed CockroachDB this way:
 
 ```shell
 sudo su
-export CRDB_VERSION=22.1.0 # set the latest available version here
+export CRDB_VERSION=22.1.8 # set the latest available version here
 wget -qO- https://binaries.cockroachdb.com/cockroach-v${CRDB_VERSION}.linux-amd64.tgz | tar xvz
 cp -i cockroach-v${CRDB_VERSION}.linux-amd64/cockroach /usr/local/bin/
 mkdir -p /usr/local/lib/cockroach
@@ -387,9 +498,8 @@ ExecStart=/usr/local/bin/cockroach start-single-node --certs-dir=/etc/cockroach/
 TimeoutStopSec=60
 Restart=always
 RestartSec=10
-StandardOutput=syslog
+StandardOutput=journal
-StandardError=syslog
+StandardError=journal
-SyslogIdentifier=cockroach
 User=sftpgo
 [Install]
 WantedBy=default.target
@@ -404,7 +514,7 @@ CREATE DATABASE
 Time: 13ms
 ```
 
-Open the SFTPGo configuration file, search for the `data_provider` section and change it as follow.
+You can open the SFTPGo configuration file, search for the `data_provider` section and change it as follow.
 
 ```json
   "data_provider": {
@@ -422,6 +532,20 @@ Open the SFTPGo configuration file, search for the `data_provider` section and c
 }
 ```
 
+Alternatively (recommended), you can use environment variables by creating the file `/etc/sftpgo/env.d/cockroachdb.env` with the following content.
+
+```shell
+SFTPGO_DATA_PROVIDER__DRIVER=cockroachdb
+SFTPGO_DATA_PROVIDER__NAME=sftpgo
+SFTPGO_DATA_PROVIDER__HOST=localhost
+SFTPGO_DATA_PROVIDER__PORT=26257
+SFTPGO_DATA_PROVIDER__USERNAME=root
+SFTPGO_DATA_PROVIDER__SSLMODE=3
+SFTPGO_DATA_PROVIDER__ROOT_CERT="/etc/cockroach/certs/ca.crt"
+SFTPGO_DATA_PROVIDER__CLIENT_CERT="/etc/cockroach/certs/client.root.crt"
+SFTPGO_DATA_PROVIDER__CLIENT_KEY="/etc/cockroach/certs/client.root.key"
+```
+
 Confirm that the database connection works by initializing the data provider.
 
 ```shell
@@ -452,7 +576,7 @@ Restart SFTPGo to apply the changes.
 
 ### Enable FTP service
 
-Open the SFTPGo configuration file, search for the `ftpd` section and change it as follow.
+You can set the configuration options to enable the FTP service by opening the SFTPGo configuration file, looking for the `ftpd` section and editing it as follows.
 
 ```json
   "ftpd": {
@@ -485,6 +609,12 @@ Open the SFTPGo configuration file, search for the `ftpd` section and change it
   }
 ```
 
+Alternatively (recommended), you can use environment variables by creating the file `/etc/sftpgo/env.d/ftpd.env` with the following content.
+
+```shell
+SFTPGO_FTPD__BINDINGS__0__PORT=2121
+```
+
 Restart SFTPGo to apply the changes. The FTP service is now available on port `2121`.
 
 You can also configure the passive ports range (`50000-50100` by default), these ports must be reachable for passive FTP to work. If your FTP server is on the private network side of a NAT configuration you have to set `force_passive_ip` to your external IP address. You may also need to open the passive port range on your firewall.
@@ -493,7 +623,7 @@ It is recommended that you provide a certificate and key file to expose FTP over
 
 ### Enable WebDAV service
 
-Open the SFTPGo configuration file, search for the `webdavd` section and change it as follow.
+You can set the configuration options to enable the FTP service by opening the SFTPGo configuration file, looking for the `webdavd` section and editing it as follows.
 
 ```json
   "webdavd": {
@@ -510,11 +640,18 @@ Open the SFTPGo configuration file, search for the `webdavd` section and change
         "prefix": "",
         "proxy_allowed": [],
         "client_ip_proxy_header": "",
-        "client_ip_header_depth": 0
+        "client_ip_header_depth": 0,
+        "disable_www_auth_header": false
       }
     ],
     ...
   }
 ```
 
+Alternatively (recommended), you can use environment variables by creating the file `/etc/sftpgo/env.d/webdavd.env` with the following content.
+
+```shell
+SFTPGO_WEBDAVD__BINDINGS__0__PORT=10080
+```
+
 Restart SFTPGo to apply the changes. The WebDAV service is now available on port `10080`. It is recommended that you provide a certificate and key file to expose WebDAV over https.

docs/howto/lets-encrypt-certificate.md

@@ -117,7 +117,7 @@ When the certificate is renewed you should see SFTPGo logs like the following to
 
 ## Obtaining a certificate using the ACME protocol built into SFTPGo
 
-Open the SFTPGo configuration file, search for the `acme` section and change it as follow.
+You can open the SFTPGo configuration file, search for the `acme` section and change it as follow.
 
 ```json
   "acme": {
@@ -138,6 +138,14 @@ Open the SFTPGo configuration file, search for the `acme` section and change it
   }
 ```
 
+Alternatively (recommended), you can use environment variables by creating the file `/etc/sftpgo/env.d/acme.env` with the following content.
+
+```shell
+SFTPGO_ACME__DOMAINS="sftpgo.com"
+SFTPGO_ACME__EMAIL="<you email address here>"
+SFTPGO_ACME__HTTP01_CHALLENGE__WEBROOT="/var/www/sftpgo.com"
+```
+
 Make sure that the `sftpgo` user can write to the `/var/www/sftpgo.com` directory or pre-create the `/var/www/sftpgo.com/.well-known/acme-challenge` directory with the appropriate permissions.
 This directory must be publicly served by your web server.
 
@@ -151,7 +159,7 @@ If this command completes successfully, you are done. The SFTPGo service will ta
 
 ## Enable HTTPS for SFTPGo Web UI and REST API
 
-Open the SFTPGo configuration file, search for the `httpd` section and change it as follow.
+You can open the SFTPGo configuration file, search for the `httpd` section and change it as follow.
 
 ```json
   "httpd": {
@@ -161,17 +169,27 @@ Open the SFTPGo configuration file, search for the `httpd` section and change it
         "address": "",
         "enable_web_admin": true,
         "enable_web_client": true,
+        "enable_rest_api": true,
         "enable_https": true,
         "certificate_file": "/var/lib/sftpgo/certs/sftpgo.com.crt",
         "certificate_key_file": "/var/lib/sftpgo/certs/sftpgo.com.key",
         .....
 ```
 
+Alternatively (recommended), you can use environment variables by creating the file `/etc/sftpgo/env.d/httpd.env` with the following content.
+
+```shell
+SFTPGO_HTTPD__BINDINGS__0__PORT=9443
+SFTPGO_HTTPD__BINDINGS__0__ENABLE_HTTPS=1
+SFTPGO_HTTPD__BINDINGS__0__CERTIFICATE_FILE="/var/lib/sftpgo/certs/sftpgo.com.crt"
+SFTPGO_HTTPD__BINDINGS__0__CERTIFICATE_KEY_FILE="/var/lib/sftpgo/certs/sftpgo.com.key"
+```
+
 Restart SFTPGo to apply the changes. The HTTPS service is now available on port `9443`.
 
 ## Enable HTTPS for WebDAV service
 
-Open the SFTPGo configuration file, search for the `webdavd` section and change it as follow.
+You can open the SFTPGo configuration file, search for the `webdavd` section and change it as follow.
 
 ```json
   "webdavd": {
@@ -185,11 +203,20 @@ Open the SFTPGo configuration file, search for the `webdavd` section and change
         ...
 ```
 
+Alternatively (recommended), you can use environment variables by creating the file `/etc/sftpgo/env.d/webdavd.env` with the following content.
+
+```shell
+SFTPGO_WEBDAVD__BINDINGS__0__PORT=10443
+SFTPGO_WEBDAVD__BINDINGS__0__ENABLE_HTTPS=1
+SFTPGO_WEBDAVD__CERTIFICATE_FILE="/var/lib/sftpgo/certs/sftpgo.com.crt"
+SFTPGO_WEBDAVD__CERTIFICATE_KEY_FILE="/var/lib/sftpgo/certs/sftpgo.com.key"
+```
+
 Restart SFTPGo to apply the changes. WebDAV is now availble over HTTPS on port `10443`.
 
 ## Enable explicit FTP over TLS
 
-Open the SFTPGo configuration file, search for the `ftpd` section and change it as follow.
+You can open the SFTPGo configuration file, search for the `ftpd` section and change it as follow.
 
 ```json
   "ftpd": {
@@ -204,4 +231,13 @@ Open the SFTPGo configuration file, search for the `ftpd` section and change it
         ...
 ```
 
+Alternatively (recommended), you can use environment variables by creating the file `/etc/sftpgo/env.d/ftpd.env` with the following content.
+
+```shell
+SFTPGO_FTPD__BINDINGS__0__PORT=2121
+SFTPGO_FTPD__BINDINGS__0__TLS_MODE=1
+SFTPGO_FTPD__BINDINGS__0__CERTIFICATE_FILE="/var/lib/sftpgo/certs/sftpgo.com.crt"
+SFTPGO_FTPD__BINDINGS__0__CERTIFICATE_KEY_FILE="/var/lib/sftpgo/certs/sftpgo.com.key"
+```
+
 Restart SFTPGo to apply the changes. FTPES service is now available on port `2121` and TLS is required for both control and data connection (`tls_mode` is 1).

docs/howto/postgresql-s3.md

@@ -144,6 +144,17 @@ Search for the `data_provider` section and change it as follow.
 }
 ```
 
+Alternatively (recommended), you can use environment variables by creating the file `/etc/sftpgo/env.d/postgresql.env` with the following content.
+
+```shell
+SFTPGO_DATA_PROVIDER__DRIVER=postgresql
+SFTPGO_DATA_PROVIDER__NAME="sftpgo.db"
+SFTPGO_DATA_PROVIDER__HOST=127.0.0.1
+SFTPGO_DATA_PROVIDER__PORT=5432
+SFTPGO_DATA_PROVIDER__USERNAME=sftpgo
+SFTPGO_DATA_PROVIDER__PASSWORD=sftpgo_pg_pwd
+```
+
 This way we set the PostgreSQL connection parameters.
 
 If you want to connect to PostgreSQL over a Unix Domain socket you have to set the value `/var/run/postgresql` for the `host` configuration key instead of `127.0.0.1`.

docs/howto/two-factor-authentication.md

@@ -42,6 +42,8 @@ SFTPGo can use 2FA for `HTTP`, `SSH` (SFTP, SCP) and `FTP` protocols. If you pla
     ...
 ```
 
+Or setting the environment variable `SFTPGO_SFTPD__KEYBOARD_INTERACTIVE_AUTHENTICATION=1`.
+
 ## Enable 2FA for admins
 
 Each admin can view/change his/her two-factor authentication by selecting the `Two-Factor Auth` link from the top-right web UI menu.

go.mod

@@ -34,7 +34,7 @@ require (
 	github.com/hashicorp/go-hclog v1.3.1
 	github.com/hashicorp/go-plugin v1.4.5
 	github.com/hashicorp/go-retryablehttp v0.7.1
-	github.com/jackc/pgx/v5 v5.0.2
+	github.com/jackc/pgx/v5 v5.0.3
 	github.com/jlaffaye/ftp v0.0.0-20201112195030-9aae4d151126
 	github.com/klauspost/compress v1.15.11
 	github.com/lestrrat-go/jwx v1.2.25
@@ -57,7 +57,7 @@ require (
 	github.com/spf13/cobra v1.6.0
 	github.com/spf13/viper v1.13.0
 	github.com/stretchr/testify v1.8.0
-	github.com/studio-b12/gowebdav v0.0.0-20221012160928-e70a598e946e
+	github.com/studio-b12/gowebdav v0.0.0-20221015232716-17255f2e7423
 	github.com/subosito/gotenv v1.4.1
 	github.com/unrolled/secure v1.13.0
 	github.com/wagslane/go-password-validator v0.3.0
@@ -68,10 +68,10 @@ require (
 	gocloud.dev v0.27.0
 	golang.org/x/crypto v0.0.0-20221012134737-56aed061732a
 	golang.org/x/net v0.0.0-20221014081412-f15817d10f9b
-	golang.org/x/oauth2 v0.0.0-20221006150949-b44042a4b9c1
+	golang.org/x/oauth2 v0.0.0-20221014153046-6fdb5e3db783
 	golang.org/x/sys v0.0.0-20221013171732-95e765b1cc43
 	golang.org/x/time v0.0.0-20220922220347-f3bd1da661af
-	google.golang.org/api v0.98.0
+	google.golang.org/api v0.99.0
 	gopkg.in/natefinch/lumberjack.v2 v2.0.0
 )
 
@@ -111,7 +111,7 @@ require (
 	github.com/golang/protobuf v1.5.2 // indirect
 	github.com/google/go-cmp v0.5.9 // indirect
 	github.com/googleapis/enterprise-certificate-proxy v0.2.0 // indirect
-	github.com/googleapis/gax-go/v2 v2.5.1 // indirect
+	github.com/googleapis/gax-go/v2 v2.6.0 // indirect
 	github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
 	github.com/hashicorp/hcl v1.0.0 // indirect
 	github.com/hashicorp/yamux v0.1.1 // indirect
@@ -159,8 +159,8 @@ require (
 	golang.org/x/tools v0.1.12 // indirect
 	golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 // indirect
 	google.golang.org/appengine v1.6.7 // indirect
-	google.golang.org/genproto v0.0.0-20221013201013-33fc6f83cba4 // indirect
+	google.golang.org/genproto v0.0.0-20221014213838-99cd37c6964a // indirect
-	google.golang.org/grpc v1.50.0 // indirect
+	google.golang.org/grpc v1.50.1 // indirect
 	google.golang.org/protobuf v1.28.1 // indirect
 	gopkg.in/ini.v1 v1.67.0 // indirect
 	gopkg.in/square/go-jose.v2 v2.6.0 // indirect

go.sum

@@ -863,8 +863,8 @@ github.com/googleapis/gax-go/v2 v2.1.1/go.mod h1:hddJymUZASv3XPyGkUpKj8pPO47Rmb0
 github.com/googleapis/gax-go/v2 v2.2.0/go.mod h1:as02EH8zWkzwUoLbBaFeQ+arQaj/OthfcblKl4IGNaM=
 github.com/googleapis/gax-go/v2 v2.3.0/go.mod h1:b8LNqSzNabLiUpXKkY7HAR5jr6bIT99EXz9pXxye9YM=
 github.com/googleapis/gax-go/v2 v2.4.0/go.mod h1:XOTVJ59hdnfJLIP/dh8n5CGryZR2LxK9wbMD5+iXC6c=
-github.com/googleapis/gax-go/v2 v2.5.1 h1:kBRZU0PSuI7PspsSb/ChWoVResUcwNVIdpB049pKTiw=
+github.com/googleapis/gax-go/v2 v2.6.0 h1:SXk3ABtQYDT/OH8jAyvEOQ58mgawq5C4o/4/89qN2ZU=
-github.com/googleapis/gax-go/v2 v2.5.1/go.mod h1:h6B0KMMFNtI2ddbGJn3T3ZbwkeT6yqEF02fYlzkUCyo=
+github.com/googleapis/gax-go/v2 v2.6.0/go.mod h1:1mjbznJAPHFpesgE5ucqfYEscaz5kMdcIDwU/6+DDoY=
 github.com/googleapis/gnostic v0.4.1/go.mod h1:LRhVm6pbyptWbWbuZ38d1eyptfvIytN3ir6b65WBswg=
 github.com/googleapis/gnostic v0.5.1/go.mod h1:6U4PtQXGIEt/Z3h5MAT7FNofLnw9vXk2cUuW7uA/OeU=
 github.com/googleapis/gnostic v0.5.5/go.mod h1:7+EbHbldMins07ALC74bsA81Ovc97DwqyJO1AENw9kA=
@@ -1015,8 +1015,8 @@ github.com/jackc/pgx/v4 v4.0.0-pre1.0.20190824185557-6972a5742186/go.mod h1:X+GQ
 github.com/jackc/pgx/v4 v4.12.1-0.20210724153913-640aa07df17c/go.mod h1:1QD0+tgSXP7iUjYm9C1NxKhny7lq6ee99u/z+IHFcgs=
 github.com/jackc/pgx/v4 v4.16.0/go.mod h1:N0A9sFdWzkw/Jy1lwoiB64F2+ugFZi987zRxcPez/wI=
 github.com/jackc/pgx/v4 v4.16.1/go.mod h1:SIhx0D5hoADaiXZVyv+3gSm3LCIIINTVO0PficsvWGQ=
-github.com/jackc/pgx/v5 v5.0.2 h1:V+EonE9i33VwJR9YIHRdglAmrODLLkwIdHjko6b1rRk=
+github.com/jackc/pgx/v5 v5.0.3 h1:4flM5ecR/555F0EcnjdaZa6MhBU+nr0QbZIo5vaKjuM=
-github.com/jackc/pgx/v5 v5.0.2/go.mod h1:JBbvW3Hdw77jKl9uJrEDATUZIFM2VFPzRq4RWIhkF4o=
+github.com/jackc/pgx/v5 v5.0.3/go.mod h1:JBbvW3Hdw77jKl9uJrEDATUZIFM2VFPzRq4RWIhkF4o=
 github.com/jackc/puddle v0.0.0-20190413234325-e4ced69a3a2b/go.mod h1:m4B5Dj62Y0fbyuIc15OsIqK0+JU8nkqQjsgx7dvjSWk=
 github.com/jackc/puddle v0.0.0-20190608224051-11cab39313c9/go.mod h1:m4B5Dj62Y0fbyuIc15OsIqK0+JU8nkqQjsgx7dvjSWk=
 github.com/jackc/puddle v1.1.3/go.mod h1:m4B5Dj62Y0fbyuIc15OsIqK0+JU8nkqQjsgx7dvjSWk=
@@ -1528,8 +1528,8 @@ github.com/stretchr/testify v1.7.2/go.mod h1:R6va5+xMeoiuVRoj+gSkQ7d3FALtqAAGI1F
 github.com/stretchr/testify v1.7.5/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
 github.com/stretchr/testify v1.8.0 h1:pSgiaMZlXftHpm5L7V1+rVB+AZJydKsMxsQBIJw4PKk=
 github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
-github.com/studio-b12/gowebdav v0.0.0-20221012160928-e70a598e946e h1:YYZypSKAFKjg4Qfdq1iSYckvfLW7Wl7az7I4TQxogRw=
+github.com/studio-b12/gowebdav v0.0.0-20221015232716-17255f2e7423 h1:Wd8WDEEusB5+En4PiRWJp1cP59QLNsQun+mOTW8+s6s=
-github.com/studio-b12/gowebdav v0.0.0-20221012160928-e70a598e946e/go.mod h1:bHA7t77X/QFExdeAnDzK6vKM34kEZAcE1OX4MfiwjkE=
+github.com/studio-b12/gowebdav v0.0.0-20221015232716-17255f2e7423/go.mod h1:bHA7t77X/QFExdeAnDzK6vKM34kEZAcE1OX4MfiwjkE=
 github.com/subosito/gotenv v1.2.0/go.mod h1:N0PQaV/YGNqwC0u51sEeR/aUtSLEXKX9iv69rRypqCw=
 github.com/subosito/gotenv v1.4.1 h1:jyEFiXpy21Wm81FBN71l9VoMMV8H8jG+qIK3GCpY6Qs=
 github.com/subosito/gotenv v1.4.1/go.mod h1:ayKnFf/c6rvx/2iiLrJUk1e6plDbT3edrFNGqEflhK0=
@@ -1762,8 +1762,8 @@ golang.org/x/oauth2 v0.0.0-20220622183110-fd043fe589d2/go.mod h1:jaDAt6Dkxork7Lm
 golang.org/x/oauth2 v0.0.0-20220628200809-02e64fa58f26/go.mod h1:jaDAt6Dkxork7LmZnYtzbRWj0W47D86a3TGe0YHBvmE=
 golang.org/x/oauth2 v0.0.0-20220722155238-128564f6959c/go.mod h1:h4gKUeWbJ4rQPri7E0u6Gs4e9Ri2zaLxzw5DI5XGrYg=
 golang.org/x/oauth2 v0.0.0-20220822191816-0ebed06d0094/go.mod h1:h4gKUeWbJ4rQPri7E0u6Gs4e9Ri2zaLxzw5DI5XGrYg=
-golang.org/x/oauth2 v0.0.0-20221006150949-b44042a4b9c1 h1:3VPzK7eqH25j7GYw5w6g/GzNRc0/fYtrxz27z1gD4W0=
+golang.org/x/oauth2 v0.0.0-20221014153046-6fdb5e3db783 h1:nt+Q6cXKz4MosCSpnbMtqiQ8Oz0pxTef2B4Vca2lvfk=
-golang.org/x/oauth2 v0.0.0-20221006150949-b44042a4b9c1/go.mod h1:h4gKUeWbJ4rQPri7E0u6Gs4e9Ri2zaLxzw5DI5XGrYg=
+golang.org/x/oauth2 v0.0.0-20221014153046-6fdb5e3db783/go.mod h1:h4gKUeWbJ4rQPri7E0u6Gs4e9Ri2zaLxzw5DI5XGrYg=
 golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
@@ -1777,8 +1777,8 @@ golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJ
 golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20220601150217-0de741cfad7f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
-golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4 h1:uVc8UZUe6tr40fFVnUP5Oj+veunVezqYl9z7DYw9xzw=
 golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20220929204114-8fcdb60fdcc0 h1:cu5kTvlzcw1Q5S9f5ip1/cpiB4nXvw1XYzFPGgzLUOY=
 golang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
@@ -2097,8 +2097,8 @@ google.golang.org/api v0.85.0/go.mod h1:AqZf8Ep9uZ2pyTvgL+x0D3Zt0eoT9b5E8fmzfu6F
 google.golang.org/api v0.86.0/go.mod h1:+Sem1dnrKlrXMR/X0bPnMWyluQe4RsNoYfmNLhOIkzw=
 google.golang.org/api v0.90.0/go.mod h1:+Sem1dnrKlrXMR/X0bPnMWyluQe4RsNoYfmNLhOIkzw=
 google.golang.org/api v0.91.0/go.mod h1:+Sem1dnrKlrXMR/X0bPnMWyluQe4RsNoYfmNLhOIkzw=
-google.golang.org/api v0.98.0 h1:yxZrcxXESimy6r6mdL5Q6EnZwmewDJK2dVg3g75s5Dg=
+google.golang.org/api v0.99.0 h1:tsBtOIklCE2OFxhmcYSVqGwSAN/Y897srxmcvAQnwK8=
-google.golang.org/api v0.98.0/go.mod h1:w7wJQLTM+wvQpNf5JyEcBoxK0RH7EDrh/L4qfsuJ13s=
+google.golang.org/api v0.99.0/go.mod h1:1YOf74vkVndF7pG6hIHuINsM7eWwpVTAfNMNiL91A08=
 google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
 google.golang.org/appengine v1.2.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
 google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
@@ -2209,8 +2209,8 @@ google.golang.org/genproto v0.0.0-20220617124728-180714bec0ad/go.mod h1:KEWEmljW
 google.golang.org/genproto v0.0.0-20220624142145-8cd45d7dbd1f/go.mod h1:KEWEmljWE5zPzLBa/oHl6DaEt9LmfH6WtH1OHIvleBA=
 google.golang.org/genproto v0.0.0-20220628213854-d9e0b6570c03/go.mod h1:KEWEmljWE5zPzLBa/oHl6DaEt9LmfH6WtH1OHIvleBA=
 google.golang.org/genproto v0.0.0-20220802133213-ce4fa296bf78/go.mod h1:iHe1svFLAZg9VWz891+QbRMwUv9O/1Ww+/mngYeThbc=
-google.golang.org/genproto v0.0.0-20221013201013-33fc6f83cba4 h1:nZ28yoLJWNLTcERW43BN+JDsNQOdiZOFB9Dly/IUrjw=
+google.golang.org/genproto v0.0.0-20221014213838-99cd37c6964a h1:GH6UPn3ixhWcKDhpnEC55S75cerLPdpp3hrhfKYjZgw=
-google.golang.org/genproto v0.0.0-20221013201013-33fc6f83cba4/go.mod h1:1vXfmgAz9N9Jx0QA82PqRVauvCz1SGSz739p0f183jM=
+google.golang.org/genproto v0.0.0-20221014213838-99cd37c6964a/go.mod h1:1vXfmgAz9N9Jx0QA82PqRVauvCz1SGSz739p0f183jM=
 google.golang.org/grpc v0.0.0-20160317175043-d3ddb4469d5a/go.mod h1:yo6s7OP7yaDglbqo1J04qKzAhqBH6lvTonzMVmEdcZw=
 google.golang.org/grpc v1.17.0/go.mod h1:6QZJwpn2B+Zp71q/5VxRsJ6NXXVCE5NRUHRo+f3cWCs=
 google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
@@ -2252,8 +2252,8 @@ google.golang.org/grpc v1.46.0/go.mod h1:vN9eftEi1UMyUsIF80+uQXhHjbXYbm0uXoFCACu
 google.golang.org/grpc v1.46.2/go.mod h1:vN9eftEi1UMyUsIF80+uQXhHjbXYbm0uXoFCACuMGWk=
 google.golang.org/grpc v1.47.0/go.mod h1:vN9eftEi1UMyUsIF80+uQXhHjbXYbm0uXoFCACuMGWk=
 google.golang.org/grpc v1.48.0/go.mod h1:vN9eftEi1UMyUsIF80+uQXhHjbXYbm0uXoFCACuMGWk=
-google.golang.org/grpc v1.50.0 h1:fPVVDxY9w++VjTZsYvXWqEf9Rqar/e+9zYfxKK+W+YU=
+google.golang.org/grpc v1.50.1 h1:DS/BukOZWp8s6p4Dt/tOaJaTQyPyOoCcrjroHuCeLzY=
-google.golang.org/grpc v1.50.0/go.mod h1:ZgQEeidpAuNRZ8iRrlBKXZQP1ghovWIVhdJRyCDK+GI=
+google.golang.org/grpc v1.50.1/go.mod h1:ZgQEeidpAuNRZ8iRrlBKXZQP1ghovWIVhdJRyCDK+GI=
 google.golang.org/grpc/cmd/protoc-gen-go-grpc v1.1.0/go.mod h1:6Kw0yEErY5E/yWrBtf03jp27GLLJujG4z/JK95pnjjw=
 google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
 google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=

pkgs/build.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 
-NFPM_VERSION=2.19.2
+NFPM_VERSION=2.20.0
 NFPM_ARCH=${NFPM_ARCH:-amd64}
 if [ -z ${SFTPGO_VERSION} ]
 then

pkgs/choco/tools/ChocolateyInstall.ps1

@@ -39,14 +39,14 @@ Write-Output "Default data location:"
 Write-Output "`t$DefaultDataPath"
 Write-Output "Default configuration file location:"
 Write-Output "`t$DefaultConfigurationFilePath"
-Write-Output "Directory to create environment variable files to set configuration options:"
+Write-Output "Directory to create environment variable files to set custom configurations:"
 Write-Output "`t$EnvDirPath"
 Write-Output "If the SFTPGo service does not start, make sure that TCP ports 2022 and 8080 are"
 Write-Output "not used by other services or change the SFTPGo configuration to suit your needs."
 Write-Output ""
 Write-Output "General information (README) location:"
 Write-Output "`thttps://github.com/drakkan/sftpgo"
-Write-Output "Getting start guide location:"
+Write-Output "Getting started guide location:"
 Write-Output "`thttps://github.com/drakkan/sftpgo/blob/v2.3.6/docs/howto/getting-started.md"
 Write-Output "Detailed information (docs folder) location:"
 Write-Output "`thttps://github.com/drakkan/sftpgo/tree/v2.3.6/docs"

windows-installer/README.txt

@@ -25,6 +25,13 @@ Directory to create environment variable files to set configuration options:
 
 C:\ProgramData\SFTPGo\env.d
 
+It is recommended that you set custom configurations as environment variables by creating files in
+the env.d directory.
+This eliminates the need to merge your changes with the default configuration file after each update.
+You can simply replace the configuration file with the default one after updating SFTPGo.
+
+https://github.com/drakkan/sftpgo/blob/main/docs/full-configuration.md
+
 Getting started guide:
 
 https://github.com/drakkan/sftpgo/blob/main/docs/howto/getting-started.md