Support multiple private (host) keys

With this patch, one can configure one or more private (host) keys in
the configuration file.

I made it a nested struct, so we can add more options later. Eg. host
certificates might be a useful addition if I can figure out how this is
done in golang's crypto/ssh...

Signed-off-by: Jo Vandeginste <Jo.Vandeginste@kuleuven.be>

README.md

@@ -6,28 +6,28 @@ Full featured and highly configurable SFTP server software
 ## Features
 
 - Each account is chrooted to his Home Dir
-- SFTP accounts are virtual accounts stored in a "data provider" 
+- SFTP accounts are virtual accounts stored in a "data provider"
 - SQLite, MySQL and PostgreSQL data providers are supported. The `Provider` interface could be extended to support non SQL backends too
 - Public key and password authentication
 - Quota support: accounts can have individual quota expressed as max number of files and max total size
 - Bandwidth throttling is supported, with distinct settings for upload and download
 - Per user maximum concurrent sessions
 - Per user permissions: list directories content, upload, download, delete, rename, create directories, create symlinks can be enabled or disabled
-- Per user files/folders ownership: you can map all the users to the system account that runs SFTPGo (all platforms are supported) or you can run SFTPGo as root user and map each user or group of users to a different system account (*NIX only) 
+- Per user files/folders ownership: you can map all the users to the system account that runs SFTPGo (all platforms are supported) or you can run SFTPGo as root user and map each user or group of users to a different system account (*NIX only)
 - Configurable custom commands and/or HTTP notifications on SFTP upload, download, delete or rename
-- REST API for users and quota management and real time reports for the active connections with possibility of forcibly closing a connection  
+- REST API for users and quota management and real time reports for the active connections with possibility of forcibly closing a connection
 - Log files are accurate and they are saved in the easily parsable JSON format
 - Automatically terminating idle connections
 
 ## Platforms
 
-SFTPGo is developed and tested on Linux. After each commit the code is automatically built and tested on Linux and macOS using Travis CI. 
+SFTPGo is developed and tested on Linux. After each commit the code is automatically built and tested on Linux and macOS using Travis CI.
 Regularly the test cases are manually executed and pass on Windows. Other UNIX variants such as *BSD should work too.
 
 ## Requirements
 
 - Go 1.12 or higher
-- A suitable SQL server to use as data provider: PostreSQL (9+) or MySQL (4.1+) or SQLite 3.x 
+- A suitable SQL server to use as data provider: PostreSQL (9+) or MySQL (4.1+) or SQLite 3.x
 
 ## Installation
 
@@ -49,7 +49,7 @@ Alternately you can use distro packages:
 
 The `sftpgo` executable supports the following command line flags:
 
-- `-config-dir` string. Location of the config dir. This directory should contain the `sftpgo.conf` configuration file, the private key for the SFTP server (`id_rsa` file) and the SQLite database if you use SQLite as data provider. The server private key will be autogenerated if the user that executes SFTPGo has write access to the config-dir. The default value is "."
+- `-config-dir` string. Location of the config dir. This directory should contain the `sftpgo.conf` configuration file and is used as the base for files with a relative path (eg. the private keys for the SFTP server, the SQLite database if you use SQLite as data provider). The default value is "."
 - `-log-file-path` string. Location for the log file, default "sftpgo.log"
 - `-log-max-size` int. Maximum size in megabytes of the log file before it gets rotated. Default 10
 - `-log-max-backups` int. Maximum number of old log files to retain. Default 5
@@ -57,6 +57,8 @@ The `sftpgo` executable supports the following command line flags:
 - `-log-compress` boolean. Determine if the rotated log files should be compressed using gzip
 - `-log-verbose` boolean. Enable verbose logs. Default `true`
 
+If you don't configure any private host keys, the daemon will use `id_rsa` in the configuration directory. If that file doesn't exist, the daemon will attempt to autogenerate it (if the user that executes SFTPGo has write access to the config-dir). The server supports any private key format supported by [`crypto/ssh`](https://github.com/golang/crypto/blob/master/ssh/keys.go#L32).
+
 Before starting `sftpgo` a dataprovider must be configured.
 
 Sample SQL scripts to create the required database structure can be found insite the source tree [sql](https://github.com/drakkan/sftpgo/tree/master/sql "sql") directory. The SQL scripts filename's is, by convention, the date as `YYYYMMDD` and the suffix `.sql`. You need to apply all the SQL scripts for your database ordered by name, for example `20190706.sql` must be applied before `20190728.sql` and so on.
@@ -72,7 +74,7 @@ The `sftpgo.conf` configuration file contains the following sections:
     - `banner`, string. Identification string used by the server. Default "SFTPGo"
     - `actions`, struct. It contains the command to execute and/or the HTTP URL to notify and the trigger conditions
         - `execute_on`, list of strings. Valid values are `download`, `upload`, `delete`, `rename`. On folder deletion a `delete` notification will be sent for each deleted file. Leave empty to disable actions.
-        - `command`, string. Absolute path to the command to execute. Leave empty to disable. The command is invoked with the following arguments: 
+        - `command`, string. Absolute path to the command to execute. Leave empty to disable. The command is invoked with the following arguments:
             - `action`, any valid `execute_on` string
             - `username`, user who did the action
             - `path` to the affected file. For `rename` action this is the old file name
@@ -82,6 +84,8 @@ The `sftpgo.conf` configuration file contains the following sections:
             - `username`
             - `path`
             - `target_path`, added for `rename` action only
+    - `keys`, struct array. It contains the daemon's private keys
+        - `private_key`, path to the private key file
 - **"data_provider"**, the configuration for the data provider
     - `driver`, string. Supported drivers are `sqlite`, `mysql`, `postgresql`
     - `name`, string. Database name. For driver `sqlite` this can be the database name relative to the config dir or the absolute path to the SQLite database.
@@ -105,36 +109,44 @@ Here is a full example showing the default config:
 
 ```json
 {
-   "sftpd":{
+  "sftpd": {
-        "bind_port":2022,
+    "bind_port": 2022,
-        "bind_address":"",
+    "bind_address": "",
-        "idle_timeout":15,
+    "idle_timeout": 15,
-        "max_auth_tries":0,
+    "max_auth_tries": 0,
-        "umask":"0022",
+    "umask": "0022",
-        "banner":"SFTPGo",
+    "banner": "SFTPGo",
-        "actions":{
+    "actions": {
-            "execute_on":[],
+      "execute_on": [],
-            "command":"",
+      "command": "",
-            "http_notification_url":""
+      "http_notification_url": ""
-        }
-   },
-   "data_provider":{
-        "driver":"sqlite",
-        "name":"sftpgo.db",
-        "host":"",
-        "port":5432,
-        "username":"",
-        "password":"",
-        "sslmode":0,
-        "connection_string":"",
-        "users_table":"users",
-        "manage_users":1,
-        "track_quota":2
     },
-    "httpd":{
+    "keys": [
-        "bind_port":8080,
+      {
-        "bind_address":"127.0.0.1"
+        "private_key": "id_rsa"
-    }
+      },
+      {
+        "private_key": "id_ecdsa"
+      }
+    ]
+  },
+  "data_provider": {
+    "driver": "sqlite",
+    "name": "sftpgo.db",
+    "host": "",
+    "port": 5432,
+    "username": "",
+    "password": "",
+    "sslmode": 0,
+    "connection_string": "",
+    "users_table": "users",
+    "manage_users": 1,
+    "track_quota": 2
+  },
+  "httpd": {
+    "bind_port": 8080,
+    "bind_address": "127.0.0.1"
+  }
 }
 ```
 
@@ -142,8 +154,8 @@ Here is a full example showing the default config:
 
 For each account the following properties can be configured:
 
-- `username` 
+- `username`
-- `password` used for password authentication. For users created using SFTPGo REST API the password will be stored using argon2id hashing algo. SFTPGo supports checking passwords stored with bcrypt too. Currently, as fallback, there is a clear text password checking but you should not store passwords as clear text and this support could be removed at any time, so please don't depend on it. 
+- `password` used for password authentication. For users created using SFTPGo REST API the password will be stored using argon2id hashing algo. SFTPGo supports checking passwords stored with bcrypt too. Currently, as fallback, there is a clear text password checking but you should not store passwords as clear text and this support could be removed at any time, so please don't depend on it.
 - `public_key` used for public key authentication. At least one between password and public key is mandatory. Multiple public keys are supported, newline delimited (unix line break unescaped).
 - `home_dir` The user cannot upload or download files outside this directory. Must be an absolute path
 - `uid`, `gid`. If sftpgo runs as root system user then the created files and directories will be assigned to this system uid/gid. Ignored on windows and if sftpgo runs as non root user: in this case files and directories for all SFTP users will be owned by the system user that runs sftpgo.
@@ -172,7 +184,7 @@ If quota tracking is enabled in `sftpgo.conf` configuration file, then the used
 
 REST API is designed to run on localhost or on a trusted network, if you need https or authentication you can setup a reverse proxy using an HTTP Server such as Apache or NGNIX.
 
-The OpenAPI 3 schema for the exposed API can be found inside the source tree: [openapi.yaml](https://github.com/drakkan/sftpgo/tree/master/api/schema/openapi.yaml "OpenAPI 3 specs"). 
+The OpenAPI 3 schema for the exposed API can be found inside the source tree: [openapi.yaml](https://github.com/drakkan/sftpgo/tree/master/api/schema/openapi.yaml "OpenAPI 3 specs").
 
 ## Logs
 
@@ -201,7 +213,7 @@ The logs can be divided into the following categories:
     - `file_path` string
     - `target_path` string
     - `connection_id` string. Unique SFTP connection identifier
-- **"http logs"**, REST API logs:    
+- **"http logs"**, REST API logs:
     - `sender` string. `httpd`
     - `level` string
     - `remote_addr` string. IP and port of the remote client
@@ -213,7 +225,7 @@ The logs can be divided into the following categories:
     - `resp_size` integer. Size in bytes of the HTTP response
     - `elapsed_ms` int64. Elapsed time, as milliseconds, to complete the request
     - `request_id` string. Unique request identifier
-    
+
 ## Acknowledgements
 
 - [pkg/sftp](https://github.com/pkg/sftp)

main.go

@@ -36,9 +36,8 @@ func main() {
 		logCompress   bool
 		logVerbose    bool
 	)
-	flag.StringVar(&configDir, "config-dir", ".", "Location for SFTPGo config dir. It must contain sftpgo.conf, "+
+	flag.StringVar(&configDir, "config-dir", ".", "Location for SFTPGo config dir. It must contain sftpgo.conf "+
-		"the private key for the SFTP server (id_rsa file) and the SQLite database if you use SQLite as data provider. "+
+		"and is used as the base for files with a relative path (eg. the private keys for the SFTP server, the SQLite database if you use SQLite as data provider).")
-		"The server private key will be autogenerated if the user that executes SFTPGo has write access to the config-dir")
 	flag.StringVar(&logFilePath, "log-file-path", "sftpgo.log", "Location for the log file")
 	flag.IntVar(&logMaxSize, "log-max-size", 10, "Maximum size in megabytes of the log file before it gets rotated.")
 	flag.IntVar(&logMaxBackups, "log-max-backups", 5, "Maximum number of old log files to retain")

sftpd/server.go

@@ -25,6 +25,8 @@ import (
 	"golang.org/x/crypto/ssh"
 )
 
+const defaultPrivateKeyName = "id_rsa"
+
 // Configuration for the SFTP server
 type Configuration struct {
 	// Identification string used by the server
@@ -43,6 +45,14 @@ type Configuration struct {
 	Umask string `json:"umask"`
 	// Actions to execute on SFTP create, download, delete and rename
 	Actions Actions `json:"actions"`
+	// Keys are a list of host keys
+	Keys []Key `json:"keys"`
+}
+
+// Struct containing information about host keys
+type Key struct {
+	// The private key
+	PrivateKey string `json:"private_key"`
 }
 
 // Initialize the SFTP server and add a persistent listener to handle inbound SFTP connections.
@@ -76,28 +86,41 @@ func (c Configuration) Initialize(configDir string) error {
 		ServerVersion: "SSH-2.0-" + c.Banner,
 	}
 
-	if _, err := os.Stat(filepath.Join(configDir, "id_rsa")); os.IsNotExist(err) {
+	if len(c.Keys) == 0 {
-		logger.Info(logSender, "creating new private key for server")
+		autoFile := filepath.Join(configDir, defaultPrivateKeyName)
-		logger.InfoToConsole("id_rsa does not exist, creating new private key for server")
+		if _, err := os.Stat(autoFile); os.IsNotExist(err) {
-		if err := c.generatePrivateKey(configDir); err != nil {
+			logger.Info(logSender, "No host keys configured and %s does not exist; creating new private key for server", autoFile)
+			logger.InfoToConsole("No host keys configured and %s does not exist; creating new private key for server", autoFile)
+			if err := c.generatePrivateKey(autoFile); err != nil {
+				return err
+			}
+		} else if err != nil {
 			return err
 		}
-	} else if err != nil {
+
-		return err
+		c.Keys = append(c.Keys, Key{PrivateKey: defaultPrivateKeyName})
 	}
 
-	privateBytes, err := ioutil.ReadFile(filepath.Join(configDir, "id_rsa"))
+	for _, k := range c.Keys {
-	if err != nil {
+		privateFile := k.PrivateKey
-		return err
+		if !filepath.IsAbs(privateFile) {
-	}
+			privateFile = filepath.Join(configDir, privateFile)
+		}
+		logger.Info(logSender, "Loading private key: %s", privateFile)
 
-	private, err := ssh.ParsePrivateKey(privateBytes)
+		privateBytes, err := ioutil.ReadFile(privateFile)
-	if err != nil {
+		if err != nil {
-		return err
+			return err
-	}
+		}
 
-	// Add our private key to the server configuration.
+		private, err := ssh.ParsePrivateKey(privateBytes)
-	serverConfig.AddHostKey(private)
+		if err != nil {
+			return err
+		}
+
+		// Add our private key to the server configuration.
+		serverConfig.AddHostKey(private)
+	}
 
 	listener, err := net.Listen("tcp", fmt.Sprintf("%s:%d", c.BindAddress, c.BindPort))
 	if err != nil {
@@ -273,13 +296,13 @@ func (c Configuration) validatePasswordCredentials(conn ssh.ConnMetadata, pass [
 }
 
 // Generates a private key that will be used by the SFTP server.
-func (c Configuration) generatePrivateKey(configDir string) error {
+func (c Configuration) generatePrivateKey(file string) error {
 	key, err := rsa.GenerateKey(rand.Reader, 4096)
 	if err != nil {
 		return err
 	}
 
-	o, err := os.OpenFile(filepath.Join(configDir, "id_rsa"), os.O_RDWR|os.O_CREATE|os.O_TRUNC, 0600)
+	o, err := os.OpenFile(file, os.O_RDWR|os.O_CREATE|os.O_TRUNC, 0600)
 	if err != nil {
 		return err
 	}

sftpgo.conf

@@ -1,32 +1,32 @@
 {
-   "sftpd":{
+  "sftpd": {
-        "bind_port":2022,
+    "bind_port": 2022,
-        "bind_address":"",
+    "bind_address": "",
-        "idle_timeout":15,
+    "idle_timeout": 15,
-        "max_auth_tries":0,
+    "max_auth_tries": 0,
-        "umask":"0022",
+    "umask": "0022",
-        "banner":"SFTPGo",
+    "banner": "SFTPGo",
-        "actions":{
+    "actions": {
-            "execute_on":[],
+      "execute_on": [],
-            "command":"",
+      "command": "",
-            "http_notification_url":""
+      "http_notification_url": ""
-        }
-   },
-   "data_provider":{
-        "driver":"sqlite",
-        "name":"sftpgo.db",
-        "host":"",
-        "port":5432,
-        "username":"",
-        "password":"",
-        "sslmode":0,
-        "connection_string":"",
-        "users_table":"users",
-        "manage_users":1,
-        "track_quota":2
-    },
-    "httpd":{
-        "bind_port":8080,
-        "bind_address":"127.0.0.1"
     }
+  },
+  "data_provider": {
+    "driver": "sqlite",
+    "name": "sftpgo.db",
+    "host": "",
+    "port": 5432,
+    "username": "",
+    "password": "",
+    "sslmode": 0,
+    "connection_string": "",
+    "users_table": "users",
+    "manage_users": 1,
+    "track_quota": 2
+  },
+  "httpd": {
+    "bind_port": 8080,
+    "bind_address": "127.0.0.1"
+  }
 }