diff --git a/docs/full-configuration.md b/docs/full-configuration.md index 2cf04f00..600aacce 100644 --- a/docs/full-configuration.md +++ b/docs/full-configuration.md @@ -179,7 +179,7 @@ The configuration file contains the following sections: - `hash_support`, integer. Set to `1` to enable FTP commands that allow to calculate the hash value of files. These FTP commands will be enabled: `HASH`, `XCRC`, `MD5/XMD5`, `XSHA/XSHA1`, `XSHA256`, `XSHA512`. Please keep in mind that to calculate the hash we need to read the whole file, for remote backends this means downloading the file, for the encrypted backend this means decrypting the file. Default `0`. - `combine_support`, integer. Set to 1 to enable support for the non standard `COMB` FTP command. Combine is only supported for local filesystem, for cloud backends it has no advantage as it will download the partial files and will upload the combined one. Cloud backends natively support multipart uploads. Default `0`. - `certificate_file`, string. Certificate for FTPS. This can be an absolute path or a path relative to the config dir. - - `certificate_key_file`, string. Private key matching the above certificate. This can be an absolute path or a path relative to the config dir. A certificate and the private key are required to enable explicit and implicit TLS. Certificate and key files can be reloaded on demand sending a `SIGHUP` signal on Unix based systems and a `paramchange` request to the running service on Windows. If the integrated ACME protocol is disabled and therefore the certificates are not automatically renewed and reloaded, the certificates are polled for changes every 8 hours. + - `certificate_key_file`, string. Private key matching the above certificate. This can be an absolute path or a path relative to the config dir. A certificate and the private key are required to enable explicit and implicit TLS. Certificate and key files can be reloaded on demand sending a `SIGHUP` signal on Unix based systems and a `paramchange` request to the running service on Windows. The certificates are also polled for changes every 8 hours. - `ca_certificates`, list of strings. Set of root certificate authorities to be used to verify client certificates. - `ca_revocation_lists`, list of strings. Set a revocation lists, one for each root CA, to be used to check if a client certificate has been revoked. The revocation lists can be reloaded on demand sending a `SIGHUP` signal on Unix based systems and a `paramchange` request to the running service on Windows. @@ -204,7 +204,7 @@ The configuration file contains the following sections: - `certificate_file`, string. Certificate for WebDAV over HTTPS. This can be an absolute path or a path relative to the config dir. - `certificate_key_file`, string. Private key matching the above certificate. This can be an absolute path or a path relative to the config dir. A certificate and a private key are required to enable HTTPS connections. Certificate and key files can be reloaded on demand sending a `SIGHUP` signal on Unix based systems and a `paramchange` request to the running service on Windows. - `ca_certificates`, list of strings. Set of root certificate authorities to be used to verify client certificates. - - `ca_revocation_lists`, list of strings. Set a revocation lists, one for each root CA, to be used to check if a client certificate has been revoked. The revocation lists can be reloaded on demand sending a `SIGHUP` signal on Unix based systems and a `paramchange` request to the running service on Windows. If the integrated ACME protocol is disabled and therefore the certificates are not automatically renewed and reloaded, the certificates are polled for changes every 8 hours. + - `ca_revocation_lists`, list of strings. Set a revocation lists, one for each root CA, to be used to check if a client certificate has been revoked. The revocation lists can be reloaded on demand sending a `SIGHUP` signal on Unix based systems and a `paramchange` request to the running service on Windows. The certificates are also polled for changes every 8 hours. - `cors` struct containing CORS configuration. SFTPGo uses [Go CORS handler](https://github.com/rs/cors), please refer to upstream documentation for fields meaning and their default values. - `enabled`, boolean, set to true to enable CORS. - `allowed_origins`, list of strings. @@ -356,7 +356,7 @@ The configuration file contains the following sections: - `openapi_path`, string. Path to the directory that contains the OpenAPI schema and the default renderer. This can be an absolute path or a path relative to the config dir. If empty the OpenAPI schema and the renderer will not be served regardless of the `render_openapi` directive - `web_root`, string. Defines a base URL for the web admin and client interfaces. If empty web admin and client resources will be available at the root ("/") URI. If defined it must be an absolute URI or it will be ignored - `certificate_file`, string. Certificate for HTTPS. This can be an absolute path or a path relative to the config dir. - - `certificate_key_file`, string. Private key matching the above certificate. This can be an absolute path or a path relative to the config dir. If both the certificate and the private key are provided, you can enable HTTPS for the configured bindings. Certificate and key files can be reloaded on demand sending a `SIGHUP` signal on Unix based systems and a `paramchange` request to the running service on Windows. If the integrated ACME protocol is disabled and therefore the certificates are not automatically renewed and reloaded, the certificates are polled for changes every 8 hours. + - `certificate_key_file`, string. Private key matching the above certificate. This can be an absolute path or a path relative to the config dir. If both the certificate and the private key are provided, you can enable HTTPS for the configured bindings. Certificate and key files can be reloaded on demand sending a `SIGHUP` signal on Unix based systems and a `paramchange` request to the running service on Windows. The certificates are also polled for changes every 8 hours. - `ca_certificates`, list of strings. Set of root certificate authorities to be used to verify client certificates. - `ca_revocation_lists`, list of strings. Set a revocation lists, one for each root CA, to be used to check if a client certificate has been revoked. The revocation lists can be reloaded on demand sending a `SIGHUP` signal on Unix based systems and a `paramchange` request to the running service on Windows. - `signing_passphrase`, string. Passphrase to use to derive the signing key for JWT and CSRF tokens. If empty a random signing key will be generated each time SFTPGo starts. If you set a signing passphrase you should consider rotating it periodically for added security. diff --git a/docs/logs.md b/docs/logs.md index 136c3620..a1b82080 100644 --- a/docs/logs.md +++ b/docs/logs.md @@ -8,6 +8,7 @@ The logs can be divided into the following categories: - `sender` string. This is generally the package name that emits the log - `time` string. Date/time with millisecond precision - `level` string + - `connection_id`, string, optional - `message` string - **"transfer logs"**, SFTP/SCP transfer logs: - `sender` string. `Upload` or `Download` @@ -20,7 +21,7 @@ The logs can be divided into the following categories: - `username`, string - `file_path` string - `connection_id` string. Unique connection identifier - - `protocol` string. `SFTP`, `SCP`, `SSH`, `FTP`, `HTTP`, `DAV`, `DataRetention`, `EventAction` + - `protocol` string. `SFTP`, `SCP`, `SSH`, `FTP`, `HTTP`, `HTTPShare`, `DAV`, `DataRetention`, `EventAction` - `ftp_mode`, string. `active` or `passive`. Included only for `FTP` protocol - **"command logs"**, SFTP/SCP command logs: - `sender` string. `Rename`, `Rmdir`, `Mkdir`, `Symlink`, `Remove`, `Chmod`, `Chown`, `Chtimes`, `Truncate`, `Copy`, `SSHCommand` @@ -43,10 +44,12 @@ The logs can be divided into the following categories: - **"http logs"**, REST API logs: - `sender` string. `httpd` - `level` string + - `time` string. Date/time with millisecond precision - `local_addr` string. IP/port of the local address the connection arrived on. For example `127.0.0.1:1234` - `remote_addr` string. IP and, optionally, port of the remote client. For example `127.0.0.1:1234` or `127.0.0.1` - `proto` string, for example `HTTP/1.1` - `method` string. HTTP method (`GET`, `POST`, `PUT`, `DELETE` etc.) + - `request_id` string. Omitted in telemetry logs - `user_agent` string - `uri` string. Full uri - `resp_status` integer. HTTP response status code @@ -56,6 +59,7 @@ The logs can be divided into the following categories: - **"connection failed logs"**, logs for failed attempts to initialize a connection. A connection can fail for an authentication error or other errors such as a client abort or a timeout if the login does not happen in two minutes - `sender` string. `connection_failed` - `level` string + - `time` string. Date/time with millisecond precision - `username`, string. Can be empty if the connection is closed before an authentication attempt - `client_ip` string. - `protocol` string. Possible values are `SSH`, `FTP`, `DAV` diff --git a/go.mod b/go.mod index 9f107ecd..e74ad82e 100644 --- a/go.mod +++ b/go.mod @@ -9,14 +9,14 @@ require ( github.com/GehirnInc/crypt v0.0.0-20200316065508-bb7000b8a962 github.com/alexedwards/argon2id v0.0.0-20211130144151-3585854a6387 github.com/aws/aws-sdk-go-v2 v1.17.5 - github.com/aws/aws-sdk-go-v2/config v1.18.14 - github.com/aws/aws-sdk-go-v2/credentials v1.13.14 + github.com/aws/aws-sdk-go-v2/config v1.18.15 + github.com/aws/aws-sdk-go-v2/credentials v1.13.15 github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.12.23 - github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.11.54 - github.com/aws/aws-sdk-go-v2/service/marketplacemetering v1.14.4 - github.com/aws/aws-sdk-go-v2/service/s3 v1.30.4 - github.com/aws/aws-sdk-go-v2/service/secretsmanager v1.18.5 - github.com/aws/aws-sdk-go-v2/service/sts v1.18.4 + github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.11.55 + github.com/aws/aws-sdk-go-v2/service/marketplacemetering v1.14.5 + github.com/aws/aws-sdk-go-v2/service/s3 v1.30.5 + github.com/aws/aws-sdk-go-v2/service/secretsmanager v1.18.6 + github.com/aws/aws-sdk-go-v2/service/sts v1.18.5 github.com/bmatcuk/doublestar/v4 v4.6.0 github.com/cockroachdb/cockroach-go/v2 v2.2.20 github.com/coreos/go-oidc/v3 v3.5.0 @@ -54,7 +54,7 @@ require ( github.com/rs/zerolog v1.29.0 github.com/sftpgo/sdk v0.1.3-0.20230213182959-2d89540f8810 github.com/shirou/gopsutil/v3 v3.23.1 - github.com/spf13/afero v1.9.3 + github.com/spf13/afero v1.9.4 github.com/spf13/cobra v1.6.1 github.com/spf13/viper v1.15.0 github.com/stretchr/testify v1.8.1 @@ -93,8 +93,8 @@ require ( github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.1.24 // indirect github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.23 // indirect github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.13.23 // indirect - github.com/aws/aws-sdk-go-v2/service/sso v1.12.3 // indirect - github.com/aws/aws-sdk-go-v2/service/ssooidc v1.14.3 // indirect + github.com/aws/aws-sdk-go-v2/service/sso v1.12.4 // indirect + github.com/aws/aws-sdk-go-v2/service/ssooidc v1.14.4 // indirect github.com/aws/smithy-go v1.13.5 // indirect github.com/beorn7/perks v1.0.1 // indirect github.com/boombuler/barcode v1.0.1 // indirect @@ -157,7 +157,7 @@ require ( golang.org/x/tools v0.6.0 // indirect golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 // indirect google.golang.org/appengine v1.6.7 // indirect - google.golang.org/genproto v0.0.0-20230221151758-ace64dc21148 // indirect + google.golang.org/genproto v0.0.0-20230222225845-10f96fb3dbec // indirect google.golang.org/grpc v1.53.0 // indirect google.golang.org/protobuf v1.28.1 // indirect gopkg.in/ini.v1 v1.67.0 // indirect @@ -166,6 +166,6 @@ require ( replace ( github.com/jlaffaye/ftp => github.com/drakkan/ftp v0.0.0-20201114075148-9b9adce499a9 - github.com/robfig/cron/v3 => github.com/drakkan/cron/v3 v3.0.0-20230221091953-db24a8bf7e83 + github.com/robfig/cron/v3 => github.com/drakkan/cron/v3 v3.0.0-20230222140221-217a1e4d96c0 golang.org/x/crypto => github.com/drakkan/crypto v0.0.0-20230209112458-e15d12511558 ) diff --git a/go.sum b/go.sum index e93d9fdf..fe3f99f0 100644 --- a/go.sum +++ b/go.sum @@ -539,17 +539,17 @@ github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.4.9/go.mod h1:vCmV1q1VK github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.4.10 h1:dK82zF6kkPeCo8J1e+tGx4JdvDIQzj7ygIoLg8WMuGs= github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.4.10/go.mod h1:VeTZetY5KRJLuD/7fkQXMU6Mw7H5m/KP2J5Iy9osMno= github.com/aws/aws-sdk-go-v2/config v1.18.3/go.mod h1:BYdrbeCse3ZnOD5+2/VE/nATOK8fEUpBtmPMdKSyhMU= -github.com/aws/aws-sdk-go-v2/config v1.18.14 h1:rI47jCe0EzuJlAO5ptREe3LIBAyP5c7gR3wjyYVjuOM= -github.com/aws/aws-sdk-go-v2/config v1.18.14/go.mod h1:0pI6JQBHKwd0JnwAZS3VCapLKMO++UL2BOkWwyyzTnA= +github.com/aws/aws-sdk-go-v2/config v1.18.15 h1:509yMO0pJUGUugBP2H9FOFyV+7Mz7sRR+snfDN5W4NY= +github.com/aws/aws-sdk-go-v2/config v1.18.15/go.mod h1:vS0tddZqpE8cD9CyW0/kITHF5Bq2QasW9Y1DFHD//O0= github.com/aws/aws-sdk-go-v2/credentials v1.13.3/go.mod h1:/rOMmqYBcFfNbRPU0iN9IgGqD5+V2yp3iWNmIlz0wI4= -github.com/aws/aws-sdk-go-v2/credentials v1.13.14 h1:jE34fUepssrhmYpvPpdbd+d39PHpuignDpNPNJguP60= -github.com/aws/aws-sdk-go-v2/credentials v1.13.14/go.mod h1:85ckagDuzdIOnZRwws1eLKnymJs3ZM1QwVC1XcuNGOY= +github.com/aws/aws-sdk-go-v2/credentials v1.13.15 h1:0rZQIi6deJFjOEgHI9HI2eZcLPPEGQPictX66oRFLL8= +github.com/aws/aws-sdk-go-v2/credentials v1.13.15/go.mod h1:vRMLMD3/rXU+o6j2MW5YefrGMBmdTvkLLGqFwMLBHQc= github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.12.19/go.mod h1:VihW95zQpeKQWVPGkwT+2+WJNQV8UXFfMTWdU6VErL8= github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.12.23 h1:Kbiv9PGnQfG/imNI4L/heyUXvzKmcWSBeDvkrQz5pFc= github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.12.23/go.mod h1:mOtmAg65GT1HIL/HT/PynwPbS+UG0BgCZ6vhkPqnxWo= github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.11.42/go.mod h1:LHOsygMiW/14CkFxdXxvzKyMh3jbk/QfZVaDtCbLkl8= -github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.11.54 h1:u4Cyifho7bnp6NeTCS8zAuxqzycHla4PSJvwXlU8ELI= -github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.11.54/go.mod h1:a8gjZYNkBoxPMaA4mIoBT1M+4rOAcJUgFeaxVopMS+k= +github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.11.55 h1:ClZKHmu2QIRQCEQ2Y2upfu4JPO0pG69Ce5eiq3PS2V4= +github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.11.55/go.mod h1:L/h5B6I7reig2QJXCGY0e0NVx4hYCcjETmsfR02hFng= github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.25/go.mod h1:Zb29PYkf42vVYQY6pvSyJCJcFHlPIiY+YKdPtwnvMkY= github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.29 h1:9/aKwwus0TQxppPXFmf010DFrE+ssSbzroLVYINA+xE= github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.29/go.mod h1:Dip3sIGv485+xerzVv24emnjX5Sg88utCL8fwGmCeWg= @@ -575,26 +575,26 @@ github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.13.19/go.mod h1:BmQWRV github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.13.23 h1:qc+RW0WWZ2KApMnsu/EVCPqLTyIH55uc7YQq7mq4XqE= github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.13.23/go.mod h1:FJhZWVWBCcgAF8jbep7pxQ1QUsjzTwa9tvEXGw2TDRo= github.com/aws/aws-sdk-go-v2/service/kms v1.19.0/go.mod h1:kZodDPTQjSH/qM6/OvyTfM5mms5JHB/EKYp5dhn/vI4= -github.com/aws/aws-sdk-go-v2/service/marketplacemetering v1.14.4 h1:gYNfHRTtnKPH7yeYqG7SF5hMnWhJy3EAP0QMekYo0K4= -github.com/aws/aws-sdk-go-v2/service/marketplacemetering v1.14.4/go.mod h1:Nlw/9tgFims+/X+xwFLy/EG6E+NYkZKFXDtLmKJNDA0= +github.com/aws/aws-sdk-go-v2/service/marketplacemetering v1.14.5 h1:L5uD73sZtrTDxn/WTv0LEL00NHCDZmbMUItKHrSdHFs= +github.com/aws/aws-sdk-go-v2/service/marketplacemetering v1.14.5/go.mod h1:Nlw/9tgFims+/X+xwFLy/EG6E+NYkZKFXDtLmKJNDA0= github.com/aws/aws-sdk-go-v2/service/s3 v1.29.4/go.mod h1:/NHbqPRiwxSPVOB2Xr+StDEH+GWV/64WwnUjv4KYzV0= -github.com/aws/aws-sdk-go-v2/service/s3 v1.30.4 h1:0eeEl2lyZkZPhPCt9ggIr3PbCbvae3vfggTkeqJ4O98= -github.com/aws/aws-sdk-go-v2/service/s3 v1.30.4/go.mod h1:Dze3kNt4T+Dgb8YCfuIFSBLmE6hadKNxqfdF0Xmqz1I= +github.com/aws/aws-sdk-go-v2/service/s3 v1.30.5 h1:kFfb+NMap4R7nDvBYyABa/nw7KFMtAfygD1Hyoxh4uE= +github.com/aws/aws-sdk-go-v2/service/s3 v1.30.5/go.mod h1:Dze3kNt4T+Dgb8YCfuIFSBLmE6hadKNxqfdF0Xmqz1I= github.com/aws/aws-sdk-go-v2/service/secretsmanager v1.16.8/go.mod h1:k6CPuxyzO247nYEM1baEwHH1kRtosRCvgahAepaaShw= -github.com/aws/aws-sdk-go-v2/service/secretsmanager v1.18.5 h1:8J8gcY1Nepvta2YpZJO7deIOmFAZngHWh8+ULVsfklk= -github.com/aws/aws-sdk-go-v2/service/secretsmanager v1.18.5/go.mod h1:CJcdJtrO6ulXfI8l2DotKWmJShhXHCEcd9Wibyx3kC0= +github.com/aws/aws-sdk-go-v2/service/secretsmanager v1.18.6 h1:VjvQw/1Qf/rhDSl+NNOeybSpdPRjBfH60//5vzveVsY= +github.com/aws/aws-sdk-go-v2/service/secretsmanager v1.18.6/go.mod h1:CJcdJtrO6ulXfI8l2DotKWmJShhXHCEcd9Wibyx3kC0= github.com/aws/aws-sdk-go-v2/service/sns v1.18.6/go.mod h1:2cPUjR63iE9MPMPJtSyzYmsTFCNrN/Xi9j0v9BL5OU0= github.com/aws/aws-sdk-go-v2/service/sqs v1.19.15/go.mod h1:DKX/7/ZiAzHO6p6AhArnGdrV4r+d461weby8KeVtvC4= github.com/aws/aws-sdk-go-v2/service/ssm v1.33.1/go.mod h1:rEsqsZrOp9YvSGPOrcL3pR9+i/QJaWRkAYbuxMa7yCU= github.com/aws/aws-sdk-go-v2/service/sso v1.11.25/go.mod h1:IARHuzTXmj1C0KS35vboR0FeJ89OkEy1M9mWbK2ifCI= -github.com/aws/aws-sdk-go-v2/service/sso v1.12.3 h1:bUeZTWfF1vBdZnoNnnq70rB/CzdZD7NR2Jg2Ax+rvjA= -github.com/aws/aws-sdk-go-v2/service/sso v1.12.3/go.mod h1:jtLIhd+V+lft6ktxpItycqHqiVXrPIRjWIsFIlzMriw= +github.com/aws/aws-sdk-go-v2/service/sso v1.12.4 h1:qJdM48OOLl1FBSzI7ZrA1ZfLwOyCYqkXV5lko1hYDBw= +github.com/aws/aws-sdk-go-v2/service/sso v1.12.4/go.mod h1:jtLIhd+V+lft6ktxpItycqHqiVXrPIRjWIsFIlzMriw= github.com/aws/aws-sdk-go-v2/service/ssooidc v1.13.8/go.mod h1:er2JHN+kBY6FcMfcBBKNGCT3CarImmdFzishsqBmSRI= -github.com/aws/aws-sdk-go-v2/service/ssooidc v1.14.3 h1:G/+7NUi+q+H0LG3v32jfV4OkaQIcpI92g0owbXKk6NY= -github.com/aws/aws-sdk-go-v2/service/ssooidc v1.14.3/go.mod h1:zVwRrfdSmbRZWkUkWjOItY7SOalnFnq/Yg2LVPqDjwc= +github.com/aws/aws-sdk-go-v2/service/ssooidc v1.14.4 h1:YRkWXQveFb0tFC0TLktmmhGsOcCgLwvq88MC2al47AA= +github.com/aws/aws-sdk-go-v2/service/ssooidc v1.14.4/go.mod h1:zVwRrfdSmbRZWkUkWjOItY7SOalnFnq/Yg2LVPqDjwc= github.com/aws/aws-sdk-go-v2/service/sts v1.17.5/go.mod h1:bXcN3koeVYiJcdDU89n3kCYILob7Y34AeLopUbZgLT4= -github.com/aws/aws-sdk-go-v2/service/sts v1.18.4 h1:j0USUNbl9c/8tBJ8setEbwxc7wva0WyoeAaFRiyTUT8= -github.com/aws/aws-sdk-go-v2/service/sts v1.18.4/go.mod h1:1mKZHLLpDMHTNSYPJ7qrcnCQdHCWsNQaT0xRvq2u80s= +github.com/aws/aws-sdk-go-v2/service/sts v1.18.5 h1:L1600eLr0YvTT7gNh3Ni24yGI7NSHkq9Gp62vijPRCs= +github.com/aws/aws-sdk-go-v2/service/sts v1.18.5/go.mod h1:1mKZHLLpDMHTNSYPJ7qrcnCQdHCWsNQaT0xRvq2u80s= github.com/aws/smithy-go v1.13.4/go.mod h1:Tg+OJXh4MB2R/uN61Ko2f6hTZwB/ZYGOtib8J3gBHzA= github.com/aws/smithy-go v1.13.5 h1:hgz0X/DX0dGqTYpGALqXJoRKRj5oQ7150i5FdTePzO8= github.com/aws/smithy-go v1.13.5/go.mod h1:Tg+OJXh4MB2R/uN61Ko2f6hTZwB/ZYGOtib8J3gBHzA= @@ -847,8 +847,8 @@ github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDD github.com/docker/libtrust v0.0.0-20150114040149-fa567046d9b1/go.mod h1:cyGadeNEkKy96OOhEzfZl+yxihPEzKnqJwvfuSUqbZE= github.com/docker/spdystream v0.0.0-20160310174837-449fdfce4d96/go.mod h1:Qh8CwZgvJUkLughtfhJv5dyTYa91l1fOUCrgjqmcifM= github.com/docopt/docopt-go v0.0.0-20180111231733-ee0de3bc6815/go.mod h1:WwZ+bS3ebgob9U8Nd0kOddGdZWjyMGR8Wziv+TBNwSE= -github.com/drakkan/cron/v3 v3.0.0-20230221091953-db24a8bf7e83 h1:PqD3xF2E/kKjrJct7giLNLK0EuvEfuVl2GqNnlQ9QLk= -github.com/drakkan/cron/v3 v3.0.0-20230221091953-db24a8bf7e83/go.mod h1:eQICP3HwyT7UooqI/z+Ov+PtYAWygg1TEWWzGIFLtro= +github.com/drakkan/cron/v3 v3.0.0-20230222140221-217a1e4d96c0 h1:EW9gIJRmt9lzk66Fhh4S8VEtURA6QHZqGeSRE9Nb2/U= +github.com/drakkan/cron/v3 v3.0.0-20230222140221-217a1e4d96c0/go.mod h1:eQICP3HwyT7UooqI/z+Ov+PtYAWygg1TEWWzGIFLtro= github.com/drakkan/crypto v0.0.0-20230209112458-e15d12511558 h1:M4nv9gf47uCKouIe/FR3RzBPZdx+vquMR1mIRSEa2uw= github.com/drakkan/crypto v0.0.0-20230209112458-e15d12511558/go.mod h1:20JIOkADKNe0e6yKflVpVinG/uP19j94rhQlU7Ea/hQ= github.com/drakkan/ftp v0.0.0-20201114075148-9b9adce499a9 h1:LPH1dEblAOO/LoG7yHPMtBLXhQmjaga91/DDjWk9jWA= @@ -1833,8 +1833,8 @@ github.com/spf13/afero v1.2.2/go.mod h1:9ZxEEn6pIJ8Rxe320qSDBk6AsU0r9pR7Q4OcevTd github.com/spf13/afero v1.3.3/go.mod h1:5KUK8ByomD5Ti5Artl0RtHeI5pTF7MIDuXL3yY520V4= github.com/spf13/afero v1.6.0/go.mod h1:Ai8FlHk4v/PARR026UzYexafAt9roJ7LcLMAmO6Z93I= github.com/spf13/afero v1.9.2/go.mod h1:iUV7ddyEEZPO5gA3zD4fJt6iStLlL+Lg4m2cihcDf8Y= -github.com/spf13/afero v1.9.3 h1:41FoI0fD7OR7mGcKE/aOiLkGreyf8ifIOQmJANWogMk= -github.com/spf13/afero v1.9.3/go.mod h1:iUV7ddyEEZPO5gA3zD4fJt6iStLlL+Lg4m2cihcDf8Y= +github.com/spf13/afero v1.9.4 h1:Sd43wM1IWz/s1aVXdOBkjJvuP8UdyqioeE4AmM0QsBs= +github.com/spf13/afero v1.9.4/go.mod h1:iUV7ddyEEZPO5gA3zD4fJt6iStLlL+Lg4m2cihcDf8Y= github.com/spf13/cast v1.3.0/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE= github.com/spf13/cast v1.3.1/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE= github.com/spf13/cast v1.5.0 h1:rj3WzYc11XZaIZMPKmwP96zkFEnnAmV8s6XbB2aY32w= @@ -2715,8 +2715,8 @@ google.golang.org/genproto v0.0.0-20221109142239-94d6d90a7d66/go.mod h1:rZS5c/ZV google.golang.org/genproto v0.0.0-20221118155620-16455021b5e6/go.mod h1:rZS5c/ZVYMaOGBfO68GWtjOw/eLaZM1X6iVtgjZ+EWg= google.golang.org/genproto v0.0.0-20221201164419-0e50fba7f41c/go.mod h1:rZS5c/ZVYMaOGBfO68GWtjOw/eLaZM1X6iVtgjZ+EWg= google.golang.org/genproto v0.0.0-20221201204527-e3fa12d562f3/go.mod h1:rZS5c/ZVYMaOGBfO68GWtjOw/eLaZM1X6iVtgjZ+EWg= -google.golang.org/genproto v0.0.0-20230221151758-ace64dc21148 h1:muK+gVBJBfFb4SejshDBlN2/UgxCCOKH9Y34ljqEGOc= -google.golang.org/genproto v0.0.0-20230221151758-ace64dc21148/go.mod h1:3Dl5ZL0q0isWJt+FVcfpQyirqemEuLAK/iFvg1UP1Hw= +google.golang.org/genproto v0.0.0-20230222225845-10f96fb3dbec h1:6rwgChOSUfpzJF2/KnLgo+gMaxGpujStSkPWrbhXArU= +google.golang.org/genproto v0.0.0-20230222225845-10f96fb3dbec/go.mod h1:3Dl5ZL0q0isWJt+FVcfpQyirqemEuLAK/iFvg1UP1Hw= google.golang.org/grpc v0.0.0-20160317175043-d3ddb4469d5a/go.mod h1:yo6s7OP7yaDglbqo1J04qKzAhqBH6lvTonzMVmEdcZw= google.golang.org/grpc v1.17.0/go.mod h1:6QZJwpn2B+Zp71q/5VxRsJ6NXXVCE5NRUHRo+f3cWCs= google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c= diff --git a/internal/acme/acme.go b/internal/acme/acme.go index 2ec72171..c25906de 100644 --- a/internal/acme/acme.go +++ b/internal/acme/acme.go @@ -46,6 +46,7 @@ import ( "github.com/robfig/cron/v3" "github.com/drakkan/sftpgo/v2/internal/common" + "github.com/drakkan/sftpgo/v2/internal/dataprovider" "github.com/drakkan/sftpgo/v2/internal/ftpd" "github.com/drakkan/sftpgo/v2/internal/httpd" "github.com/drakkan/sftpgo/v2/internal/logger" @@ -60,12 +61,24 @@ const ( ) var ( - config *Configuration - scheduler *cron.Cron - logMode int + config *Configuration + initialConfig Configuration + scheduler *cron.Cron + logMode int + supportedKeyTypes = []string{ + string(certcrypto.EC256), + string(certcrypto.EC384), + string(certcrypto.RSA2048), + string(certcrypto.RSA4096), + string(certcrypto.RSA8192), + } ) -// GetCertificates tries to obtain the certificates for the configured domains +func init() { + httpd.SetCertificatesGetter(getCertificatesForConfig) +} + +// GetCertificates tries to obtain the certificates using the global configuration func GetCertificates() error { if config == nil { return errors.New("acme is disabled") @@ -73,6 +86,78 @@ func GetCertificates() error { return config.getCertificates() } +// getCertificatesForConfig tries to obtain the certificates using the provided +// configuration override. This is a NOOP if we already have certificates +func getCertificatesForConfig(c *dataprovider.ACMEConfigs, configDir string) error { + if c.Domain == "" { + acmeLog(logger.LevelDebug, "no domain configured, nothing to do") + return nil + } + config := mergeConfig(getConfiguration(), c) + if err := config.Initialize(configDir); err != nil { + return err + } + hasCerts, err := config.hasCertificates(c.Domain) + if err != nil { + return fmt.Errorf("unable to check if we already have certificates for domain %q: %w", c.Domain, err) + } + if hasCerts { + return nil + } + return config.getCertificates() +} + +func mergeConfig(config Configuration, c *dataprovider.ACMEConfigs) Configuration { + config.Domains = []string{c.Domain} + config.Email = c.Email + config.HTTP01Challenge.Port = c.HTTP01Challenge.Port + config.TLSALPN01Challenge.Port = 0 + return config +} + +// getConfiguration returns the configuration set using config file and env vars +func getConfiguration() Configuration { + return initialConfig +} + +func loadProviderConf(c Configuration) (Configuration, error) { + configs, err := dataprovider.GetConfigs() + if err != nil { + return c, fmt.Errorf("unable to load config from provider: %w", err) + } + configs.SetNilsToEmpty() + if configs.ACME.Domain == "" { + return c, nil + } + return mergeConfig(c, configs.ACME), nil +} + +// Initialize validates and set the configuration +func Initialize(c Configuration, configDir string, checkRenew bool) error { + config = nil + initialConfig = c + c, err := loadProviderConf(c) + if err != nil { + return err + } + util.CertsBasePath = "" + setLogMode(checkRenew) + + if err := c.Initialize(configDir); err != nil { + return err + } + if len(c.Domains) == 0 { + return nil + } + util.CertsBasePath = c.CertsPath + acmeLog(logger.LevelInfo, "configured domains: %+v, certs base path %q", c.Domains, c.CertsPath) + config = &c + if checkRenew { + return startScheduler() + } + return nil +} + // HTTP01Challenge defines the configuration for HTTP-01 challenge type type HTTP01Challenge struct { Port int `json:"port" mapstructure:"port"` @@ -141,72 +226,52 @@ type Configuration struct { tempDir string } -// Initialize validates and set the configuration -func (c *Configuration) Initialize(configDir string, checkRenew bool) error { - common.SetCertAutoReloadMode(true) - config = nil - setLogMode(checkRenew) +// Initialize validates and initialize the configuration +func (c *Configuration) Initialize(configDir string) error { c.checkDomains() if len(c.Domains) == 0 { acmeLog(logger.LevelInfo, "no domains configured, acme disabled") return nil } if c.Email == "" || !util.IsEmailValid(c.Email) { - return fmt.Errorf("invalid email address %#v", c.Email) + return fmt.Errorf("invalid email address %q", c.Email) } if c.RenewDays < 1 { return fmt.Errorf("invalid number of days remaining before renewal: %d", c.RenewDays) } - supportedKeyTypes := []string{ - string(certcrypto.EC256), - string(certcrypto.EC384), - string(certcrypto.RSA2048), - string(certcrypto.RSA4096), - string(certcrypto.RSA8192), - } if !util.Contains(supportedKeyTypes, c.KeyType) { - return fmt.Errorf("invalid key type %#v", c.KeyType) + return fmt.Errorf("invalid key type %q", c.KeyType) } caURL, err := url.Parse(c.CAEndpoint) if err != nil { return fmt.Errorf("invalid CA endopoint: %w", err) } if !util.IsFileInputValid(c.CertsPath) { - return fmt.Errorf("invalid certs path %#v", c.CertsPath) + return fmt.Errorf("invalid certs path %q", c.CertsPath) } if !filepath.IsAbs(c.CertsPath) { c.CertsPath = filepath.Join(configDir, c.CertsPath) } err = os.MkdirAll(c.CertsPath, 0700) if err != nil { - return fmt.Errorf("unable to create certs path %#v: %w", c.CertsPath, err) + return fmt.Errorf("unable to create certs path %q: %w", c.CertsPath, err) } c.tempDir = filepath.Join(c.CertsPath, "temp") err = os.MkdirAll(c.CertsPath, 0700) if err != nil { - return fmt.Errorf("unable to create certs temp path %#v: %w", c.tempDir, err) + return fmt.Errorf("unable to create certs temp path %q: %w", c.tempDir, err) } serverPath := strings.NewReplacer(":", "_", "/", string(os.PathSeparator)).Replace(caURL.Host) accountPath := filepath.Join(c.CertsPath, serverPath) err = os.MkdirAll(accountPath, 0700) if err != nil { - return fmt.Errorf("unable to create account path %#v: %w", accountPath, err) + return fmt.Errorf("unable to create account path %q: %w", accountPath, err) } c.accountConfigPath = filepath.Join(accountPath, c.Email+".json") c.accountKeyPath = filepath.Join(accountPath, c.Email+".key") c.lockPath = filepath.Join(c.CertsPath, "lock") - if err = c.validateChallenges(); err != nil { - return err - } - - acmeLog(logger.LevelInfo, "configured domains: %+v", c.Domains) - common.SetCertAutoReloadMode(false) - config = c - if checkRenew { - return startScheduler() - } - return nil + return c.validateChallenges() } func (c *Configuration) validateChallenges() error { @@ -240,10 +305,10 @@ func (c *Configuration) setLockTime() error { lockTime := fmt.Sprintf("%v", util.GetTimeAsMsSinceEpoch(time.Now())) err := os.WriteFile(c.lockPath, []byte(lockTime), 0600) if err != nil { - acmeLog(logger.LevelError, "unable to save lock time to %#v: %v", c.lockPath, err) + acmeLog(logger.LevelError, "unable to save lock time to %q: %v", c.lockPath, err) return fmt.Errorf("unable to save lock time: %w", err) } - acmeLog(logger.LevelDebug, "lock time saved: %#v", lockTime) + acmeLog(logger.LevelDebug, "lock time saved: %q", lockTime) return nil } @@ -251,10 +316,10 @@ func (c *Configuration) getLockTime() (time.Time, error) { content, err := os.ReadFile(c.lockPath) if err != nil { if os.IsNotExist(err) { - acmeLog(logger.LevelDebug, "lock file %#v not found", c.lockPath) + acmeLog(logger.LevelDebug, "lock file %q not found", c.lockPath) return time.Time{}, nil } - acmeLog(logger.LevelError, "unable to read lock file %#v: %v", c.lockPath, err) + acmeLog(logger.LevelError, "unable to read lock file %q: %v", c.lockPath, err) return time.Time{}, err } msec, err := strconv.ParseInt(strings.TrimSpace(string(content)), 10, 64) @@ -272,7 +337,7 @@ func (c *Configuration) saveAccount(account *account) error { } err = os.WriteFile(c.accountConfigPath, jsonBytes, 0600) if err != nil { - acmeLog(logger.LevelError, "unable to save account to file %#v: %v", c.accountConfigPath, err) + acmeLog(logger.LevelError, "unable to save account to file %q: %v", c.accountConfigPath, err) return fmt.Errorf("unable to save account: %w", err) } return nil @@ -287,7 +352,7 @@ func (c *Configuration) getAccount(privateKey crypto.PrivateKey) (account, error var account account fileBytes, err := os.ReadFile(c.accountConfigPath) if err != nil { - acmeLog(logger.LevelError, "unable to read account from file %#v: %v", c.accountConfigPath, err) + acmeLog(logger.LevelError, "unable to read account from file %q: %v", c.accountConfigPath, err) return account, fmt.Errorf("unable to read account from file: %w", err) } err = json.Unmarshal(fileBytes, &account) @@ -316,7 +381,7 @@ func (c *Configuration) getAccount(privateKey crypto.PrivateKey) (account, error func (c *Configuration) loadPrivateKey() (crypto.PrivateKey, error) { keyBytes, err := os.ReadFile(c.accountKeyPath) if err != nil { - acmeLog(logger.LevelError, "unable to read account key from file %#v: %v", c.accountKeyPath, err) + acmeLog(logger.LevelError, "unable to read account key from file %q: %v", c.accountKeyPath, err) return nil, fmt.Errorf("unable to read account key: %w", err) } @@ -329,10 +394,10 @@ func (c *Configuration) loadPrivateKey() (crypto.PrivateKey, error) { case "EC PRIVATE KEY": privateKey, err = x509.ParseECPrivateKey(keyBlock.Bytes) default: - err = fmt.Errorf("unknown private key type %#v", keyBlock.Type) + err = fmt.Errorf("unknown private key type %q", keyBlock.Type) } if err != nil { - acmeLog(logger.LevelError, "unable to parse private key from file %#v: %v", c.accountKeyPath, err) + acmeLog(logger.LevelError, "unable to parse private key from file %q: %v", c.accountKeyPath, err) return privateKey, fmt.Errorf("unable to parse private key: %w", err) } return privateKey, nil @@ -346,7 +411,7 @@ func (c *Configuration) generatePrivateKey() (crypto.PrivateKey, error) { } certOut, err := os.Create(c.accountKeyPath) if err != nil { - acmeLog(logger.LevelError, "unable to save private key to file %#v: %v", c.accountKeyPath, err) + acmeLog(logger.LevelError, "unable to save private key to file %q: %v", c.accountKeyPath, err) return nil, fmt.Errorf("unable to save private key: %w", err) } defer certOut.Close() @@ -365,25 +430,25 @@ func (c *Configuration) generatePrivateKey() (crypto.PrivateKey, error) { func (c *Configuration) getPrivateKey() (crypto.PrivateKey, error) { _, err := os.Stat(c.accountKeyPath) if err != nil && os.IsNotExist(err) { - acmeLog(logger.LevelDebug, "private key file %#v does not exist, generating new private key", c.accountKeyPath) + acmeLog(logger.LevelDebug, "private key file %q does not exist, generating new private key", c.accountKeyPath) return c.generatePrivateKey() } - acmeLog(logger.LevelDebug, "loading private key from file %#v, stat error: %v", c.accountKeyPath, err) + acmeLog(logger.LevelDebug, "loading private key from file %q, stat error: %v", c.accountKeyPath, err) return c.loadPrivateKey() } func (c *Configuration) loadCertificatesForDomain(domain string) ([]*x509.Certificate, error) { - domain = sanitizedDomain(domain) - acmeLog(logger.LevelDebug, "loading certificates for domain %#v", domain) + domain = util.SanitizeDomain(domain) + acmeLog(logger.LevelDebug, "loading certificates for domain %q", domain) content, err := os.ReadFile(filepath.Join(c.CertsPath, domain+".crt")) if err != nil { - acmeLog(logger.LevelError, "unable to load certificates for domain %#v: %v", domain, err) - return nil, fmt.Errorf("unable to load certificates for domain %#v: %w", domain, err) + acmeLog(logger.LevelError, "unable to load certificates for domain %q: %v", domain, err) + return nil, fmt.Errorf("unable to load certificates for domain %q: %w", domain, err) } certs, err := certcrypto.ParsePEMBundle(content) if err != nil { - acmeLog(logger.LevelError, "unable to parse certificates for domain %#v: %v", domain, err) - return certs, fmt.Errorf("unable to parse certificates for domain %#v: %w", domain, err) + acmeLog(logger.LevelError, "unable to parse certificates for domain %q: %v", domain, err) + return certs, fmt.Errorf("unable to parse certificates for domain %q: %w", domain, err) } return certs, nil } @@ -395,7 +460,7 @@ func (c *Configuration) needRenewal(x509Cert *x509.Certificate, domain string) b } notAfter := int(time.Until(x509Cert.NotAfter).Hours() / 24.0) if notAfter > c.RenewDays { - acmeLog(logger.LevelDebug, "the certificate for domain %#v expires in %d days, no renewal", domain, notAfter) + acmeLog(logger.LevelDebug, "the certificate for domain %q expires in %d days, no renewal", domain, notAfter) return false } return true @@ -430,10 +495,10 @@ func (c *Configuration) setupChalleges(client *lego.Client) error { client.Challenge.Remove(challenge.DNS01) if c.HTTP01Challenge.isEnabled() { if c.HTTP01Challenge.WebRoot != "" { - acmeLog(logger.LevelDebug, "configuring HTTP-01 web root challenge, path %#v", c.HTTP01Challenge.WebRoot) + acmeLog(logger.LevelDebug, "configuring HTTP-01 web root challenge, path %q", c.HTTP01Challenge.WebRoot) providerServer, err := webroot.NewHTTPProvider(c.HTTP01Challenge.WebRoot) if err != nil { - acmeLog(logger.LevelError, "unable to create HTTP-01 web root challenge provider from path %#v: %v", + acmeLog(logger.LevelError, "unable to create HTTP-01 web root challenge provider from path %q: %v", c.HTTP01Challenge.WebRoot, err) return fmt.Errorf("unable to create HTTP-01 web root challenge provider: %w", err) } @@ -490,6 +555,18 @@ func (c *Configuration) tryRecoverRegistration(privateKey crypto.PrivateKey) (*r return client.Registration.ResolveAccountByKey() } +func (c *Configuration) getCrtPath(domain string) string { + return filepath.Join(c.CertsPath, domain+".crt") +} + +func (c *Configuration) getKeyPath(domain string) string { + return filepath.Join(c.CertsPath, domain+".key") +} + +func (c *Configuration) getResourcePath(domain string) string { + return filepath.Join(c.CertsPath, domain+".json") +} + func (c *Configuration) obtainAndSaveCertificate(client *lego.Client, domain string) error { domains := getDomains(domain) acmeLog(logger.LevelInfo, "requesting certificates for domains %+v", domains) @@ -505,15 +582,15 @@ func (c *Configuration) obtainAndSaveCertificate(client *lego.Client, domain str acmeLog(logger.LevelError, "unable to obtain certificates for domains %+v: %v", domains, err) return fmt.Errorf("unable to obtain certificates: %w", err) } - domain = sanitizedDomain(domain) - err = os.WriteFile(filepath.Join(c.CertsPath, domain+".crt"), cert.Certificate, 0600) + domain = util.SanitizeDomain(domain) + err = os.WriteFile(c.getCrtPath(domain), cert.Certificate, 0600) if err != nil { - acmeLog(logger.LevelError, "unable to save certificate for domain %v: %v", domain, err) + acmeLog(logger.LevelError, "unable to save certificate for domain %s: %v", domain, err) return fmt.Errorf("unable to save certificate: %w", err) } - err = os.WriteFile(filepath.Join(c.CertsPath, domain+".key"), cert.PrivateKey, 0600) + err = os.WriteFile(c.getKeyPath(domain), cert.PrivateKey, 0600) if err != nil { - acmeLog(logger.LevelError, "unable to save private key for domain %v: %v", domain, err) + acmeLog(logger.LevelError, "unable to save private key for domain %s: %v", domain, err) return fmt.Errorf("unable to save private key: %w", err) } jsonBytes, err := json.MarshalIndent(cert, "", "\t") @@ -521,7 +598,7 @@ func (c *Configuration) obtainAndSaveCertificate(client *lego.Client, domain str acmeLog(logger.LevelError, "unable to marshal certificate resources for domain %v: %v", domain, err) return err } - err = os.WriteFile(filepath.Join(c.CertsPath, domain+".json"), jsonBytes, 0600) + err = os.WriteFile(c.getResourcePath(domain), jsonBytes, 0600) if err != nil { acmeLog(logger.LevelError, "unable to save certificate resources for domain %v: %v", domain, err) return fmt.Errorf("unable to save certificate resources: %w", err) @@ -531,6 +608,25 @@ func (c *Configuration) obtainAndSaveCertificate(client *lego.Client, domain str return nil } +// hasCertificates returns true if certificates for the specified domain has already been issued +func (c *Configuration) hasCertificates(domain string) (bool, error) { + domain = util.SanitizeDomain(domain) + if _, err := os.Stat(c.getCrtPath(domain)); err != nil { + if os.IsNotExist(err) { + return false, nil + } + return false, err + } + if _, err := os.Stat(c.getKeyPath(domain)); err != nil { + if os.IsNotExist(err) { + return false, nil + } + return false, err + } + return true, nil +} + +// getCertificates tries to obtain the certificates for the configured domains func (c *Configuration) getCertificates() error { account, client, err := c.setup() if err != nil { @@ -664,11 +760,7 @@ func getDomains(domain string) []string { domains = append(domains, d) } } - return domains -} - -func sanitizedDomain(domain string) string { - return strings.NewReplacer(":", "_", "*", "_", ",", "_", " ", "_").Replace(domain) + return util.RemoveDuplicates(domains, false) } func stopScheduler() { diff --git a/internal/cmd/acme.go b/internal/cmd/acme.go index c188c31f..5bac6186 100644 --- a/internal/cmd/acme.go +++ b/internal/cmd/acme.go @@ -22,6 +22,7 @@ import ( "github.com/drakkan/sftpgo/v2/internal/acme" "github.com/drakkan/sftpgo/v2/internal/config" + "github.com/drakkan/sftpgo/v2/internal/dataprovider" "github.com/drakkan/sftpgo/v2/internal/logger" "github.com/drakkan/sftpgo/v2/internal/util" ) @@ -49,10 +50,29 @@ renewed by the SFTPGo service logger.ErrorToConsole("Unable to initialize ACME, config load error: %v", err) return } + kmsConfig := config.GetKMSConfig() + err = kmsConfig.Initialize() + if err != nil { + logger.ErrorToConsole("unable to initialize KMS: %v", err) + os.Exit(1) + } + mfaConfig := config.GetMFAConfig() + err = mfaConfig.Initialize() + if err != nil { + logger.ErrorToConsole("Unable to initialize MFA: %v", err) + os.Exit(1) + } + providerConf := config.GetProviderConf() + err = dataprovider.Initialize(providerConf, configDir, false) + if err != nil { + logger.ErrorToConsole("error initializing data provider: %v", err) + os.Exit(1) + } acmeConfig := config.GetACMEConfig() - err = acmeConfig.Initialize(configDir, false) + err = acme.Initialize(acmeConfig, configDir, false) if err != nil { logger.ErrorToConsole("Unable to initialize ACME configuration: %v", err) + os.Exit(1) } if err = acme.GetCertificates(); err != nil { logger.ErrorToConsole("Cannot get certificates: %v", err) diff --git a/internal/common/protocol_test.go b/internal/common/protocol_test.go index 8ec9c85f..5cc07782 100644 --- a/internal/common/protocol_test.go +++ b/internal/common/protocol_test.go @@ -120,7 +120,6 @@ func TestMain(m *testing.M) { logger.WarnToConsole("error initializing common: %v", err) os.Exit(1) } - common.SetCertAutoReloadMode(true) httpConfig := config.GetHTTPConfig() httpConfig.Timeout = 5 diff --git a/internal/common/tlsutils.go b/internal/common/tlsutils.go index 13aa6c96..41f48204 100644 --- a/internal/common/tlsutils.go +++ b/internal/common/tlsutils.go @@ -38,17 +38,9 @@ const ( ) var ( - certAutoReload bool - pemCRLPrefix = []byte("-----BEGIN X509 CRL") + pemCRLPrefix = []byte("-----BEGIN X509 CRL") ) -// SetCertAutoReloadMode sets if the certificate must be monitored for changes and -// automatically reloaded -func SetCertAutoReloadMode(val bool) { - certAutoReload = val - logger.Debug(logSender, "", "is certificate monitoring enabled? %t", certAutoReload) -} - // TLSKeyPair defines the paths and the unique identifier for a TLS key pair type TLSKeyPair struct { Cert string @@ -302,11 +294,11 @@ func NewCertManager(keyPairs []TLSKeyPair, configDir, logSender string) (*CertMa if err != nil { return nil, err } - if certAutoReload { - randSecs := rand.Intn(59) - manager.monitor() - _, err := eventScheduler.AddFunc(fmt.Sprintf("@every 8h0m%ds", randSecs), manager.monitor) - util.PanicOnError(err) + randSecs := rand.Intn(59) + manager.monitor() + if eventScheduler != nil { + logger.Debug(manager.logSender, "", "starting certificates monitoring tasks") + _, err = eventScheduler.AddFunc(fmt.Sprintf("@every 8h0m%ds", randSecs), manager.monitor) } - return manager, nil + return manager, err } diff --git a/internal/dataprovider/configs.go b/internal/dataprovider/configs.go index 25d93f50..74067d61 100644 --- a/internal/dataprovider/configs.go +++ b/internal/dataprovider/configs.go @@ -218,11 +218,72 @@ func (c *SMTPConfigs) getACopy() *SMTPConfigs { } } +// ACMEHTTP01Challenge defines the configuration for HTTP-01 challenge type +type ACMEHTTP01Challenge struct { + Port int `json:"port"` +} + +// ACMEConfigs defines ACME related configuration +type ACMEConfigs struct { + Domain string `json:"domain"` + Email string `json:"email"` + HTTP01Challenge ACMEHTTP01Challenge `json:"http01_challenge"` + // apply the certificate for the specified protocols: + // + // 1 means HTTP + // 2 means FTP + // 4 means WebDAV + // + // Protocols can be combined + Protocols int `json:"protocols"` +} + +func (c *ACMEConfigs) isEmpty() bool { + return c.Domain == "" +} + +func (c *ACMEConfigs) validate() error { + if c.Domain == "" { + return nil + } + if c.Email == "" && !util.IsEmailValid(c.Email) { + return util.NewValidationError(fmt.Sprintf("acme: invalid email %q", c.Email)) + } + if c.HTTP01Challenge.Port <= 0 || c.HTTP01Challenge.Port > 65535 { + return util.NewValidationError(fmt.Sprintf("acme: invalid HTTP-01 challenge port %d", c.HTTP01Challenge.Port)) + } + return nil +} + +// HasProtocol returns true if the ACME certificate must be used for the specified protocol +func (c *ACMEConfigs) HasProtocol(protocol string) bool { + switch protocol { + case protocolHTTP: + return c.Protocols&1 != 0 + case protocolFTP: + return c.Protocols&2 != 0 + case protocolWebDAV: + return c.Protocols&4 != 0 + default: + return false + } +} + +func (c *ACMEConfigs) getACopy() *ACMEConfigs { + return &ACMEConfigs{ + Email: c.Email, + Domain: c.Domain, + HTTP01Challenge: ACMEHTTP01Challenge{Port: c.HTTP01Challenge.Port}, + Protocols: c.Protocols, + } +} + // Configs allows to set configuration keys disabled by default without // modifying the config file or setting env vars type Configs struct { SFTPD *SFTPDConfigs `json:"sftpd,omitempty"` SMTP *SMTPConfigs `json:"smtp,omitempty"` + ACME *ACMEConfigs `json:"acme,omitempty"` UpdatedAt int64 `json:"updated_at,omitempty"` } @@ -237,6 +298,11 @@ func (c *Configs) validate() error { return err } } + if c.ACME != nil { + if err := c.ACME.validate(); err != nil { + return err + } + } return nil } @@ -250,6 +316,9 @@ func (c *Configs) PrepareForRendering() { if c.SMTP != nil && c.SMTP.isEmpty() { c.SMTP = nil } + if c.ACME != nil && c.ACME.isEmpty() { + c.ACME = nil + } if c.SMTP != nil && c.SMTP.Password != nil { c.SMTP.Password.Hide() if c.SMTP.Password.IsEmpty() { @@ -269,6 +338,9 @@ func (c *Configs) SetNilsToEmpty() { if c.SMTP.Password == nil { c.SMTP.Password = kms.NewEmptySecret() } + if c.ACME == nil { + c.ACME = &ACMEConfigs{} + } } // RenderAsJSON implements the renderer interface used within plugins @@ -294,6 +366,9 @@ func (c *Configs) getACopy() Configs { if c.SMTP != nil { result.SMTP = c.SMTP.getACopy() } + if c.ACME != nil { + result.ACME = c.ACME.getACopy() + } result.UpdatedAt = c.UpdatedAt return result } diff --git a/internal/ftpd/ftpd.go b/internal/ftpd/ftpd.go index c215473a..628b72c7 100644 --- a/internal/ftpd/ftpd.go +++ b/internal/ftpd/ftpd.go @@ -19,12 +19,14 @@ import ( "errors" "fmt" "net" + "os" "path/filepath" "strings" ftpserver "github.com/fclairamb/ftpserverlib" "github.com/drakkan/sftpgo/v2/internal/common" + "github.com/drakkan/sftpgo/v2/internal/dataprovider" "github.com/drakkan/sftpgo/v2/internal/logger" "github.com/drakkan/sftpgo/v2/internal/util" ) @@ -267,6 +269,7 @@ type Configuration struct { CombineSupport int `json:"combine_support" mapstructure:"combine_support"` // Port Range for data connections. Random if not specified PassivePortRange PortRange `json:"passive_port_range" mapstructure:"passive_port_range"` + acmeDomain string } // ShouldBind returns true if there is at least a valid binding @@ -294,8 +297,13 @@ func (c *Configuration) getKeyPairs(configDir string) []common.TLSKeyPair { }) } } - certificateFile := getConfigPath(c.CertificateFile, configDir) - certificateKeyFile := getConfigPath(c.CertificateKeyFile, configDir) + var certificateFile, certificateKeyFile string + if c.acmeDomain != "" { + certificateFile, certificateKeyFile = util.GetACMECertificateKeyPair(c.acmeDomain) + } else { + certificateFile = getConfigPath(c.CertificateFile, configDir) + certificateKeyFile = getConfigPath(c.CertificateKeyFile, configDir) + } if certificateFile != "" && certificateKeyFile != "" { keyPairs = append(keyPairs, common.TLSKeyPair{ Cert: certificateFile, @@ -306,8 +314,37 @@ func (c *Configuration) getKeyPairs(configDir string) []common.TLSKeyPair { return keyPairs } +func (c *Configuration) loadFromProvider() error { + configs, err := dataprovider.GetConfigs() + if err != nil { + return fmt.Errorf("unable to load config from provider: %w", err) + } + configs.SetNilsToEmpty() + if configs.ACME.Domain == "" || !configs.ACME.HasProtocol(common.ProtocolFTP) { + return nil + } + crt, key := util.GetACMECertificateKeyPair(configs.ACME.Domain) + if crt != "" && key != "" { + if _, err := os.Stat(crt); err != nil { + logger.Error(logSender, "", "unable to load acme cert file %q: %v", crt, err) + return nil + } + if _, err := os.Stat(key); err != nil { + logger.Error(logSender, "", "unable to load acme key file %q: %v", key, err) + return nil + } + c.acmeDomain = configs.ACME.Domain + logger.Info(logSender, "", "acme domain set to %q", c.acmeDomain) + return nil + } + return nil +} + // Initialize configures and starts the FTP server func (c *Configuration) Initialize(configDir string) error { + if err := c.loadFromProvider(); err != nil { + return err + } logger.Info(logSender, "", "initializing FTP server with config %+v", *c) if !c.ShouldBind() { return common.ErrNoBinding diff --git a/internal/ftpd/ftpd_test.go b/internal/ftpd/ftpd_test.go index 81ebf3f8..e32a7101 100644 --- a/internal/ftpd/ftpd_test.go +++ b/internal/ftpd/ftpd_test.go @@ -503,6 +503,18 @@ func TestInitializationFailure(t *testing.T) { ftpdConf.Bindings[1].ForcePassiveIP = "" err = ftpdConf.Initialize(configDir) require.Error(t, err) + + err = dataprovider.Close() + assert.NoError(t, err) + err = ftpdConf.Initialize(configDir) + if assert.Error(t, err) { + assert.Contains(t, err.Error(), "unable to load config from provider") + } + err = config.LoadConfig(configDir, "") + assert.NoError(t, err) + providerConf := config.GetProviderConf() + err = dataprovider.Initialize(providerConf, configDir, true) + assert.NoError(t, err) } func TestBasicFTPHandling(t *testing.T) { diff --git a/internal/ftpd/internal_test.go b/internal/ftpd/internal_test.go index e96aeefd..d2526d42 100644 --- a/internal/ftpd/internal_test.go +++ b/internal/ftpd/internal_test.go @@ -36,6 +36,7 @@ import ( "github.com/drakkan/sftpgo/v2/internal/common" "github.com/drakkan/sftpgo/v2/internal/dataprovider" + "github.com/drakkan/sftpgo/v2/internal/util" "github.com/drakkan/sftpgo/v2/internal/vfs" ) @@ -1059,3 +1060,70 @@ func TestRelativePath(t *testing.T) { rel = getPathRelativeTo("/dir3", "dir3") assert.Equal(t, "dir3", rel) } + +func TestConfigsFromProvider(t *testing.T) { + err := dataprovider.UpdateConfigs(nil, "", "", "") + assert.NoError(t, err) + c := Configuration{} + err = c.loadFromProvider() + assert.NoError(t, err) + assert.Empty(t, c.acmeDomain) + configs := dataprovider.Configs{ + ACME: &dataprovider.ACMEConfigs{ + Domain: "domain.com", + Email: "info@domain.com", + HTTP01Challenge: dataprovider.ACMEHTTP01Challenge{Port: 80}, + Protocols: 2, + }, + } + err = dataprovider.UpdateConfigs(&configs, "", "", "") + assert.NoError(t, err) + util.CertsBasePath = "" + // crt and key empty + err = c.loadFromProvider() + assert.NoError(t, err) + assert.Empty(t, c.acmeDomain) + util.CertsBasePath = filepath.Clean(os.TempDir()) + // crt not found + err = c.loadFromProvider() + assert.NoError(t, err) + assert.Empty(t, c.acmeDomain) + keyPairs := c.getKeyPairs(configDir) + assert.Len(t, keyPairs, 0) + crtPath := filepath.Join(util.CertsBasePath, util.SanitizeDomain(configs.ACME.Domain)+".crt") + err = os.WriteFile(crtPath, nil, 0666) + assert.NoError(t, err) + // key not found + err = c.loadFromProvider() + assert.NoError(t, err) + assert.Empty(t, c.acmeDomain) + keyPairs = c.getKeyPairs(configDir) + assert.Len(t, keyPairs, 0) + keyPath := filepath.Join(util.CertsBasePath, util.SanitizeDomain(configs.ACME.Domain)+".key") + err = os.WriteFile(keyPath, nil, 0666) + assert.NoError(t, err) + // acme cert used + err = c.loadFromProvider() + assert.NoError(t, err) + assert.Equal(t, configs.ACME.Domain, c.acmeDomain) + keyPairs = c.getKeyPairs(configDir) + assert.Len(t, keyPairs, 1) + // protocols does not match + configs.ACME.Protocols = 5 + err = dataprovider.UpdateConfigs(&configs, "", "", "") + assert.NoError(t, err) + c.acmeDomain = "" + err = c.loadFromProvider() + assert.NoError(t, err) + assert.Empty(t, c.acmeDomain) + keyPairs = c.getKeyPairs(configDir) + assert.Len(t, keyPairs, 0) + + err = os.Remove(crtPath) + assert.NoError(t, err) + err = os.Remove(keyPath) + assert.NoError(t, err) + util.CertsBasePath = "" + err = dataprovider.UpdateConfigs(nil, "", "", "") + assert.NoError(t, err) +} diff --git a/internal/httpd/httpd.go b/internal/httpd/httpd.go index 0ffa2fe2..0881f01d 100644 --- a/internal/httpd/httpd.go +++ b/internal/httpd/httpd.go @@ -23,6 +23,7 @@ import ( "fmt" "net" "net/http" + "os" "path" "path/filepath" "runtime" @@ -275,6 +276,8 @@ var ( installationCode string installationCodeHint string fnInstallationCodeResolver FnInstallationCodeResolver + configurationDir string + fnGetCetificates FnGetCertificates ) func init() { @@ -286,6 +289,9 @@ func init() { // If the installation code cannot be resolved the provided default must be returned type FnInstallationCodeResolver func(defaultInstallationCode string) string +// FnGetCertificates defines the method to call to get TLS certificates +type FnGetCertificates func(*dataprovider.ACMEConfigs, string) error + // HTTPSProxyHeader defines an HTTPS proxy header as key/value. // For example Key could be "X-Forwarded-Proto" and Value "https" type HTTPSProxyHeader struct { @@ -740,6 +746,7 @@ type Conf struct { Setup SetupConfig `json:"setup" mapstructure:"setup"` // If enabled, the link to the sponsors section will not appear on the setup screen page HideSupportLink bool `json:"hide_support_link" mapstructure:"hide_support_link"` + acmeDomain string } type apiResponse struct { @@ -820,8 +827,13 @@ func (c *Conf) getKeyPairs(configDir string) []common.TLSKeyPair { }) } } - certificateFile := getConfigPath(c.CertificateFile, configDir) - certificateKeyFile := getConfigPath(c.CertificateKeyFile, configDir) + var certificateFile, certificateKeyFile string + if c.acmeDomain != "" { + certificateFile, certificateKeyFile = util.GetACMECertificateKeyPair(c.acmeDomain) + } else { + certificateFile = getConfigPath(c.CertificateFile, configDir) + certificateKeyFile = getConfigPath(c.CertificateKeyFile, configDir) + } if certificateFile != "" && certificateKeyFile != "" { keyPairs = append(keyPairs, common.TLSKeyPair{ Cert: certificateFile, @@ -840,9 +852,42 @@ func (c *Conf) setTokenValidationMode() { } } +func (c *Conf) loadFromProvider() error { + configs, err := dataprovider.GetConfigs() + if err != nil { + return fmt.Errorf("unable to load config from provider: %w", err) + } + configs.SetNilsToEmpty() + if configs.ACME.Domain == "" || !configs.ACME.HasProtocol(common.ProtocolHTTP) { + return nil + } + crt, key := util.GetACMECertificateKeyPair(configs.ACME.Domain) + if crt != "" && key != "" { + if _, err := os.Stat(crt); err != nil { + logger.Error(logSender, "", "unable to load acme cert file %q: %v", crt, err) + return nil + } + if _, err := os.Stat(key); err != nil { + logger.Error(logSender, "", "unable to load acme key file %q: %v", key, err) + return nil + } + for idx := range c.Bindings { + c.Bindings[idx].EnableHTTPS = true + } + c.acmeDomain = configs.ACME.Domain + logger.Info(logSender, "", "acme domain set to %q", c.acmeDomain) + return nil + } + return nil +} + // Initialize configures and starts the HTTP server func (c *Conf) Initialize(configDir string, isShared int) error { + if err := c.loadFromProvider(); err != nil { + return err + } logger.Info(logSender, "", "initializing HTTP server with config %+v", c.getRedacted()) + configurationDir = configDir resetCodesMgr = newResetCodeManager(isShared) oidcMgr = newOIDCManager(isShared) staticFilesPath := util.FindSharedDataPath(c.StaticFilesPath, configDir) @@ -1144,3 +1189,15 @@ func resolveInstallationCode() string { } return installationCode } + +// SetCertificatesGetter sets the function to call to get TLS certificates +func SetCertificatesGetter(fn FnGetCertificates) { + fnGetCetificates = fn +} + +func getTLSCertificates(c *dataprovider.ACMEConfigs) error { + if fnGetCetificates == nil { + return errors.New("unable to get TLS certificates, callback not defined") + } + return fnGetCetificates(c, configurationDir) +} diff --git a/internal/httpd/httpd_test.go b/internal/httpd/httpd_test.go index 022c5006..2e8911ea 100644 --- a/internal/httpd/httpd_test.go +++ b/internal/httpd/httpd_test.go @@ -568,6 +568,17 @@ func TestInitialization(t *testing.T) { if assert.Error(t, err) { assert.Contains(t, err.Error(), "no login method available for WebClient UI") } + err = dataprovider.Close() + assert.NoError(t, err) + err = httpdConf.Initialize(configDir, isShared) + if assert.Error(t, err) { + assert.Contains(t, err.Error(), "unable to load config from provider") + } + err = config.LoadConfig(configDir, "") + assert.NoError(t, err) + providerConf := config.GetProviderConf() + err = dataprovider.Initialize(providerConf, configDir, true) + assert.NoError(t, err) } func TestBasicUserHandling(t *testing.T) { @@ -12322,15 +12333,16 @@ func TestWebConfigsMock(t *testing.T) { form.Set("smtp_username", defaultUsername) form.Set("smtp_password", defaultPassword) form.Set("smtp_domain", "localdomain") + form.Set("smtp_auth", "100") req, err = http.NewRequest(http.MethodPost, webConfigsPath, bytes.NewBuffer([]byte(form.Encode()))) assert.NoError(t, err) setJWTCookieForReq(req, webToken) req.Header.Set("Content-Type", "application/x-www-form-urlencoded") rr = executeRequest(req) checkResponseCode(t, http.StatusOK, rr) - assert.Contains(t, rr.Body.String(), "Validation error") // port is not passed so 0 + assert.Contains(t, rr.Body.String(), "Validation error") // invalid smtp_auth // set valid parameters - form.Set("smtp_port", "465") + form.Set("smtp_port", "a") // converted to 587 form.Set("smtp_auth", "1") form.Set("smtp_encryption", "2") req, err = http.NewRequest(http.MethodPost, webConfigsPath, bytes.NewBuffer([]byte(form.Encode()))) @@ -12348,7 +12360,7 @@ func TestWebConfigsMock(t *testing.T) { assert.Contains(t, configs.SFTPD.HostKeyAlgos, ssh.CertAlgoRSAv01) assert.Len(t, configs.SFTPD.Moduli, 2) assert.Equal(t, "mail.example.net", configs.SMTP.Host) - assert.Equal(t, 465, configs.SMTP.Port) + assert.Equal(t, 587, configs.SMTP.Port) assert.Equal(t, "Example ", configs.SMTP.From) assert.Equal(t, defaultUsername, configs.SMTP.User) err = configs.SMTP.Password.Decrypt() @@ -12359,6 +12371,8 @@ func TestWebConfigsMock(t *testing.T) { assert.Equal(t, "localdomain", configs.SMTP.Domain) // set a redacted password, the current password must be preserved form.Set("smtp_password", redactedSecret) + form.Set("smtp_auth", "") + configs.SMTP.AuthType = 0 // empty will be converted to 0 req, err = http.NewRequest(http.MethodPost, webConfigsPath, bytes.NewBuffer([]byte(form.Encode()))) assert.NoError(t, err) setJWTCookieForReq(req, webToken) @@ -12385,6 +12399,76 @@ func TestWebConfigsMock(t *testing.T) { rr = executeRequest(req) checkResponseCode(t, http.StatusOK, rr) assert.Contains(t, rr.Body.String(), "Configurations updated") + // test ACME configs, set a fake callback to avoid Let's encrypt calls + httpd.SetCertificatesGetter(func(a *dataprovider.ACMEConfigs, s string) error { return nil }) + form.Set("form_action", "acme_submit") + form.Set("acme_port", "") // on error will be set to 80 + form.Set("acme_protocols", "1") + form.Add("acme_protocols", "2") + form.Add("acme_protocols", "3") + form.Set("acme_domain", "example.com") + // no email set, validation will fail + req, err = http.NewRequest(http.MethodPost, webConfigsPath, bytes.NewBuffer([]byte(form.Encode()))) + assert.NoError(t, err) + setJWTCookieForReq(req, webToken) + req.Header.Set("Content-Type", "application/x-www-form-urlencoded") + rr = executeRequest(req) + checkResponseCode(t, http.StatusOK, rr) + assert.Contains(t, rr.Body.String(), "Validation error") + form.Set("acme_domain", "") + req, err = http.NewRequest(http.MethodPost, webConfigsPath, bytes.NewBuffer([]byte(form.Encode()))) + assert.NoError(t, err) + setJWTCookieForReq(req, webToken) + req.Header.Set("Content-Type", "application/x-www-form-urlencoded") + rr = executeRequest(req) + checkResponseCode(t, http.StatusOK, rr) + assert.Contains(t, rr.Body.String(), "Configurations updated") + // check + configs, err = dataprovider.GetConfigs() + assert.NoError(t, err) + assert.Len(t, configs.SFTPD.HostKeyAlgos, 2) + assert.Contains(t, configs.SFTPD.HostKeyAlgos, ssh.KeyAlgoRSA) + assert.Contains(t, configs.SFTPD.HostKeyAlgos, ssh.CertAlgoRSAv01) + assert.Len(t, configs.SFTPD.Moduli, 2) + assert.Equal(t, 80, configs.ACME.HTTP01Challenge.Port) + assert.Equal(t, 7, configs.ACME.Protocols) + assert.Empty(t, configs.ACME.Domain) + assert.Empty(t, configs.ACME.Email) + assert.True(t, configs.ACME.HasProtocol(common.ProtocolFTP)) + assert.True(t, configs.ACME.HasProtocol(common.ProtocolWebDAV)) + assert.True(t, configs.ACME.HasProtocol(common.ProtocolHTTP)) + + form.Set("acme_port", "402") + form.Set("acme_protocols", "1") + form.Add("acme_protocols", "1000") + form.Set("acme_domain", "acme.example.com") + form.Set("acme_email", "email@example.com") + req, err = http.NewRequest(http.MethodPost, webConfigsPath, bytes.NewBuffer([]byte(form.Encode()))) + assert.NoError(t, err) + setJWTCookieForReq(req, webToken) + req.Header.Set("Content-Type", "application/x-www-form-urlencoded") + rr = executeRequest(req) + checkResponseCode(t, http.StatusOK, rr) + assert.Contains(t, rr.Body.String(), "Configurations updated") + configs, err = dataprovider.GetConfigs() + assert.NoError(t, err) + assert.Len(t, configs.SFTPD.HostKeyAlgos, 2) + assert.Equal(t, 402, configs.ACME.HTTP01Challenge.Port) + assert.Equal(t, 1, configs.ACME.Protocols) + assert.Equal(t, "acme.example.com", configs.ACME.Domain) + assert.Equal(t, "email@example.com", configs.ACME.Email) + assert.False(t, configs.ACME.HasProtocol(common.ProtocolFTP)) + assert.False(t, configs.ACME.HasProtocol(common.ProtocolWebDAV)) + assert.True(t, configs.ACME.HasProtocol(common.ProtocolHTTP)) + // updates will fail, the get certificate fn will return error with nil callback + httpd.SetCertificatesGetter(nil) + req, err = http.NewRequest(http.MethodPost, webConfigsPath, bytes.NewBuffer([]byte(form.Encode()))) + assert.NoError(t, err) + setJWTCookieForReq(req, webToken) + req.Header.Set("Content-Type", "application/x-www-form-urlencoded") + rr = executeRequest(req) + checkResponseCode(t, http.StatusOK, rr) + assert.Contains(t, rr.Body.String(), "unable to get TLS certificates") err = dataprovider.UpdateConfigs(nil, "", "", "") assert.NoError(t, err) diff --git a/internal/httpd/internal_test.go b/internal/httpd/internal_test.go index e19f932f..0988f0d5 100644 --- a/internal/httpd/internal_test.go +++ b/internal/httpd/internal_test.go @@ -3055,6 +3055,80 @@ func TestEventsCSV(t *testing.T) { assert.Equal(t, "Quota exceeded", data[5]) } +func TestConfigsFromProvider(t *testing.T) { + err := dataprovider.UpdateConfigs(nil, "", "", "") + assert.NoError(t, err) + c := Conf{ + Bindings: []Binding{ + { + Port: 1234, + }, + }, + } + err = c.loadFromProvider() + assert.NoError(t, err) + assert.Empty(t, c.acmeDomain) + configs := dataprovider.Configs{ + ACME: &dataprovider.ACMEConfigs{ + Domain: "domain.com", + Email: "info@domain.com", + HTTP01Challenge: dataprovider.ACMEHTTP01Challenge{Port: 80}, + Protocols: 1, + }, + } + err = dataprovider.UpdateConfigs(&configs, "", "", "") + assert.NoError(t, err) + util.CertsBasePath = "" + // crt and key empty + err = c.loadFromProvider() + assert.NoError(t, err) + assert.Empty(t, c.acmeDomain) + util.CertsBasePath = filepath.Clean(os.TempDir()) + // crt not found + err = c.loadFromProvider() + assert.NoError(t, err) + assert.Empty(t, c.acmeDomain) + keyPairs := c.getKeyPairs(configDir) + assert.Len(t, keyPairs, 0) + crtPath := filepath.Join(util.CertsBasePath, util.SanitizeDomain(configs.ACME.Domain)+".crt") + err = os.WriteFile(crtPath, nil, 0666) + assert.NoError(t, err) + // key not found + err = c.loadFromProvider() + assert.NoError(t, err) + assert.Empty(t, c.acmeDomain) + keyPairs = c.getKeyPairs(configDir) + assert.Len(t, keyPairs, 0) + keyPath := filepath.Join(util.CertsBasePath, util.SanitizeDomain(configs.ACME.Domain)+".key") + err = os.WriteFile(keyPath, nil, 0666) + assert.NoError(t, err) + // acme cert used + err = c.loadFromProvider() + assert.NoError(t, err) + assert.Equal(t, configs.ACME.Domain, c.acmeDomain) + keyPairs = c.getKeyPairs(configDir) + assert.Len(t, keyPairs, 1) + assert.True(t, c.Bindings[0].EnableHTTPS) + // protocols does not match + configs.ACME.Protocols = 6 + err = dataprovider.UpdateConfigs(&configs, "", "", "") + assert.NoError(t, err) + c.acmeDomain = "" + err = c.loadFromProvider() + assert.NoError(t, err) + assert.Empty(t, c.acmeDomain) + keyPairs = c.getKeyPairs(configDir) + assert.Len(t, keyPairs, 0) + + err = os.Remove(crtPath) + assert.NoError(t, err) + err = os.Remove(keyPath) + assert.NoError(t, err) + util.CertsBasePath = "" + err = dataprovider.UpdateConfigs(nil, "", "", "") + assert.NoError(t, err) +} + func isSharedProviderSupported() bool { // SQLite shares the implementation with other SQL-based provider but it makes no sense // to use it outside test cases diff --git a/internal/httpd/webadmin.go b/internal/httpd/webadmin.go index d863a1cb..34e86cd7 100644 --- a/internal/httpd/webadmin.go +++ b/internal/httpd/webadmin.go @@ -903,6 +903,9 @@ func (s *httpdServer) renderConfigsPage(w http.ResponseWriter, r *http.Request, if configs.SMTP.Port == 0 { configs.SMTP.Port = 587 } + if configs.ACME.HTTP01Challenge.Port == 0 { + configs.ACME.HTTP01Challenge.Port = 80 + } data := configsPage{ basePage: s.getBasePageData(pageConfigsTitle, webConfigsPath, r), Configs: configs, @@ -2544,10 +2547,35 @@ func getSFTPConfigsFromPostFields(r *http.Request) *dataprovider.SFTPDConfigs { } } +func getACMEConfigsFromPostFields(r *http.Request) *dataprovider.ACMEConfigs { + port, err := strconv.Atoi(r.Form.Get("acme_port")) + if err != nil { + port = 80 + } + var protocols int + for _, val := range r.Form["acme_protocols"] { + switch val { + case "1": + protocols++ + case "2": + protocols += 2 + case "3": + protocols += 4 + } + } + + return &dataprovider.ACMEConfigs{ + Domain: strings.TrimSpace(r.Form.Get("acme_domain")), + Email: strings.TrimSpace(r.Form.Get("acme_email")), + HTTP01Challenge: dataprovider.ACMEHTTP01Challenge{Port: port}, + Protocols: protocols, + } +} + func getSMTPConfigsFromPostFields(r *http.Request) *dataprovider.SMTPConfigs { port, err := strconv.Atoi(r.Form.Get("smtp_port")) if err != nil { - port = 0 + port = 587 } authType, err := strconv.Atoi(r.Form.Get("smtp_auth")) if err != nil { @@ -4042,8 +4070,16 @@ func (s *httpdServer) handleWebConfigsPost(w http.ResponseWriter, r *http.Reques configSection = 1 sftpConfigs := getSFTPConfigsFromPostFields(r) configs.SFTPD = sftpConfigs - case "smtp_submit": + case "acme_submit": configSection = 2 + acmeConfigs := getACMEConfigsFromPostFields(r) + configs.ACME = acmeConfigs + if err := getTLSCertificates(acmeConfigs); err != nil { + s.renderConfigsPage(w, r, configs, err.Error(), configSection) + return + } + case "smtp_submit": + configSection = 3 smtpConfigs := getSMTPConfigsFromPostFields(r) if smtpConfigs.Password.IsNotPlainAndNotEmpty() { smtpConfigs.Password = configs.SMTP.Password @@ -4059,7 +4095,7 @@ func (s *httpdServer) handleWebConfigsPost(w http.ResponseWriter, r *http.Reques s.renderConfigsPage(w, r, configs, err.Error(), configSection) return } - if configSection == 2 { + if configSection == 3 { err := configs.SMTP.Password.TryDecrypt() if err == nil { smtp.Activate(configs.SMTP) diff --git a/internal/service/service.go b/internal/service/service.go index dbeeec22..7abbf432 100644 --- a/internal/service/service.go +++ b/internal/service/service.go @@ -23,6 +23,7 @@ import ( "github.com/rs/zerolog" + "github.com/drakkan/sftpgo/v2/internal/acme" "github.com/drakkan/sftpgo/v2/internal/common" "github.com/drakkan/sftpgo/v2/internal/config" "github.com/drakkan/sftpgo/v2/internal/dataprovider" @@ -169,7 +170,7 @@ func (s *Service) initializeServices(disableAWSInstallationCode bool) error { } } else { acmeConfig := config.GetACMEConfig() - err = acmeConfig.Initialize(s.ConfigDir, true) + err = acme.Initialize(acmeConfig, s.ConfigDir, true) if err != nil { logger.Error(logSender, "", "error initializing ACME configuration: %v", err) logger.ErrorToConsole("error initializing ACME configuration: %v", err) diff --git a/internal/telemetry/router.go b/internal/telemetry/router.go index 5372fefe..7364a029 100644 --- a/internal/telemetry/router.go +++ b/internal/telemetry/router.go @@ -30,6 +30,7 @@ func initializeRouter(enableProfiler bool) { router = chi.NewRouter() router.Use(middleware.GetHead) + router.Use(logger.NewStructuredLogger(logger.GetLogger())) router.Use(middleware.Recoverer) router.Group(func(r chi.Router) { diff --git a/internal/util/util.go b/internal/util/util.go index e20969a9..69380b35 100644 --- a/internal/util/util.go +++ b/internal/util/util.go @@ -61,6 +61,9 @@ var ( emailRegex = regexp.MustCompile("^(?:(?:(?:(?:[a-zA-Z]|\\d|[!#\\$%&'\\*\\+\\-\\/=\\?\\^_`{\\|}~]|[\\x{00A0}-\\x{D7FF}\\x{F900}-\\x{FDCF}\\x{FDF0}-\\x{FFEF}])+(?:\\.([a-zA-Z]|\\d|[!#\\$%&'\\*\\+\\-\\/=\\?\\^_`{\\|}~]|[\\x{00A0}-\\x{D7FF}\\x{F900}-\\x{FDCF}\\x{FDF0}-\\x{FFEF}])+)*)|(?:(?:\\x22)(?:(?:(?:(?:\\x20|\\x09)*(?:\\x0d\\x0a))?(?:\\x20|\\x09)+)?(?:(?:[\\x01-\\x08\\x0b\\x0c\\x0e-\\x1f\\x7f]|\\x21|[\\x23-\\x5b]|[\\x5d-\\x7e]|[\\x{00A0}-\\x{D7FF}\\x{F900}-\\x{FDCF}\\x{FDF0}-\\x{FFEF}])|(?:(?:[\\x01-\\x09\\x0b\\x0c\\x0d-\\x7f]|[\\x{00A0}-\\x{D7FF}\\x{F900}-\\x{FDCF}\\x{FDF0}-\\x{FFEF}]))))*(?:(?:(?:\\x20|\\x09)*(?:\\x0d\\x0a))?(\\x20|\\x09)+)?(?:\\x22))))@(?:(?:(?:[a-zA-Z]|\\d|[\\x{00A0}-\\x{D7FF}\\x{F900}-\\x{FDCF}\\x{FDF0}-\\x{FFEF}])|(?:(?:[a-zA-Z]|\\d|[\\x{00A0}-\\x{D7FF}\\x{F900}-\\x{FDCF}\\x{FDF0}-\\x{FFEF}])(?:[a-zA-Z]|\\d|-|\\.|~|[\\x{00A0}-\\x{D7FF}\\x{F900}-\\x{FDCF}\\x{FDF0}-\\x{FFEF}])*(?:[a-zA-Z]|\\d|[\\x{00A0}-\\x{D7FF}\\x{F900}-\\x{FDCF}\\x{FDF0}-\\x{FFEF}])))\\.)+(?:(?:[a-zA-Z]|[\\x{00A0}-\\x{D7FF}\\x{F900}-\\x{FDCF}\\x{FDF0}-\\x{FFEF}])|(?:(?:[a-zA-Z]|[\\x{00A0}-\\x{D7FF}\\x{F900}-\\x{FDCF}\\x{FDF0}-\\x{FFEF}])(?:[a-zA-Z]|\\d|-|\\.|~|[\\x{00A0}-\\x{D7FF}\\x{F900}-\\x{FDCF}\\x{FDF0}-\\x{FFEF}])*(?:[a-zA-Z]|[\\x{00A0}-\\x{D7FF}\\x{F900}-\\x{FDCF}\\x{FDF0}-\\x{FFEF}])))\\.?$") // this can be set at build time additionalSharedDataSearchPath = "" + // CertsBasePath defines base path for certificates obtained using the built-in ACME protocol. + // It is empty is ACME support is disabled + CertsBasePath string ) // IEC Sizes. @@ -754,6 +757,11 @@ func IsEmailValid(email string) bool { return emailRegex.MatchString(email) } +// SanitizeDomain return the specified domain name in a form suitable to save as file +func SanitizeDomain(domain string) string { + return strings.NewReplacer(":", "_", "*", "_", ",", "_", " ", "_").Replace(domain) +} + // PanicOnError calls panic if err is not nil func PanicOnError(err error) { if err != nil { @@ -777,6 +785,15 @@ func GetAbsolutePath(name string) (string, error) { return filepath.Join(curDir, name), nil } +// GetACMECertificateKeyPair returns the path to the ACME TLS crt and key for the specified domain +func GetACMECertificateKeyPair(domain string) (string, string) { + if CertsBasePath == "" { + return "", "" + } + domain = SanitizeDomain(domain) + return filepath.Join(CertsBasePath, domain+".crt"), filepath.Join(CertsBasePath, domain+".key") +} + // GetLastIPForPrefix returns the last IP for the given prefix // https://github.com/go4org/netipx/blob/8449b0a6169f5140fb0340cb4fc0de4c9b281ef6/netipx.go#L173 func GetLastIPForPrefix(p netip.Prefix) netip.Addr { diff --git a/internal/webdavd/internal_test.go b/internal/webdavd/internal_test.go index 7d1d9169..c8622307 100644 --- a/internal/webdavd/internal_test.go +++ b/internal/webdavd/internal_test.go @@ -1584,3 +1584,78 @@ func TestMisc(t *testing.T) { certMgr = oldCertMgr } + +func TestConfigsFromProvider(t *testing.T) { + configDir := "." + err := dataprovider.UpdateConfigs(nil, "", "", "") + assert.NoError(t, err) + c := Configuration{ + Bindings: []Binding{ + { + Port: 1234, + }, + }, + } + err = c.loadFromProvider() + assert.NoError(t, err) + assert.Empty(t, c.acmeDomain) + configs := dataprovider.Configs{ + ACME: &dataprovider.ACMEConfigs{ + Domain: "domain.com", + Email: "info@domain.com", + HTTP01Challenge: dataprovider.ACMEHTTP01Challenge{Port: 80}, + Protocols: 7, + }, + } + err = dataprovider.UpdateConfigs(&configs, "", "", "") + assert.NoError(t, err) + util.CertsBasePath = "" + // crt and key empty + err = c.loadFromProvider() + assert.NoError(t, err) + assert.Empty(t, c.acmeDomain) + util.CertsBasePath = filepath.Clean(os.TempDir()) + // crt not found + err = c.loadFromProvider() + assert.NoError(t, err) + assert.Empty(t, c.acmeDomain) + keyPairs := c.getKeyPairs(configDir) + assert.Len(t, keyPairs, 0) + crtPath := filepath.Join(util.CertsBasePath, util.SanitizeDomain(configs.ACME.Domain)+".crt") + err = os.WriteFile(crtPath, nil, 0666) + assert.NoError(t, err) + // key not found + err = c.loadFromProvider() + assert.NoError(t, err) + assert.Empty(t, c.acmeDomain) + keyPairs = c.getKeyPairs(configDir) + assert.Len(t, keyPairs, 0) + keyPath := filepath.Join(util.CertsBasePath, util.SanitizeDomain(configs.ACME.Domain)+".key") + err = os.WriteFile(keyPath, nil, 0666) + assert.NoError(t, err) + // acme cert used + err = c.loadFromProvider() + assert.NoError(t, err) + assert.Equal(t, configs.ACME.Domain, c.acmeDomain) + keyPairs = c.getKeyPairs(configDir) + assert.Len(t, keyPairs, 1) + assert.True(t, c.Bindings[0].EnableHTTPS) + // protocols does not match + configs.ACME.Protocols = 3 + err = dataprovider.UpdateConfigs(&configs, "", "", "") + assert.NoError(t, err) + c.acmeDomain = "" + err = c.loadFromProvider() + assert.NoError(t, err) + assert.Empty(t, c.acmeDomain) + keyPairs = c.getKeyPairs(configDir) + assert.Len(t, keyPairs, 0) + + err = os.Remove(crtPath) + assert.NoError(t, err) + err = os.Remove(keyPath) + assert.NoError(t, err) + util.CertsBasePath = "" + err = dataprovider.UpdateConfigs(nil, "", "", "") + assert.NoError(t, err) +} diff --git a/internal/webdavd/webdavd.go b/internal/webdavd/webdavd.go index ff3c14f7..16c762d5 100644 --- a/internal/webdavd/webdavd.go +++ b/internal/webdavd/webdavd.go @@ -18,6 +18,7 @@ package webdavd import ( "fmt" "net" + "os" "path/filepath" "github.com/go-chi/chi/v5/middleware" @@ -181,7 +182,8 @@ type Configuration struct { // CORS configuration Cors CorsConfig `json:"cors" mapstructure:"cors"` // Cache configuration - Cache Cache `json:"cache" mapstructure:"cache"` + Cache Cache `json:"cache" mapstructure:"cache"` + acmeDomain string } // GetStatus returns the server status @@ -214,8 +216,13 @@ func (c *Configuration) getKeyPairs(configDir string) []common.TLSKeyPair { }) } } - certificateFile := getConfigPath(c.CertificateFile, configDir) - certificateKeyFile := getConfigPath(c.CertificateKeyFile, configDir) + var certificateFile, certificateKeyFile string + if c.acmeDomain != "" { + certificateFile, certificateKeyFile = util.GetACMECertificateKeyPair(c.acmeDomain) + } else { + certificateFile = getConfigPath(c.CertificateFile, configDir) + certificateKeyFile = getConfigPath(c.CertificateKeyFile, configDir) + } if certificateFile != "" && certificateKeyFile != "" { keyPairs = append(keyPairs, common.TLSKeyPair{ Cert: certificateFile, @@ -226,8 +233,40 @@ func (c *Configuration) getKeyPairs(configDir string) []common.TLSKeyPair { return keyPairs } +func (c *Configuration) loadFromProvider() error { + configs, err := dataprovider.GetConfigs() + if err != nil { + return fmt.Errorf("unable to load config from provider: %w", err) + } + configs.SetNilsToEmpty() + if configs.ACME.Domain == "" || !configs.ACME.HasProtocol(common.ProtocolWebDAV) { + return nil + } + crt, key := util.GetACMECertificateKeyPair(configs.ACME.Domain) + if crt != "" && key != "" { + if _, err := os.Stat(crt); err != nil { + logger.Error(logSender, "", "unable to load acme cert file %q: %v", crt, err) + return nil + } + if _, err := os.Stat(key); err != nil { + logger.Error(logSender, "", "unable to load acme key file %q: %v", key, err) + return nil + } + for idx := range c.Bindings { + c.Bindings[idx].EnableHTTPS = true + } + c.acmeDomain = configs.ACME.Domain + logger.Info(logSender, "", "acme domain set to %q", c.acmeDomain) + return nil + } + return nil +} + // Initialize configures and starts the WebDAV server func (c *Configuration) Initialize(configDir string) error { + if err := c.loadFromProvider(); err != nil { + return err + } logger.Info(logSender, "", "initializing WebDAV server with config %+v", *c) mimeTypeCache = mimeCache{ maxSize: c.Cache.MimeTypes.MaxSize, diff --git a/internal/webdavd/webdavd_test.go b/internal/webdavd/webdavd_test.go index e5bb5d48..01109a3b 100644 --- a/internal/webdavd/webdavd_test.go +++ b/internal/webdavd/webdavd_test.go @@ -510,6 +510,17 @@ func TestInitialization(t *testing.T) { cfg.Bindings[0].ProxyAllowed = nil err = cfg.Initialize(configDir) assert.Error(t, err) + err = dataprovider.Close() + assert.NoError(t, err) + err = cfg.Initialize(configDir) + if assert.Error(t, err) { + assert.Contains(t, err.Error(), "unable to load config from provider") + } + err = config.LoadConfig(configDir, "") + assert.NoError(t, err) + providerConf := config.GetProviderConf() + err = dataprovider.Initialize(providerConf, configDir, true) + assert.NoError(t, err) } func TestBasicHandling(t *testing.T) { diff --git a/templates/webadmin/configs.html b/templates/webadmin/configs.html index a62797db..c9505070 100644 --- a/templates/webadmin/configs.html +++ b/templates/webadmin/configs.html @@ -116,6 +116,77 @@ along with this program. If not, see . +
+
+

+ +

+
+ +
+
+
+
From this section you can request free TLS certificates for your SFTPGo services using the ACME protocol and the HTTP-01 challenge type. You must create a DNS entry under a custom domain that you own which resolves to your SFTPGo public IP address and the port 80 must be publicly reachable. You can set the configuration options for the most common use cases and single node setups here, for advanced configurations refer to the SFTPGo docs. A service restart is required to apply changes
+
+ +
+ +
+ + + Multiple domains can be specified comma or space separated. They will be included in the same certificate + +
+
+ +
+ +
+ + + Email used for registration and recovery contact + +
+
+ +
+ +
+ + + If different from 80 you have to configure a reverse proxy + +
+
+ +
+ +
+ + + Use the obtained certificates for the specified protocols + +
+
+ +
+ +
+ +
+
+ +

@@ -126,7 +197,7 @@ along with this program. If not, see .

-
+
Set the SMTP configuration replacing the one defined using env vars or config file if any.
@@ -144,7 +215,7 @@ along with this program. If not, see .
-
@@ -268,6 +339,10 @@ along with this program. If not, see .