Commit graph

148 commits

Author SHA1 Message Date
Nicola Murino
570964deb3
add post-disconnect hook
Fixes #587
2021-10-29 19:55:18 +02:00
Nicola Murino
4aa9686e3b
refactor custom actions
SFTPGo is now fully auditable, all fs and provider events that change
something are notified and can be collected using hooks/plugins.

There are some backward incompatible changes for command hooks
2021-10-10 13:08:05 +02:00
Nicola Murino
ea01c3a125
rate limiting: allow to exclude IP addresses/ranges
Fixes #563
2021-10-03 20:50:05 +02:00
Nicola Murino
1b4a1fbbe5
add data retention check hook 2021-10-03 15:17:49 +02:00
Nicola Murino
cc134cad9a
data retention: allow to notify results via e-mail 2021-10-02 22:25:41 +02:00
Nicola Murino
da0ccc6426
add SMTP support
it will be used in future update to add email sending capabilities
2021-09-26 20:25:37 +02:00
Nicola Murino
8a4c21b64a
add builtin two-factor auth support
The builtin two-factor authentication is based on time-based one time
passwords (RFC 6238) which works with Authy, Google Authenticator and
other compatible apps.
2021-09-04 12:11:04 +02:00
Nicola Murino
bcf088f586
data provider: update internal caches if the data provider is shared 2021-08-20 09:35:06 +02:00
Nicola Murino
ced2e16f41
add support for password validation rules
Fixes #494
2021-08-06 18:56:07 +02:00
Nicola Murino
a3c087456b
ftpd: add some security checks 2021-08-05 18:38:15 +02:00
mmcgeefeedo
3ae8abda9e
sftpd: add folder prefix middleware 2021-07-29 00:32:55 +02:00
Nicola Murino
90b324d707
Add a link on the login pages to switch between admin and web client login
The links are hidden if only the web admin or only thw web client is
enabled and can also be controlled using the "hide_login_url" setting

Fixes #485
2021-07-27 18:43:00 +02:00
Nicola Murino
ae8ccadad2
users API: add API to create, delete, rename files and directories 2021-07-23 10:19:27 +02:00
Nicola Murino
5967aa1aa5
FTP: enable ftpserverlib logging and make debug mode configurable 2021-07-20 17:22:08 +02:00
Nicola Murino
5a568b4077
KMS: allow to provide the master encryption key as string 2021-07-17 15:34:48 +02:00
Nicola Murino
bd5191dfc5
add experimental plugin system 2021-07-11 15:26:51 +02:00
Nicola Murino
ff19879ffd
allow to use a persistent signing key for JWT and CSRF tokens
Fixes #466
2021-07-01 20:17:40 +02:00
Nicola Murino
3b46e6a6fb
add support for a global temp path
Fixes #436
2021-05-27 15:38:27 +02:00
Nicola Murino
600268ebb8
httpclient: allow to set custom headers 2021-05-25 08:36:01 +02:00
Nicola Murino
f2b93c0402
add a setup screen to create the first admin user
If you prefer to auto-create the first admin you can enable the
"create_default_admin" configuration key and SFTPGo will work as before.

You can also create the first admin by loading initial data: now you can
set both username and password, before you could only change the password
2021-05-14 19:21:15 +02:00
Nicola Murino
fa45c9c138
allow to execute actions for file operations and SSH commands synchronously
The actions to run synchronously can be configured via the `execute_sync`
configuration key.

Executing an action synchronously means that SFTPGo will not return a result
code to the client until your hook have completed its execution.

Fixes #409
2021-05-11 12:45:14 +02:00
Nicola Murino
c8f7fc9bc9
httpd/webdav: add a list of hosts allowed to send proxy headers
X-Forwarded-For, X-Real-IP and X-Forwarded-Proto headers will be ignored
for hosts not included in this list.

This is a backward incompatible change, before the proxy headers were
always used
2021-05-11 06:54:06 +02:00
Nicola Murino
8f6cdacd00
allow to limit the number of per-host connections 2021-05-08 19:45:21 +02:00
Nicola Murino
23d9ebfc91
add a basic front-end web interface for end-users
Fixes #339 #321 #398
2021-05-06 21:35:43 +02:00
Nicola Murino
46998252e5
use bcrypt as default password hashing algo
argon2id has a high memory cost and, if not properly tuned, it can lead to
resource starvation.

Advanced users can still configure and use argon2id.
Passwords stored as argon2id will continue to work
2021-04-25 09:38:33 +02:00
Nicola Murino
92638ce93d
add support for hashing password using bcrypt
argon2id remains the default
2021-04-20 13:55:09 +02:00
Nicola Murino
6ef85d6026
add, optional, in memory password caching
Verifying argon2 passwords has a high memory and computational cost,
by enabling, in memory, password caching you reduce this cost
2021-04-20 09:39:36 +02:00
Nicola Murino
f45c89fc46
add rate limiting support for REST API/web admin too 2021-04-19 08:14:04 +02:00
Nicola Murino
112e3b2fc2
add rate limiting support 2021-04-18 12:31:06 +02:00
Nicola Murino
c844fc7477
add support for delayed quota update
If there are a lot of close uploads, accumulating quota updates can
save you many queries to the data provider
2021-04-11 08:38:43 +02:00
Nicola Murino
0bc4db9950
web admin: make base url configurable 2021-04-09 22:02:48 +02:00
Nicola Murino
acb4310c11
add a startup hook 2021-04-05 10:07:59 +02:00
Nicola Murino
4c658bb6f0
webdav: add prefix support 2021-03-07 17:10:45 +01:00
Nicola Murino
df41f0c556
add a setting to skip natural keys validation
Enabling the "skip_natural_keys_validation" data provider setting,
the natural keys for REST API/Web Admin as usernames, admin names,
folder names are not restricted to unreserved URI chars

Fixes #334 #308
2021-03-04 09:48:53 +01:00
Nicola Murino
5da4f931c5
TLS: allow to configure cipher suites
Fixes #316
2021-02-18 20:17:16 +01:00
Nicola Murino
a21ccad174
web hooks: add mutual TLS support 2021-02-13 14:41:37 +01:00
Nicola Murino
6a6e8fffbc
web hooks: improve resilience by adding a configurable retry
the retryable http client is used for hooks that notify events
2021-02-12 21:42:49 +01:00
Nicola Murino
78bf808322
virtual folders: change dataprovider structure
This way we no longer depend on the local file system path and so we can
add support for cloud backends in future updates
2021-02-01 19:04:15 +01:00
Nicola Murino
57976b4085
httpd: add mTLS and multiple bindings support 2021-01-19 18:59:41 +01:00
Nicola Murino
778ec9b88f
REST API v2
- add JWT authentication
- admins are now stored inside the data provider
- admin access can be restricted based on the source IP: both proxy
  header and connection IP are checked
- deprecate REST API CLI: it is not relevant anymore

Some other changes to the REST API can still happen before releasing
SFTPGo 2.0.0

Fixes #197
2021-01-17 22:29:08 +01:00
Nicola Murino
684f4ba1a6
mutal TLS: add support for revocation lists 2021-01-03 17:03:04 +01:00
Nicola Murino
1e1c46ae1b
defender: minor docs improvements 2021-01-02 20:02:05 +01:00
Nicola Murino
037d89a320
add support for a basic built-in defender
It can help to prevent DoS and brute force password guessing
2021-01-02 14:05:09 +01:00
Nicola Murino
40e759c983
FTP: add support for client certificate authentication 2020-12-29 09:20:09 +01:00
Nicola Murino
141ca6777c
webdav: add support for client certificate authentication
Fixes #263
2020-12-28 19:48:23 +01:00
Nicola Murino
1dce1eff48
improve FTP support
- allow to disable active mode
- allow to disable SITE commands
- add optional support for calculating hash value of files
- add optional support for the non standard COMB command
2020-12-24 18:48:06 +01:00
Nicola Murino
c69d63c1f8
add support for multiple bindings
Fixes #253
2020-12-23 16:12:30 +01:00
Nicola Murino
bcf0fa073e
telemetry server: add optional https and authentication 2020-12-18 16:04:42 +01:00
Nicola Murino
143df87fee
add some docs for telemetry server
move pprof to the telemetry server only
2020-12-18 09:47:22 +01:00
Nicola Murino
f34462e3c3
add support for limiting max concurrent client connections 2020-12-15 19:29:30 +01:00
Nicola Murino
634b723b5d
add KMS support
Fixes #226
2020-11-30 21:46:34 +01:00
Nicola Murino
0119fd03a6
webdav: user caching is now mandatory
we cache the lock system with the user, without user caching we cannot
support locks for resource
2020-11-04 22:29:25 +01:00
Nicola Murino
0a14297b48
webdav: performance improvements and bug fixes
we need my custom golang/x/net/webdav fork for now

https://github.com/drakkan/net/tree/sftpgo
2020-11-04 19:11:40 +01:00
Nicola Murino
641493e31a
fix default config file
restore a setting changed for a local test
2020-10-31 11:34:50 +01:00
Nicola Murino
fcfdd633f6
Azure Blob: update SDK and add access tier support 2020-10-30 22:17:17 +01:00
Sean Hildebrand
db7e81e9d0
add prefer_database_credentials configuration parameter
When true, users' Google Cloud Storage credentials will be written to
the data provider instead of disk.
Pre-existing credentials on disk will be used as a fallback

Fixes #201
2020-10-22 10:42:40 +02:00
Nicola Murino
c992072286
data provider: add a setting to prevent auto-update 2020-10-05 19:42:33 +02:00
Nicola Murino
bdf18fa862 password hashing: exposes argon2 options
So the hashing complexity can be changed depending on available
memory/CPU resources and business requirements
2020-09-04 17:09:31 +02:00
Nicola Murino
a59163e56c multi-step auth: don't advertise password method if it is disabled
also rename the settings to password_authentication so it is more like
OpenSSH, add some test cases and improve documentation
2020-09-01 19:34:40 +02:00
Nicola Murino
dbed110d02 WebDAV: add caching for authenticated users
In this way we get a big performance boost
2020-08-31 19:25:17 +02:00
Nicola Murino
600a107699 initprovider: check if the provider is already initialized
exit with code 0 if no initialization is required
2020-08-30 13:50:43 +02:00
Nicola Murino
8b0a1817b3 add check password hook
its main use case is to allow to easily support things like password+OTP for
protocols without keyboard interactive support such as FTP and WebDAV
2020-08-19 19:36:12 +02:00
Nicola Murino
0dbf0cc81f WebDAV: add CORS support 2020-08-15 15:55:20 +02:00
Nicola Murino
196a56726e FTP improvements
- add a setting to require TLS
- add symlink support

require TLS 1.2 for all TLS connections
2020-08-15 13:02:25 +02:00
Nicola Murino
aa0ed5dbd0 add post-login hook
a login scope is supported too so you can get notifications for failed logins,
successful logins or both
2020-08-12 16:15:12 +02:00
Nicola Murino
a9e21c282a add WebDAV support
Fixes #147
2020-08-11 23:56:10 +02:00
Nicola Murino
22338ed478 add post connect hook
Fixes #144
2020-07-30 22:33:49 +02:00
Nicola Murino
93ce96d011 add support for the venerable FTP protocol
Fixes #46
2020-07-29 21:56:56 +02:00
Nicola Murino
4e41a5583d refactoring: add common package
The common package defines the interfaces that a protocol must implement
and contain code that can be shared among supported protocols.

This way should be easier to support new protocols
2020-07-24 23:39:38 +02:00
Nicola Murino
8306b6bde6 refactor virtual folders
The same virtual folder can now be shared among users and different
folder quota limits for each user are supported.

Fixes #120
2020-06-07 23:30:18 +02:00
Nicola Murino
c27e3ef436 actions: add a generic hook to define external commands and HTTP URL
We can only define a single hook now and it can be an HTTP notification
or an external command, not both
2020-05-24 15:29:39 +02:00
Nicola Murino
a08dd85efd sftpd: deprecate keys and add a new host_keys config param
host_key defines the private host keys as plain list of strings.

Remove the other deprecated config params from the default config too.

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2020-05-16 23:26:44 +02:00
Nicola Murino
cf148db75d add test case for expired SSH certificate
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2020-05-15 23:23:49 +02:00
Nicola Murino
738c7ab43e sftpd: add support for SSH user certificate authentication
This add support for PROTOCOL.certkeys vendor extension:

https://cvsweb.openbsd.org/src/usr.bin/ssh/PROTOCOL.certkeys?rev=1.8

Fixes #117

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2020-05-15 20:08:53 +02:00
Nicola Murino
f369fdf6f2 httpclient: add a configuration parameter to skip TLS certificate validation
In this mode, TLS is susceptible to man-in-the-middle attacks.
This should be used only for testing.
2020-05-03 11:37:50 +02:00
Nicola Murino
d377181b25 add a new configuration section for HTTP clients
HTTP clients are used for executing hooks such as the ones used for custom
actions, external authentication and pre-login user modifications.

This allows, for example, to use self-signed certificate without defeating the
purpose of using TLS
2020-04-26 23:29:09 +02:00
Nicola Murino
9046acbe68 add HTTP hooks
external auth, pre-login user modification and keyboard interactive
authentication is now supported via HTTP requests too
2020-04-01 23:25:23 +02:00
Nicola Murino
f284008fb5 enable scp in default configuration
remove the deprecated enable_scp setting
2020-03-26 23:38:24 +01:00
Jo Vandeginste
df02496145 Refactor docs 2020-03-04 23:10:58 +01:00
Nicola Murino
833b702b90 proxy protocol: add list of allowed IP addresses and IP ranges
"proxy_allowed" setting allows to specify the allowed IP address and IP
ranges that can send the proxy header. This setting combined with
"proxy_protocol" allows to ignore the header or to reject connections
that send the proxy header from a non listed IP
2020-03-01 23:12:28 +01:00
Nicola Murino
830e3d1f64 Support for HAProxy PROXY protocol
you can proxy and/or load balance the SFTP/SCP service without losing
the information about the client's address.
2020-02-27 09:21:30 +01:00
Mengsk
637463a068 Rename before_login_program to pre_login_program
and some documentation update
2020-02-25 16:34:54 +01:00
Nicola Murino
703ccc8d91 add support for dynamic users modifications
A custom program can be executed before the users login to modify the
configurations for the user trying to login.
You can, for example, allow login based on time range.

Fixes #77
2020-02-23 18:50:59 +01:00
Nicola Murino
8b039e0447 httpd: add support for basic auth and HTTPS 2020-02-04 00:08:00 +01:00
Nicola Murino
3491717c26 add support for serving Google Cloud Storage over SFTP/SCP
Each user can be mapped with a Google Cloud Storage bucket or a bucket
virtual folder
2020-01-31 19:04:00 +01:00
Nicola Murino
9ff303b8c0 add support for keyboard interactive authentication
Fixes #64
2020-01-21 10:54:05 +01:00
Nicola Murino
531091906d add support for authentication using external programs
Fixes #62
2020-01-06 21:42:41 +01:00
Nicola Murino
ae094d3479 add backup/restore REST API 2019-12-27 23:12:44 +01:00
Nicola Murino
9c4dbbc3f8 sftpd: add support for some SSH commands
md5sum, sha1sum are used by rclone.
cd, pwd improve the support for RemoteFiles mobile app.

These commands are all implemented inside SFTPGo so they work even
if the matching system commands are not available, for example on Windows
2019-11-18 23:30:37 +01:00
Nicola Murino
bb37a1c1ce sftpd: add support for chmod/chown
added matching permissions too and a new setting "setstat_mode".
Setting setstat_mode to 1 you can keep the previous behaviour that
silently ignore setstat requests
2019-11-15 12:15:07 +01:00
Nicola Murino
acdf351047 dataprovider: add custom command and/or HTTP notifications on users add, update and delete
This way custom logic can be implemented for example to create a UNIX user
as asked in #58
2019-11-14 11:06:03 +01:00
Nicola Murino
afd312f26a add a basic web interface
The builtin web interface allows to manage users and connections
2019-10-07 18:19:01 +02:00
Nicola Murino
00dd5db226 add support for users' default base dir 2019-09-28 22:48:52 +02:00
Nicola Murino
e7eb3476b7 dataprovider: remove transaction for quota update
The update is atomic so no transaction is needed.
Addionally a transaction will ask for a new connection to the pool
and this can deadlock if the pool has a max connection limit too low.

Also make configurable the pool size instead of hard code to the cpu number.

Fixes #47
2019-09-13 08:14:07 +02:00
Nicola Murino
6b33d3d9f1 add version to the default server banner 2019-09-12 09:21:58 +02:00
Nicola Murino
49a40f7a0b sftpd: add configuration options for allowed ciphers, MACs and KEX algorithms
add support for login banner too

Fixes #32
2019-09-03 12:08:09 +02:00
Nicola Murino
e50c521c33 add SCP support
SCP is an experimental feature, we have our own SCP implementation
since we can't rely on scp system command to proper handle permissions,
quota and user's home dir restrictions. The SCP protocol is quite simple
but there is no official docs about it, so we need more testing and
feedbacks before enabling it by default.
We may not handle some borderline cases or have sneaky bugs.

This commit contains some breaking changes to the REST API.
SFTPGo API should be stable now and I hope no more breaking changes
before the first stable release.
2019-08-24 14:41:15 +02:00
Nicola Murino
8f421b7d0f switch to viper for configuration and use cobra for cli 2019-08-07 22:46:52 +02:00
Renamed from sftpgo.conf (Browse further)