Nicola Murino
a587228cf0
add support for metadata plugins
2021-12-16 18:18:36 +01:00
Nicola Murino
1472a0f415
hooks: preserve MFA related configs
...
if a user is updated using pre-login or external auth hook we need to
preserve the MFA related configs in the same way we do if the user is
updated using the REST API
2021-12-11 11:08:20 +01:00
Nicola Murino
a7c0b07a2a
add session id to notifier plugins/hook
2021-12-04 17:27:24 +01:00
Nicola Murino
dc1cc88a46
keyboard interactive hooks: allow to validate passcode
2021-12-04 15:14:44 +01:00
Nicola Murino
6092b6628e
logs: use info level for login related messages
...
so enabling debug level is not required, for example only to understand
that a user exceeded the allowed sessions.
Also set the cache update frequency as documented
2021-12-02 19:36:42 +01:00
Nicola Murino
3c24cb773f
SFTP: log users connections at info level
...
uniform SFTP and FTP logs
Fixes #626
2021-11-29 10:15:46 +01:00
Nicola Murino
f2480ce5c9
improve chtimes handling on open files
2021-11-26 19:00:44 +01:00
Nicola Murino
9fb43b2c46
docs: clarify how multi-step auth works with external authentication
...
Fixes #617
2021-11-24 11:27:32 +01:00
Nicola Murino
fb8f013ea7
web: update permissions on cookie refresh
2021-11-20 10:48:39 +01:00
Nicola Murino
094ee1522e
logger: add a flag to use UTC time for logging
2021-11-06 15:18:16 +01:00
Nicola Murino
4aa9686e3b
refactor custom actions
...
SFTPGo is now fully auditable, all fs and provider events that change
something are notified and can be collected using hooks/plugins.
There are some backward incompatible changes for command hooks
2021-10-10 13:08:05 +02:00
Nicola Murino
8a4c21b64a
add builtin two-factor auth support
...
The builtin two-factor authentication is based on time-based one time
passwords (RFC 6238) which works with Authy, Google Authenticator and
other compatible apps.
2021-09-04 12:11:04 +02:00
Nicola Murino
b903a6e46f
data provider: remove default admin
...
you need to load initial data or set "create_default_admin" to true
and the appropriate env vars if you don't want to use the web admin
setup screen to create the default admin
2021-08-20 10:37:51 +02:00
Nicola Murino
be3857d572
dataprovider: add timestamp fields for users and admins
2021-08-19 15:51:43 +02:00
Nicola Murino
0de0d3308c
improve error messages for generic failures
2021-08-08 19:30:21 +02:00
Nicola Murino
a20373b613
add support for auth plugins
2021-08-08 17:09:48 +02:00
Nicola Murino
a3c087456b
ftpd: add some security checks
2021-08-05 18:38:15 +02:00
Nicola Murino
a26962f367
add dot and dot dot directories to sftp/ftp file listing
2021-07-31 09:42:23 +02:00
Nicola Murino
f778e47d22
sftpd: minor improvements and docs for the prefix middleware
2021-07-29 20:12:23 +02:00
mmcgeefeedo
3ae8abda9e
sftpd: add folder prefix middleware
2021-07-29 00:32:55 +02:00
Nicola Murino
c41ae116eb
improve logging
...
Fixes #381
2021-07-24 20:11:17 +02:00
Nicola Murino
bd5191dfc5
add experimental plugin system
2021-07-11 15:26:51 +02:00
Nicola Murino
076b2f0ee0
modules: add v2 support
2021-06-26 07:31:41 +02:00
Nicola Murino
c1b862394d
move other errors to utils package
2021-06-19 13:06:01 +02:00
Nicola Murino
3bb0ca1d2b
config: remove deprecated configuration keys
2021-06-19 09:47:06 +02:00
Nicola Murino
9d3d7db29c
azblob: store SAS URL as kms.Secret
2021-06-11 22:27:36 +02:00
Nicola Murino
575bcf1f03
add remote address to transfer and commands logs
2021-06-01 22:28:43 +02:00
Nicola Murino
969c992bfd
pre-upload: execute the hook just before opening the target file
2021-05-31 22:40:47 +02:00
Nicola Murino
c1239fbf59
pre-upload action: add file open flags
...
Reading the flags the hook receiver can detect if the client wants to
truncate the target file
2021-05-31 22:33:23 +02:00
Nicola Murino
c63b923ec3
cryptfs: add support for atomic uploads
2021-05-31 21:45:29 +02:00
Nicola Murino
423d8306be
webclient: allow to download multiple files as zip
2021-05-30 23:07:46 +02:00
Nicola Murino
25a44030f9
actions: add pre-download and pre-upload
...
Downloads and uploads can be denied based on hook response
2021-05-26 07:48:37 +02:00
Nicola Murino
50e441849a
try to make the web admin more user friendly
...
removed all the textarea with fields separated using "::".
This should, hopefully, improve user experience
2021-05-23 22:02:01 +02:00
Nicola Murino
02bb09ec01
remove deprecated file extensions filters
...
these filters were deprecated a long time ago, everyone should use
patterns filters now
2021-05-22 12:28:05 +02:00
Nicola Murino
0cb5c49cf3
map path resolution errors to Permission errors
...
this way the affected paths will be ignored in WebDAV
Fixes #432
2021-05-21 13:04:22 +02:00
Nicola Murino
f59f62317e
sftpd: fix file upload resume detection
...
WinSCP does not set the APPEND flag while resuming a file upload,
so we detect a file upload resume if the TRUNCATE flag is not set.
The APPEND flag is now ignored.
Fixes #420
2021-05-15 08:39:01 +02:00
Nicola Murino
f2b93c0402
add a setup screen to create the first admin user
...
If you prefer to auto-create the first admin you can enable the
"create_default_admin" configuration key and SFTPGo will work as before.
You can also create the first admin by loading initial data: now you can
set both username and password, before you could only change the password
2021-05-14 19:21:15 +02:00
Nicola Murino
fa45c9c138
allow to execute actions for file operations and SSH commands synchronously
...
The actions to run synchronously can be configured via the `execute_sync`
configuration key.
Executing an action synchronously means that SFTPGo will not return a result
code to the client until your hook have completed its execution.
Fixes #409
2021-05-11 12:45:14 +02:00
Nicola Murino
b67cd0d3df
ensure no client is connected before running max connections test cases
2021-05-11 08:04:57 +02:00
Nicola Murino
c8f7fc9bc9
httpd/webdav: add a list of hosts allowed to send proxy headers
...
X-Forwarded-For, X-Real-IP and X-Forwarded-Proto headers will be ignored
for hosts not included in this list.
This is a backward incompatible change, before the proxy headers were
always used
2021-05-11 06:54:06 +02:00
Nicola Murino
8f6cdacd00
allow to limit the number of per-host connections
2021-05-08 19:45:21 +02:00
Nicola Murino
5be4b6bd44
localfs: fix subdir check if the user has the root dir as home
2021-04-25 14:36:29 +02:00
Nicola Murino
f4369cdbef
fix max connections check
...
Also make sure to close the ssh client connection in test cases
2021-04-20 18:12:16 +02:00
Nicola Murino
6ef85d6026
add, optional, in memory password caching
...
Verifying argon2 passwords has a high memory and computational cost,
by enabling, in memory, password caching you reduce this cost
2021-04-20 09:39:36 +02:00
Nicola Murino
f45c89fc46
add rate limiting support for REST API/web admin too
2021-04-19 08:14:04 +02:00
Nicola Murino
112e3b2fc2
add rate limiting support
2021-04-18 12:31:06 +02:00
Nicola Murino
74e0223eb9
remove sha256-simd usage
...
sha256-simd is now deprecated
https://github.com/minio/sha256-simd/issues/58
This could slow down sha256 computation on some CPU
2021-04-05 18:23:40 +02:00
Nicola Murino
0823928f98
allow to disable login filesystem checks
...
SFTPGo requires that the user's home directory, virtual folder root,
and intermediate paths to virtual folders exist to work properly.
If you already know that the required directories exist, disabling
these checks will speed up login.
2021-04-05 17:57:30 +02:00
Nicola Murino
fdf3f23df5
allow to disable some hooks on a per-user basis
...
This way you can, for example, mix external and internal users
2021-04-04 22:32:25 +02:00
Nicola Murino
ea26d7786c
sftpfs: add buffering support
...
this way we improve performance over high latency networks
2021-04-03 16:00:55 +02:00
Nicola Murino
2f56375121
improve SFTP loop detection
2021-04-01 18:53:48 +02:00
Nicola Murino
5f49af1780
external auth: allow to inspect and preserve an existing user
2021-03-26 15:19:01 +01:00
Nicola Murino
54c0c1b80d
Windows: manually check if we can bind on the configured port/ports
...
Windows allows the coexistence of three types of sockets on the same
transport-layer service port, for example, 127.0.0.1:8080, [::1]:8080
and [::ffff:0.0.0.0]:8080
Go don't properly handles this, so we use a ugly hack
Fixes #350
2021-03-21 22:21:04 +01:00
Nicola Murino
d6dc3a507e
extend virtual folders support to all storage backends
...
Fixes #241
2021-03-21 19:15:47 +01:00
Nicola Murino
0286da2356
try to auto create virtual folders if missing
2021-03-10 22:30:56 +01:00
Nicola Murino
055506e518
sftpfs: add an option to disable concurrent reads
2021-03-06 15:41:40 +01:00
Nicola Murino
895117718e
SSH system command: add os separator to the resolved path when appropriate
...
Fixes #327
2021-03-01 22:10:45 +01:00
Nicola Murino
a6e36e7cad
FTP: improve TLS certificate authentication
...
For each user you can now configure:
- TLS certificate auth
- TLS certificate auth and password
- Password auth
For TLS auth, the certificate common name must match the name provided
using the "USER" FTP command
2021-02-28 12:10:40 +01:00
Nicola Murino
ca3e15578e
Use new methods in the io and os packages instead of ioutil ones
...
ioutil is deprecated in Go 1.16 and SFTPGo is an application, not
a library, we have no reason to keep compatibility with old Go
versions.
Go 1.16 fix some cifs related issues too.
2021-02-25 21:53:04 +01:00
Nicola Murino
3e1b07324d
GCS: remove compat code
2021-02-22 22:06:23 +01:00
Nicola Murino
be9230e85b
micro optimizations spotted using the go-critic linter
2021-02-16 19:11:36 +01:00
Nicola Murino
46176a54b4
minor doc fixes
2021-02-14 22:08:08 +01:00
Nicola Murino
a21ccad174
web hooks: add mutual TLS support
2021-02-13 14:41:37 +01:00
Nicola Murino
51f110bc7b
sftpd: add statvfs@openssh.com support
2021-02-11 19:45:52 +01:00
Nicola Murino
1cde50f050
sftpd: improve logging if filesystem creation fails
2021-02-03 09:45:04 +01:00
Nicola Murino
78bf808322
virtual folders: change dataprovider structure
...
This way we no longer depend on the local file system path and so we can
add support for cloud backends in future updates
2021-02-01 19:04:15 +01:00
Nicola Murino
46ab8f8d78
post-login hook: add the full user JSON serialized
...
Fixes #284
2021-01-26 18:05:44 +01:00
Nicola Murino
2b9ba1d520
web admin: try to uniform UI
2021-01-23 09:28:45 +01:00
Nicola Murino
aff75953e3
ssh requests: send a reply only if the client requested it
2021-01-21 09:28:41 +01:00
Nicola Murino
c0e09374a8
scp: fix wildcard uploads
...
Fixes #285
2021-01-20 22:37:59 +01:00
Nicola Murino
57976b4085
httpd: add mTLS and multiple bindings support
2021-01-19 18:59:41 +01:00
Nicola Murino
41a1af863e
OpenAPI: minor changes
2021-01-18 13:24:38 +01:00
Nicola Murino
778ec9b88f
REST API v2
...
- add JWT authentication
- admins are now stored inside the data provider
- admin access can be restricted based on the source IP: both proxy
header and connection IP are checked
- deprecate REST API CLI: it is not relevant anymore
Some other changes to the REST API can still happen before releasing
SFTPGo 2.0.0
Fixes #197
2021-01-17 22:29:08 +01:00
Nicola Murino
a8a17a223a
scp: minor improvements
...
document that we don't support wildcard expansion.
I should refactor scp code ...
2021-01-05 22:32:30 +01:00
Nicola Murino
daac90c4e1
fix a potential race condition for pre-login and ext auth
...
hooks
doing something like this:
err = provider.updateUser(u)
...
return provider.userExists(username)
could be racy if another update happen before
provider.userExists(username)
also pass a pointer to updateUser so if the user is modified inside
"validateUser" we can just return the modified user without do a new
query
2021-01-05 09:50:22 +01:00
Nicola Murino
037d89a320
add support for a basic built-in defender
...
It can help to prevent DoS and brute force password guessing
2021-01-02 14:05:09 +01:00
Nicola Murino
1dce1eff48
improve FTP support
...
- allow to disable active mode
- allow to disable SITE commands
- add optional support for calculating hash value of files
- add optional support for the non standard COMB command
2020-12-24 18:48:06 +01:00
Nicola Murino
187a5b1908
sftpd: properly handle listener accept errors
...
continue on temporary errors and exit from the serve loop for the
other ones
2020-12-23 19:53:07 +01:00
Nicola Murino
c69d63c1f8
add support for multiple bindings
...
Fixes #253
2020-12-23 16:12:30 +01:00
Nicola Murino
143df87fee
add some docs for telemetry server
...
move pprof to the telemetry server only
2020-12-18 09:47:22 +01:00
Nicola Murino
f34462e3c3
add support for limiting max concurrent client connections
2020-12-15 19:29:30 +01:00
Nicola Murino
ed43ddd79d
enable hash commands for any supported backend
2020-12-13 15:11:55 +01:00
Nicola Murino
a6985075b9
add sftpfs storage backend
...
Fixes #224
2020-12-12 10:31:09 +01:00
Nicola Murino
50982229e1
REST API: add a method to get the status of the services
...
added a status page to the built-in web admin
2020-12-08 11:18:34 +01:00
Nicola Murino
4a88ea5c03
add Data At Rest Encryption support
2020-12-05 13:48:13 +01:00
Nicola Murino
634b723b5d
add KMS support
...
Fixes #226
2020-11-30 21:46:34 +01:00
Nicola Murino
dccc583b5d
add a dedicated struct to store encrypted credentials
...
also gcs credentials are now encrypted, both on disk and inside the
provider.
Data provider is automatically migrated and load data will accept
old format too but you should upgrade to the new format to avoid future
issues
2020-11-22 21:53:04 +01:00
Nicola Murino
5d789a01b7
update pkg/sftp
...
These patches are now merged upstream:
https://github.com/pkg/sftp/pull/392
https://github.com/pkg/sftp/pull/393
2020-11-18 19:06:12 +01:00
Nicola Murino
ca0ff0d630
add a File interface so we can avoid to use os.File directly
2020-11-17 19:36:39 +01:00
Nicola Murino
a6355e298e
add support for limit files using shell like patterns
...
Fixes #209
2020-11-15 22:04:48 +01:00
Nicola Murino
dc845fa2f4
webdav: fix permission errors if the client try to read multiple times
2020-11-14 19:19:41 +01:00
Nicola Murino
5720d40fee
add setstat_mode 2
...
in this mode chmod/chtimes/chown can be silently ignored only for cloud
based file systems
Fixes #223
2020-11-12 10:39:46 +01:00
Nicola Murino
4c5a0d663e
sftpd: return the error Operation Unsupported for unexpected reads
...
a cloud based file cannot be opened for read and write at the same
time. Return a proper error if a client try to do this.
It can happen only for SFTP
2020-11-09 21:01:56 +01:00
Nicola Murino
6ad4cc317c
cloud backends: stat and other performance improvements
2020-11-02 19:16:12 +01:00
Nicola Murino
950a5ad9ea
add a recoverer where appropriate
...
I have never seen this, but a malformed packet can easily crash pkg/sftp
2020-10-31 11:02:04 +01:00
Nicola Murino
ac3bae00fc
add support for SFTP subsystem mode
...
Fixes #204
2020-10-29 19:23:33 +01:00
Nicola Murino
975a2f3632
sftpd: fix the max upload file size check for overwrites
...
improved test case too
2020-10-25 08:52:31 +01:00
Nicola Murino
5ff8f75917
add Azure Blob support
2020-10-25 08:18:48 +01:00
Sean Hildebrand
db7e81e9d0
add prefer_database_credentials configuration parameter
...
When true, users' Google Cloud Storage credentials will be written to
the data provider instead of disk.
Pre-existing credentials on disk will be used as a fallback
Fixes #201
2020-10-22 10:42:40 +02:00
Nicola Murino
6a8039e76a
sftpd: log fingerprints for used host keys
2020-10-21 14:27:58 +02:00
