Nicola Murino
43182fc25e
OpenAPI: add users API
...
These new APIs match the web client features.
I'm aware that some API do not follow REST best practises.
I want to avoid things likes "/user/folders/<path>"
where "path" must be encoded and making it optional create issues, so
I defined resources as query parameters instead of path parameters
2021-06-05 16:07:09 +02:00
Nicola Murino
575bcf1f03
add remote address to transfer and commands logs
2021-06-01 22:28:43 +02:00
Nicola Murino
c1239fbf59
pre-upload action: add file open flags
...
Reading the flags the hook receiver can detect if the client wants to
truncate the target file
2021-05-31 22:33:23 +02:00
Nicola Murino
c63b923ec3
cryptfs: add support for atomic uploads
2021-05-31 21:45:29 +02:00
Nicola Murino
423d8306be
webclient: allow to download multiple files as zip
2021-05-30 23:07:46 +02:00
Nicola Murino
25a44030f9
actions: add pre-download and pre-upload
...
Downloads and uploads can be denied based on hook response
2021-05-26 07:48:37 +02:00
Nicola Murino
02bb09ec01
remove deprecated file extensions filters
...
these filters were deprecated a long time ago, everyone should use
patterns filters now
2021-05-22 12:28:05 +02:00
Nicola Murino
15d6cd144a
another try to better understand the random webdav test case failure
2021-05-15 08:56:36 +02:00
Nicola Murino
f59f62317e
sftpd: fix file upload resume detection
...
WinSCP does not set the APPEND flag while resuming a file upload,
so we detect a file upload resume if the TRUNCATE flag is not set.
The APPEND flag is now ignored.
Fixes #420
2021-05-15 08:39:01 +02:00
Nicola Murino
f2b93c0402
add a setup screen to create the first admin user
...
If you prefer to auto-create the first admin you can enable the
"create_default_admin" configuration key and SFTPGo will work as before.
You can also create the first admin by loading initial data: now you can
set both username and password, before you could only change the password
2021-05-14 19:21:15 +02:00
Nicola Murino
fa45c9c138
allow to execute actions for file operations and SSH commands synchronously
...
The actions to run synchronously can be configured via the `execute_sync`
configuration key.
Executing an action synchronously means that SFTPGo will not return a result
code to the client until your hook have completed its execution.
Fixes #409
2021-05-11 12:45:14 +02:00
Nicola Murino
b67cd0d3df
ensure no client is connected before running max connections test cases
2021-05-11 08:04:57 +02:00
Nicola Murino
c8f7fc9bc9
httpd/webdav: add a list of hosts allowed to send proxy headers
...
X-Forwarded-For, X-Real-IP and X-Forwarded-Proto headers will be ignored
for hosts not included in this list.
This is a backward incompatible change, before the proxy headers were
always used
2021-05-11 06:54:06 +02:00
Nicola Murino
15934d72cc
webdav test: increase log size
...
the latest 10 lines are not enough to understand the issue, try with 20
2021-05-09 10:09:25 +02:00
Nicola Murino
8f6cdacd00
allow to limit the number of per-host connections
2021-05-08 19:45:21 +02:00
Nicola Murino
8f736da4b8
webdav test: add some more logs
...
QuotaLimits test case sometime fails when running in CI, try to
understand the reason
2021-05-07 22:24:06 +02:00
Nicola Murino
4ea4202b99
httpd/webdav: use a custom listener with read and write deadlines
2021-05-07 20:41:20 +02:00
Nicola Murino
23d9ebfc91
add a basic front-end web interface for end-users
...
Fixes #339 #321 #398
2021-05-06 21:35:43 +02:00
Nicola Murino
f4369cdbef
fix max connections check
...
Also make sure to close the ssh client connection in test cases
2021-04-20 18:12:16 +02:00
Nicola Murino
6ef85d6026
add, optional, in memory password caching
...
Verifying argon2 passwords has a high memory and computational cost,
by enabling, in memory, password caching you reduce this cost
2021-04-20 09:39:36 +02:00
Nicola Murino
47317bed9b
make sure that Retry-After header has a value greater than zero
2021-04-19 09:16:27 +02:00
Nicola Murino
f45c89fc46
add rate limiting support for REST API/web admin too
2021-04-19 08:14:04 +02:00
Nicola Murino
112e3b2fc2
add rate limiting support
2021-04-18 12:31:06 +02:00
Nicola Murino
ea26d7786c
sftpfs: add buffering support
...
this way we improve performance over high latency networks
2021-04-03 16:00:55 +02:00
Nicola Murino
2f56375121
improve SFTP loop detection
2021-04-01 18:53:48 +02:00
Nicola Murino
9ad750da54
WebDAV: try to preserve the lock fs as much as possible
2021-03-27 19:10:27 +01:00
Nicola Murino
6bc5c64a3a
webdav: ignore path, perm and not exist errors in PROPFIND
...
Fixes #340
2021-03-24 13:32:20 +01:00
Nicola Murino
5e375f56dd
kms: add a lock, secrets could be modified concurrently for cached users
...
also reduce the size of the JSON payload omitting empty secrets
2021-03-22 19:03:25 +01:00
Nicola Murino
28f1d66ae5
link the Active Directory example in the howto section
2021-03-22 09:52:05 +01:00
Nicola Murino
800e64404b
update deps
2021-03-22 08:55:35 +01:00
Nicola Murino
54c0c1b80d
Windows: manually check if we can bind on the configured port/ports
...
Windows allows the coexistence of three types of sockets on the same
transport-layer service port, for example, 127.0.0.1:8080, [::1]:8080
and [::ffff:0.0.0.0]:8080
Go don't properly handles this, so we use a ugly hack
Fixes #350
2021-03-21 22:21:04 +01:00
Nicola Murino
d6dc3a507e
extend virtual folders support to all storage backends
...
Fixes #241
2021-03-21 19:15:47 +01:00
Nicola Murino
4c658bb6f0
webdav: add prefix support
2021-03-07 17:10:45 +01:00
Nicola Murino
534b253c20
WebDAV: improve TLS certificate authentication
...
For each user you can now configure:
- TLS certificate auth
- TLS certificate auth and password
- Password auth
For TLS certificate auth, the certificate common name is used as
username
2021-03-01 19:28:11 +01:00
Nicola Murino
ca3e15578e
Use new methods in the io and os packages instead of ioutil ones
...
ioutil is deprecated in Go 1.16 and SFTPGo is an application, not
a library, we have no reason to keep compatibility with old Go
versions.
Go 1.16 fix some cifs related issues too.
2021-02-25 21:53:04 +01:00
Nicola Murino
3e1b07324d
GCS: remove compat code
2021-02-22 22:06:23 +01:00
Nicola Murino
5da4f931c5
TLS: allow to configure cipher suites
...
Fixes #316
2021-02-18 20:17:16 +01:00
Nicola Murino
be9230e85b
micro optimizations spotted using the go-critic linter
2021-02-16 19:11:36 +01:00
Nicola Murino
46176a54b4
minor doc fixes
2021-02-14 22:08:08 +01:00
Nicola Murino
a21ccad174
web hooks: add mutual TLS support
2021-02-13 14:41:37 +01:00
Nicola Murino
51f110bc7b
sftpd: add statvfs@openssh.com support
2021-02-11 19:45:52 +01:00
Nicola Murino
267d9f1831
web ui: allow to create folders from a template
2021-02-04 19:09:43 +01:00
Nicola Murino
17a42a0c11
webdav: add compression support
...
Fixes #295
2021-02-04 09:06:41 +01:00
Nicola Murino
ce731020a7
webdav: remove the username path prefix
...
so we have the same URIs for all protocols
Fixes #293
2021-02-04 07:12:04 +01:00
Nicola Murino
fc9082c422
webdav: try to handle HEAD for collection too
...
The underlying golang webdav library returns Method Not Allowed for
HEAD requests on directories:
https://github.com/golang/net/blob/master/webdav/webdav.go#L210
let's see if we can workaround this inside SFTPGo itself in a similar
way as we do for GET.
The HEAD response will not return a Content-Length, we cannot handle
this inside SFTPGo.
Fixes #294
2021-02-03 22:36:13 +01:00
Nicola Murino
78bf808322
virtual folders: change dataprovider structure
...
This way we no longer depend on the local file system path and so we can
add support for cloud backends in future updates
2021-02-01 19:04:15 +01:00
Nicola Murino
46ab8f8d78
post-login hook: add the full user JSON serialized
...
Fixes #284
2021-01-26 18:05:44 +01:00
Nicola Murino
57976b4085
httpd: add mTLS and multiple bindings support
2021-01-19 18:59:41 +01:00
Nicola Murino
778ec9b88f
REST API v2
...
- add JWT authentication
- admins are now stored inside the data provider
- admin access can be restricted based on the source IP: both proxy
header and connection IP are checked
- deprecate REST API CLI: it is not relevant anymore
Some other changes to the REST API can still happen before releasing
SFTPGo 2.0.0
Fixes #197
2021-01-17 22:29:08 +01:00
Nicola Murino
daac90c4e1
fix a potential race condition for pre-login and ext auth
...
hooks
doing something like this:
err = provider.updateUser(u)
...
return provider.userExists(username)
could be racy if another update happen before
provider.userExists(username)
also pass a pointer to updateUser so if the user is modified inside
"validateUser" we can just return the modified user without do a new
query
2021-01-05 09:50:22 +01:00
Nicola Murino
684f4ba1a6
mutal TLS: add support for revocation lists
2021-01-03 17:03:04 +01:00
Nicola Murino
6d84c5b9e3
capture http servers error logs
...
otherwise they will be printed to stdout
2021-01-03 10:38:28 +01:00
Nicola Murino
4b522a2455
webdav: refactor server initialization
2021-01-03 09:51:54 +01:00
Nicola Murino
037d89a320
add support for a basic built-in defender
...
It can help to prevent DoS and brute force password guessing
2021-01-02 14:05:09 +01:00
Nicola Murino
141ca6777c
webdav: add support for client certificate authentication
...
Fixes #263
2020-12-28 19:48:23 +01:00
Nicola Murino
1dce1eff48
improve FTP support
...
- allow to disable active mode
- allow to disable SITE commands
- add optional support for calculating hash value of files
- add optional support for the non standard COMB command
2020-12-24 18:48:06 +01:00
Nicola Murino
187a5b1908
sftpd: properly handle listener accept errors
...
continue on temporary errors and exit from the serve loop for the
other ones
2020-12-23 19:53:07 +01:00
Nicola Murino
c69d63c1f8
add support for multiple bindings
...
Fixes #253
2020-12-23 16:12:30 +01:00
Nicola Murino
143df87fee
add some docs for telemetry server
...
move pprof to the telemetry server only
2020-12-18 09:47:22 +01:00
Nicola Murino
f34462e3c3
add support for limiting max concurrent client connections
2020-12-15 19:29:30 +01:00
Nicola Murino
a6985075b9
add sftpfs storage backend
...
Fixes #224
2020-12-12 10:31:09 +01:00
Nicola Murino
50982229e1
REST API: add a method to get the status of the services
...
added a status page to the built-in web admin
2020-12-08 11:18:34 +01:00
Nicola Murino
034d89876d
webdav: fix proppatch handling
...
also respect login delay for cached webdav users and check the home dir as
soon as the user authenticates
Fixes #239
2020-12-06 08:19:41 +01:00
Nicola Murino
4a88ea5c03
add Data At Rest Encryption support
2020-12-05 13:48:13 +01:00
Nicola Murino
634b723b5d
add KMS support
...
Fixes #226
2020-11-30 21:46:34 +01:00
Nicola Murino
0609188d3f
allow to disable SFTP service
...
Fixes #228
2020-11-24 13:44:57 +01:00
Nicola Murino
dccc583b5d
add a dedicated struct to store encrypted credentials
...
also gcs credentials are now encrypted, both on disk and inside the
provider.
Data provider is automatically migrated and load data will accept
old format too but you should upgrade to the new format to avoid future
issues
2020-11-22 21:53:04 +01:00
Nicola Murino
ca0ff0d630
add a File interface so we can avoid to use os.File directly
2020-11-17 19:36:39 +01:00
Nicola Murino
a6355e298e
add support for limit files using shell like patterns
...
Fixes #209
2020-11-15 22:04:48 +01:00
Nicola Murino
dc845fa2f4
webdav: fix permission errors if the client try to read multiple times
2020-11-14 19:19:41 +01:00
Nicola Murino
1d5d184720
webdav file: ensure to close the reader only once
2020-11-05 09:30:38 +01:00
Nicola Murino
0119fd03a6
webdav: user caching is now mandatory
...
we cache the lock system with the user, without user caching we cannot
support locks for resource
2020-11-04 22:29:25 +01:00
Nicola Murino
0a14297b48
webdav: performance improvements and bug fixes
...
we need my custom golang/x/net/webdav fork for now
https://github.com/drakkan/net/tree/sftpgo
2020-11-04 19:11:40 +01:00
Nicola Murino
950a5ad9ea
add a recoverer where appropriate
...
I have never seen this, but a malformed packet can easily crash pkg/sftp
2020-10-31 11:02:04 +01:00
Nicola Murino
ac3bae00fc
add support for SFTP subsystem mode
...
Fixes #204
2020-10-29 19:23:33 +01:00
Nicola Murino
975a2f3632
sftpd: fix the max upload file size check for overwrites
...
improved test case too
2020-10-25 08:52:31 +01:00
Nicola Murino
5ff8f75917
add Azure Blob support
2020-10-25 08:18:48 +01:00
Sean Hildebrand
db7e81e9d0
add prefer_database_credentials configuration parameter
...
When true, users' Google Cloud Storage credentials will be written to
the data provider instead of disk.
Pre-existing credentials on disk will be used as a fallback
Fixes #201
2020-10-22 10:42:40 +02:00
Mark Sagi-Kazar
5e2db77ef9
refactor: add an enum for filesystem providers
...
Signed-off-by: Mark Sagi-Kazar <mark.sagikazar@gmail.com>
2020-10-05 21:40:21 +02:00
Nicola Murino
d1f0e9ae9f
CGS: implement MimeTyper interface
2020-09-28 22:12:46 +02:00
Nicola Murino
2df0dd1f70
sshd: map each channel with a new connection
...
Fixes #169
2020-09-18 10:52:53 +02:00
Nicola Murino
3925c7ff95
REST API/Web admin: add a parameter to disconnect a user after an update
...
This way you can force the user to login again and so to use the updated
configuration.
A deleted user will be automatically disconnected.
Fixes #163
Improved some docs too.
2020-09-01 16:10:26 +02:00
Nicola Murino
dbed110d02
WebDAV: add caching for authenticated users
...
In this way we get a big performance boost
2020-08-31 19:25:17 +02:00
Nicola Murino
2746c0b0f1
move stat to base connection and differentiate between Stat and Lstat
...
we will use Lstat once it will be exposed in pkg/sftp
2020-08-25 18:23:00 +02:00
Nicola Murino
02e35ee002
sftpd: add Readlink support
2020-08-22 14:52:17 +02:00
Nicola Murino
5208e4a4ca
sftpd: improve truncate
...
quota usage and max allowed write size are now properly updated after a
truncate
2020-08-22 10:12:00 +02:00
Nicola Murino
8b0a1817b3
add check password hook
...
its main use case is to allow to easily support things like password+OTP for
protocols without keyboard interactive support such as FTP and WebDAV
2020-08-19 19:36:12 +02:00
Nicola Murino
bbc8c091e6
portable mode: add WebDAV support
2020-08-17 14:08:08 +02:00
Nicola Murino
f3228713bc
Allow individual protocols to be enabled per user
...
Fixes #154
2020-08-17 12:49:20 +02:00
Nicola Murino
fa5333784b
add a maximum allowed size for a single upload
2020-08-16 20:17:02 +02:00
Nicola Murino
0dbf0cc81f
WebDAV: add CORS support
2020-08-15 15:55:20 +02:00
Nicola Murino
196a56726e
FTP improvements
...
- add a setting to require TLS
- add symlink support
require TLS 1.2 for all TLS connections
2020-08-15 13:02:25 +02:00
Nicola Murino
aa0ed5dbd0
add post-login hook
...
a login scope is supported too so you can get notifications for failed logins,
successful logins or both
2020-08-12 16:15:12 +02:00
Nicola Murino
a9e21c282a
add WebDAV support
...
Fixes #147
2020-08-11 23:56:10 +02:00
