Nicola Murino
531cb5b5a1
sftpd: handle setstat requests with multiple attrs
2021-11-24 11:55:14 +01:00
Nicola Murino
e29a3efd39
add resetprovider sub-command
...
Fixes #608
2021-11-15 18:40:31 +01:00
Nicola Murino
ca730e77a5
add separate permissions to delete and rename files and dirs
...
perm_delete and perm_rename still exist for backward compatibility,
now they are an alias to assign both new split permissions
2021-11-14 16:23:33 +01:00
Nicola Murino
094ee1522e
logger: add a flag to use UTC time for logging
2021-11-06 15:18:16 +01:00
Nicola Murino
3bc58f5988
WebClient/REST API: add sharing support
2021-11-06 14:13:20 +01:00
Nicola Murino
570964deb3
add post-disconnect hook
...
Fixes #587
2021-10-29 19:55:18 +02:00
Nicola Murino
97d0a48557
plugins: improve notifier and searcher
2021-10-20 19:39:49 +02:00
Nicola Murino
a80ac80fcd
pkgs: update nfpm to 2.7 and use xz as compression for both deb and rpm
2021-10-13 09:15:04 +02:00
Nicola Murino
4aa9686e3b
refactor custom actions
...
SFTPGo is now fully auditable, all fs and provider events that change
something are notified and can be collected using hooks/plugins.
There are some backward incompatible changes for command hooks
2021-10-10 13:08:05 +02:00
Nicola Murino
ea01c3a125
rate limiting: allow to exclude IP addresses/ranges
...
Fixes #563
2021-10-03 20:50:05 +02:00
Nicola Murino
1b4a1fbbe5
add data retention check hook
2021-10-03 15:17:49 +02:00
Nicola Murino
ec81a7ac29
actions: add a specific protocol for data retention
2021-10-03 10:22:47 +02:00
Nicola Murino
cc134cad9a
data retention: allow to notify results via e-mail
2021-10-02 22:25:41 +02:00
Nicola Murino
da5a061b65
add basic REST APIs for data retention
...
Fixes #495
2021-09-25 12:20:31 +02:00
Nicola Murino
5c34d814d6
fix a possible nil pointer dereference
...
it can happen by upgrading from very old versions
2021-09-11 14:19:17 +02:00
Nicola Murino
0eca4f1866
update deps
2021-09-08 12:29:47 +02:00
Nicola Murino
8a4c21b64a
add builtin two-factor auth support
...
The builtin two-factor authentication is based on time-based one time
passwords (RFC 6238) which works with Authy, Google Authenticator and
other compatible apps.
2021-09-04 12:11:04 +02:00
Nicola Murino
bd9506da42
BaseConnection struct: ensure 64 bit alignment
...
Fixes #516
2021-08-28 10:06:49 +02:00
Nicola Murino
b903a6e46f
data provider: remove default admin
...
you need to load initial data or set "create_default_admin" to true
and the appropriate env vars if you don't want to use the web admin
setup screen to create the default admin
2021-08-20 10:37:51 +02:00
Nicola Murino
555dc3b0c0
transfer logs: add FTP mode
2021-08-10 13:07:38 +02:00
Nicola Murino
0de0d3308c
improve error messages for generic failures
2021-08-08 19:30:21 +02:00
Nicola Murino
a20373b613
add support for auth plugins
2021-08-08 17:09:48 +02:00
Nicola Murino
a3c087456b
ftpd: add some security checks
2021-08-05 18:38:15 +02:00
Nicola Murino
c41ae116eb
improve logging
...
Fixes #381
2021-07-24 20:11:17 +02:00
Nicola Murino
c900cde8e4
notifiers plugin: add settings to retry unhandled events
2021-07-20 12:51:21 +02:00
Nicola Murino
bd5191dfc5
add experimental plugin system
2021-07-11 15:26:51 +02:00
Nicola Murino
302ec2558c
add notifications for mkdir/rmdir
2021-07-03 18:07:55 +02:00
Nicola Murino
04001f7ad3
FTP: try to return more specific error codes/messages for some errors
...
We now return 552 code for quota exceeded errors and 553 in the following
cases:
- filename denied by a filter
- no upload permission
- no overwrite permission
- pre upload hook error
Fixes #442
2021-06-28 19:40:04 +02:00
Nicola Murino
076b2f0ee0
modules: add v2 support
2021-06-26 07:31:41 +02:00
Nicola Murino
e09bdd43d4
defender: fix GetHost for blocklist entries too
2021-06-20 21:57:19 +02:00
Nicola Murino
81aac15a6c
defender: don't return expired hosts/banned ip in GetHost too
2021-06-19 18:51:33 +02:00
Nicola Murino
c1b862394d
move other errors to utils package
2021-06-19 13:06:01 +02:00
Nicola Murino
f2f612b450
defender: don't return expired hosts/banned ip
2021-06-19 11:02:46 +02:00
Nicola Murino
9d3d7db29c
azblob: store SAS URL as kms.Secret
2021-06-11 22:27:36 +02:00
Nicola Murino
feec2118bb
improve defender and quotas REST API
2021-06-07 21:52:43 +02:00
Nicola Murino
43182fc25e
OpenAPI: add users API
...
These new APIs match the web client features.
I'm aware that some API do not follow REST best practises.
I want to avoid things likes "/user/folders/<path>"
where "path" must be encoded and making it optional create issues, so
I defined resources as query parameters instead of path parameters
2021-06-05 16:07:09 +02:00
Nicola Murino
575bcf1f03
add remote address to transfer and commands logs
2021-06-01 22:28:43 +02:00
Nicola Murino
c1239fbf59
pre-upload action: add file open flags
...
Reading the flags the hook receiver can detect if the client wants to
truncate the target file
2021-05-31 22:33:23 +02:00
Nicola Murino
c63b923ec3
cryptfs: add support for atomic uploads
2021-05-31 21:45:29 +02:00
Nicola Murino
3b46e6a6fb
add support for a global temp path
...
Fixes #436
2021-05-27 15:38:27 +02:00
Nicola Murino
25a44030f9
actions: add pre-download and pre-upload
...
Downloads and uploads can be denied based on hook response
2021-05-26 07:48:37 +02:00
Nicola Murino
600268ebb8
httpclient: allow to set custom headers
2021-05-25 08:36:01 +02:00
Nicola Murino
f2b93c0402
add a setup screen to create the first admin user
...
If you prefer to auto-create the first admin you can enable the
"create_default_admin" configuration key and SFTPGo will work as before.
You can also create the first admin by loading initial data: now you can
set both username and password, before you could only change the password
2021-05-14 19:21:15 +02:00
Nicola Murino
0540b8780e
redact credentials within hooks
...
go-retryablehttp does not redact credentials, so we still log them
when we use it
https://github.com/hashicorp/go-retryablehttp/pull/133
2021-05-12 22:44:17 +02:00
Nicola Murino
fa45c9c138
allow to execute actions for file operations and SSH commands synchronously
...
The actions to run synchronously can be configured via the `execute_sync`
configuration key.
Executing an action synchronously means that SFTPGo will not return a result
code to the client until your hook have completed its execution.
Fixes #409
2021-05-11 12:45:14 +02:00
Nicola Murino
b67cd0d3df
ensure no client is connected before running max connections test cases
2021-05-11 08:04:57 +02:00
Nicola Murino
c8f7fc9bc9
httpd/webdav: add a list of hosts allowed to send proxy headers
...
X-Forwarded-For, X-Real-IP and X-Forwarded-Proto headers will be ignored
for hosts not included in this list.
This is a backward incompatible change, before the proxy headers were
always used
2021-05-11 06:54:06 +02:00
Nicola Murino
8f6cdacd00
allow to limit the number of per-host connections
2021-05-08 19:45:21 +02:00
Nicola Murino
23d9ebfc91
add a basic front-end web interface for end-users
...
Fixes #339 #321 #398
2021-05-06 21:35:43 +02:00
Nicola Murino
46998252e5
use bcrypt as default password hashing algo
...
argon2id has a high memory cost and, if not properly tuned, it can lead to
resource starvation.
Advanced users can still configure and use argon2id.
Passwords stored as argon2id will continue to work
2021-04-25 09:38:33 +02:00
Nicola Murino
f4369cdbef
fix max connections check
...
Also make sure to close the ssh client connection in test cases
2021-04-20 18:12:16 +02:00
Nicola Murino
92638ce93d
add support for hashing password using bcrypt
...
argon2id remains the default
2021-04-20 13:55:09 +02:00
Nicola Murino
6ef85d6026
add, optional, in memory password caching
...
Verifying argon2 passwords has a high memory and computational cost,
by enabling, in memory, password caching you reduce this cost
2021-04-20 09:39:36 +02:00
Nicola Murino
f45c89fc46
add rate limiting support for REST API/web admin too
2021-04-19 08:14:04 +02:00
Nicola Murino
112e3b2fc2
add rate limiting support
2021-04-18 12:31:06 +02:00
Nicola Murino
c844fc7477
add support for delayed quota update
...
If there are a lot of close uploads, accumulating quota updates can
save you many queries to the data provider
2021-04-11 08:38:43 +02:00
Nicola Murino
acb4310c11
add a startup hook
2021-04-05 10:07:59 +02:00
Nicola Murino
ea26d7786c
sftpfs: add buffering support
...
this way we improve performance over high latency networks
2021-04-03 16:00:55 +02:00
Nicola Murino
2f56375121
improve SFTP loop detection
2021-04-01 18:53:48 +02:00
Nicola Murino
f249286cb1
docs: add some notes about the new virtual folders support
...
fixe a failing test case for the memory provider
2021-03-21 19:47:11 +01:00
Nicola Murino
d6dc3a507e
extend virtual folders support to all storage backends
...
Fixes #241
2021-03-21 19:15:47 +01:00
Nicola Murino
ca3e15578e
Use new methods in the io and os packages instead of ioutil ones
...
ioutil is deprecated in Go 1.16 and SFTPGo is an application, not
a library, we have no reason to keep compatibility with old Go
versions.
Go 1.16 fix some cifs related issues too.
2021-02-25 21:53:04 +01:00
Nicola Murino
be9230e85b
micro optimizations spotted using the go-critic linter
2021-02-16 19:11:36 +01:00
Nicola Murino
46176a54b4
minor doc fixes
2021-02-14 22:08:08 +01:00
Nicola Murino
a21ccad174
web hooks: add mutual TLS support
2021-02-13 14:41:37 +01:00
Nicola Murino
1ac66d27b6
Use IEC units for byte counting everywhere
2021-02-12 22:16:35 +01:00
Nicola Murino
6a6e8fffbc
web hooks: improve resilience by adding a configurable retry
...
the retryable http client is used for hooks that notify events
2021-02-12 21:42:49 +01:00
Nicola Murino
51f110bc7b
sftpd: add statvfs@openssh.com support
2021-02-11 19:45:52 +01:00
Nicola Murino
78bf808322
virtual folders: change dataprovider structure
...
This way we no longer depend on the local file system path and so we can
add support for cloud backends in future updates
2021-02-01 19:04:15 +01:00
Nicola Murino
c2bbd468c4
REST API: add logout and store invalidated token
2021-01-26 22:35:36 +01:00
Nicola Murino
46ab8f8d78
post-login hook: add the full user JSON serialized
...
Fixes #284
2021-01-26 18:05:44 +01:00
Nicola Murino
778ec9b88f
REST API v2
...
- add JWT authentication
- admins are now stored inside the data provider
- admin access can be restricted based on the source IP: both proxy
header and connection IP are checked
- deprecate REST API CLI: it is not relevant anymore
Some other changes to the REST API can still happen before releasing
SFTPGo 2.0.0
Fixes #197
2021-01-17 22:29:08 +01:00
Nicola Murino
daac90c4e1
fix a potential race condition for pre-login and ext auth
...
hooks
doing something like this:
err = provider.updateUser(u)
...
return provider.userExists(username)
could be racy if another update happen before
provider.userExists(username)
also pass a pointer to updateUser so if the user is modified inside
"validateUser" we can just return the modified user without do a new
query
2021-01-05 09:50:22 +01:00
Nicola Murino
72b2c83392
defender: allow hot-reloading for safe and block lists
2021-01-04 17:52:14 +01:00
Nicola Murino
684f4ba1a6
mutal TLS: add support for revocation lists
2021-01-03 17:03:04 +01:00
Nicola Murino
1e1c46ae1b
defender: minor docs improvements
2021-01-02 20:02:05 +01:00
Nicola Murino
d6b3acdb62
add REST API for the defender
2021-01-02 19:33:24 +01:00
Nicola Murino
037d89a320
add support for a basic built-in defender
...
It can help to prevent DoS and brute force password guessing
2021-01-02 14:05:09 +01:00
Nicola Murino
141ca6777c
webdav: add support for client certificate authentication
...
Fixes #263
2020-12-28 19:48:23 +01:00
Nicola Murino
2a95d031ea
FTP: add support for AVBL command
2020-12-25 11:14:08 +01:00
Nicola Murino
1dce1eff48
improve FTP support
...
- allow to disable active mode
- allow to disable SITE commands
- add optional support for calculating hash value of files
- add optional support for the non standard COMB command
2020-12-24 18:48:06 +01:00
Nicola Murino
c69d63c1f8
add support for multiple bindings
...
Fixes #253
2020-12-23 16:12:30 +01:00
Nicola Murino
bcf0fa073e
telemetry server: add optional https and authentication
2020-12-18 16:04:42 +01:00
Nicola Murino
65e6d5475f
update ftpserverlib to include the latest fixes and features
2020-12-18 08:49:32 +01:00
Nicola Murino
f34462e3c3
add support for limiting max concurrent client connections
2020-12-15 19:29:30 +01:00
Nicola Murino
ea0bf5e4c8
ensure 64 bit alignment for 64 bit struct fields access atomically
2020-12-14 14:52:36 +01:00
Nicola Murino
a6985075b9
add sftpfs storage backend
...
Fixes #224
2020-12-12 10:31:09 +01:00
Nicola Murino
4a88ea5c03
add Data At Rest Encryption support
2020-12-05 13:48:13 +01:00
Nicola Murino
0609188d3f
allow to disable SFTP service
...
Fixes #228
2020-11-24 13:44:57 +01:00
Nicola Murino
ca0ff0d630
add a File interface so we can avoid to use os.File directly
2020-11-17 19:36:39 +01:00
Nicola Murino
1a6863f4b1
GCS uploads: check Close() error
...
some code simplification too
2020-11-13 18:40:18 +01:00
Nicola Murino
5720d40fee
add setstat_mode 2
...
in this mode chmod/chtimes/chown can be silently ignored only for cloud
based file systems
Fixes #223
2020-11-12 10:39:46 +01:00
Nicola Murino
4edb9cd6b9
simplify some code
2020-11-07 18:05:47 +01:00
Nicola Murino
9b49f63a97
azure: implement multipart uploads using low level API
...
The high level wrapper seems to hang if there are network issues
2020-10-25 17:41:04 +01:00
Nicola Murino
5ff8f75917
add Azure Blob support
2020-10-25 08:18:48 +01:00
Mark Sagi-Kazar
56bf8364cd
test: add test for InitializeActionHandler
...
Signed-off-by: Mark Sagi-Kazar <mark.sagikazar@gmail.com>
2020-10-21 07:23:33 +02:00
Mark Sagi-Kazar
75750e3a79
feat: add support for custom action hooks
...
Signed-off-by: Mark Sagi-Kazar <mark.sagikazar@gmail.com>
2020-10-21 07:23:33 +02:00
Mark Sagi-Kazar
5e2db77ef9
refactor: add an enum for filesystem providers
...
Signed-off-by: Mark Sagi-Kazar <mark.sagikazar@gmail.com>
2020-10-05 21:40:21 +02:00
Nicola Murino
6c1a7449fe
ssh commands: return better error messages
...
This improve the fix for #171 and return better error message for
SSH commands other than SCP too
2020-09-19 10:14:30 +02:00
Nicola Murino
242dde4480
sftpd: ensure to always close idle connections
...
after the last commit this wasn't the case anymore
Completly fixes #169
2020-09-18 18:15:28 +02:00
