Commit graph

58 commits

Author SHA1 Message Date
Nicola Murino
0540b8780e
redact credentials within hooks
go-retryablehttp does not redact credentials, so we still log them
when we use it

https://github.com/hashicorp/go-retryablehttp/pull/133
2021-05-12 22:44:17 +02:00
Nicola Murino
acb4310c11
add a startup hook 2021-04-05 10:07:59 +02:00
Nicola Murino
d5f092284a
improve signals handling 2021-03-25 19:31:21 +01:00
Nicola Murino
ca3e15578e
Use new methods in the io and os packages instead of ioutil ones
ioutil is deprecated in Go 1.16 and SFTPGo is an application, not
a library, we have no reason to keep compatibility with old Go
versions.

Go 1.16 fix some cifs related issues too.
2021-02-25 21:53:04 +01:00
Nicola Murino
2146b83343
data providers: add filesystem to folder ...
... and some descriptive fields.
The filesystem support for virtual folders will be implemented in
future commits
2021-02-24 19:40:29 +01:00
Nicola Murino
a21ccad174
web hooks: add mutual TLS support 2021-02-13 14:41:37 +01:00
Nicola Murino
51f110bc7b
sftpd: add statvfs@openssh.com support 2021-02-11 19:45:52 +01:00
Nicola Murino
4ddfe41f23
loaddata: restore admins too 2021-02-11 08:33:32 +01:00
Nicola Murino
78bf808322
virtual folders: change dataprovider structure
This way we no longer depend on the local file system path and so we can
add support for cloud backends in future updates
2021-02-01 19:04:15 +01:00
Nicola Murino
778ec9b88f
REST API v2
- add JWT authentication
- admins are now stored inside the data provider
- admin access can be restricted based on the source IP: both proxy
  header and connection IP are checked
- deprecate REST API CLI: it is not relevant anymore

Some other changes to the REST API can still happen before releasing
SFTPGo 2.0.0

Fixes #197
2021-01-17 22:29:08 +01:00
Nicola Murino
daac90c4e1
fix a potential race condition for pre-login and ext auth
hooks

doing something like this:

err = provider.updateUser(u)
...
return provider.userExists(username)

could be racy if another update happen before

provider.userExists(username)

also pass a pointer to updateUser so if the user is modified inside
"validateUser" we can just return the modified user without do a new
query
2021-01-05 09:50:22 +01:00
Nicola Murino
037d89a320
add support for a basic built-in defender
It can help to prevent DoS and brute force password guessing
2021-01-02 14:05:09 +01:00
Nicola Murino
0966d44c0f
httpd: add support for listening over a Unix-domain socket
Fixes #266
2020-12-29 19:02:56 +01:00
Nicola Murino
c69d63c1f8
add support for multiple bindings
Fixes #253
2020-12-23 16:12:30 +01:00
Nicola Murino
bcf0fa073e
telemetry server: add optional https and authentication 2020-12-18 16:04:42 +01:00
Nicola Murino
143df87fee
add some docs for telemetry server
move pprof to the telemetry server only
2020-12-18 09:47:22 +01:00
Márk Sági-Kazár
6d895843dc
feat: add new telemetry server (#254)
Signed-off-by: Mark Sagi-Kazar <mark.sagikazar@gmail.com>
2020-12-18 09:01:19 +01:00
Nicola Murino
c451f742aa
revertprovider: crypted provider was not supported in v4
also ensure to initialize kms before the dataprovider, it could be
needed to downgrade secret from cloud kms providers
2020-12-06 10:36:48 +01:00
Nicola Murino
95c6d41c35
config: make config file relative to the config dir
a configuration parsing error is now fatal
2020-12-03 17:16:35 +01:00
Nicola Murino
634b723b5d
add KMS support
Fixes #226
2020-11-30 21:46:34 +01:00
Nicola Murino
6699f5c2cc
initial data loading: an error is no longer fatal
therefore it does not prevent the services from starting
2020-11-25 09:18:36 +01:00
Nicola Murino
0609188d3f
allow to disable SFTP service
Fixes #228
2020-11-24 13:44:57 +01:00
Nicola Murino
dccc583b5d
add a dedicated struct to store encrypted credentials
also gcs credentials are now encrypted, both on disk and inside the
provider.

Data provider is automatically migrated and load data will accept
old format too but you should upgrade to the new format to avoid future
issues
2020-11-22 21:53:04 +01:00
Nicola Murino
0a14297b48
webdav: performance improvements and bug fixes
we need my custom golang/x/net/webdav fork for now

https://github.com/drakkan/net/tree/sftpgo
2020-11-04 19:11:40 +01:00
Nicola Murino
f2acde789d
portable mode: add Azure Blob support 2020-10-25 21:42:43 +01:00
Nicola Murino
bb5207ad77
Add support for loading users/folders on startup
Fixes #161
2020-10-20 18:42:37 +02:00
Nicola Murino
a9e21c282a add WebDAV support
Fixes #147
2020-08-11 23:56:10 +02:00
Nicola Murino
93ce96d011 add support for the venerable FTP protocol
Fixes #46
2020-07-29 21:56:56 +02:00
Nicola Murino
4e41a5583d refactoring: add common package
The common package defines the interfaces that a protocol must implement
and contain code that can be shared among supported protocols.

This way should be easier to support new protocols
2020-07-24 23:39:38 +02:00
Nicola Murino
95fe26f3e3 keep track of services errors
So we can exit with the correct code if an error happen inside the
services goroutines

Fixes #143
2020-07-09 19:16:52 +02:00
Nicola Murino
0ea2ca3141 simplify data provider usage
remove the obsolete SQL scripts too. They are not required since v0.9.6
2020-07-08 19:59:31 +02:00
Nicola Murino
0056984d4b Allow to rotate logs on demand
Log file can be rotated sending a SIGUSR1 signal on Unix based systems and
using "sftpgo service rotatelogs" on Windows

Fixes #133
2020-06-22 19:11:53 +02:00
Nicola Murino
23a80b01b6 add build tag to disable metrics 2020-06-19 17:08:51 +02:00
Nicola Murino
ad53429cf1 add support for build tag to allow to disable some features
The following build tags are available:

- "nogcs", disable Google Cloud Storage backend
- "nos3", disable S3 Compabible Object Storage backends
- "nobolt", disable Bolt data provider
- "nomysql", disable MySQL data provider
- "nopgsql", disable PostgreSQL data provider
- "nosqlite", disable SQLite data provider
- "noportable", disable portable mode
2020-05-23 11:58:05 +02:00
Nicola Murino
f02e24437a add more linters
test cases migration to testify is now complete.
Linters are enabled for test cases too
2020-05-06 19:36:34 +02:00
Nicola Murino
d70959c34c fix some lint issues 2020-04-30 14:23:55 +02:00
Nicola Murino
d377181b25 add a new configuration section for HTTP clients
HTTP clients are used for executing hooks such as the ones used for custom
actions, external authentication and pre-login user modifications.

This allows, for example, to use self-signed certificate without defeating the
purpose of using TLS
2020-04-26 23:29:09 +02:00
Nicola Murino
81c8e8d898 add profiler support
profiling is now available via the HTTP base URL /debug/pprof/

examples, use this URL to start and download a 30 seconds CPU profile:

/debug/pprof/profile?seconds=30

use this URL to profile used memory:

/debug/pprof/heap?gc=1

use this URL to profile allocated memory:

/debug/pprof/allocs?gc=1

Full docs here:

https://golang.org/pkg/net/http/pprof/
2020-03-15 15:16:35 +01:00
Nicola Murino
ec8ab28a22 portable mode: add support for file extensions filters 2020-03-05 15:37:10 +01:00
Nicola Murino
7f1946de34 improve validations for user provided file and directory paths 2020-03-03 09:09:58 +01:00
Nicola Murino
d0a81cabab log file: if the path is not absolute make it relative to config dir
Also refuse to join invalid file name such as "."

Fixes #85
2020-03-03 00:34:06 +01:00
Nicola Murino
0f80de86b2 simplify some code
now gocyclo is happy again
2020-02-20 15:53:26 +01:00
Nicola Murino
ae8ed75ae5 gcs: add support for automatic credentials
We can now also support implicit credentials using the Application
Default Credentials strategy
2020-02-19 09:41:15 +01:00
Enes Çakır
4f5c67e7df portable: set data-provider name to empty 2020-02-14 15:13:10 +01:00
Nicola Murino
5bfaae9202 httpd: allow to reload the https certificate without restarting the service
HTTPS certificate can be reloaded on demand sending a SIGHUP signal on
Unix based systems and a "paramchange" request to the running service on
Windows
2020-02-04 23:21:33 +01:00
Nicola Murino
bcaf283c35 memory provider: load users from a dump file
The `memory` provider can load users from a dump obtained using the
`dumpdata` REST API. This dump file can be configured using the
dataprovider `name` configuration key. It will be loaded at startup
and can be reloaded on demand using a `SIGHUP` on Unix based systems
and a `paramchange` request to the running service on Windows.

Fixes #66
2020-02-02 22:20:39 +01:00
Nicola Murino
3491717c26 add support for serving Google Cloud Storage over SFTP/SCP
Each user can be mapped with a Google Cloud Storage bucket or a bucket
virtual folder
2020-01-31 19:04:00 +01:00
Nicola Murino
9c4dbbc3f8 sftpd: add support for some SSH commands
md5sum, sha1sum are used by rclone.
cd, pwd improve the support for RemoteFiles mobile app.

These commands are all implemented inside SFTPGo so they work even
if the matching system commands are not available, for example on Windows
2019-11-18 23:30:37 +01:00
Nicola Murino
acdf351047 dataprovider: add custom command and/or HTTP notifications on users add, update and delete
This way custom logic can be implemented for example to create a UNIX user
as asked in #58
2019-11-14 11:06:03 +01:00
Nicola Murino
191da1ecaf fix a typo 2019-11-08 08:46:31 +01:00