beenull/sftpgo-mirror
1
0
Fork 0
mirror of https://github.com/drakkan/sftpgo.git synced 2024-11-22 15:40:23 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
1234 commits 8 branches 49 tags 49 MiB
Commit graph

365 commits

Author SHA1 Message Date
Nicola Murino
ae8ccadad2
users API: add API to create, delete, rename files and directories 2021-07-23 10:19:27 +02:00
Nicola Murino
776dffcf12
kms: improve modularity 2021-07-13 21:17:21 +02:00
Nicola Murino
e1a2451c22
s3: allow to configure the chunk download timeout 2021-07-11 18:39:45 +02:00
Nicola Murino
bd5191dfc5
add experimental plugin system 2021-07-11 15:26:51 +02:00
Nicola Murino
ff19879ffd
allow to use a persistent signing key for JWT and CSRF tokens 
Fixes #466
 2021-07-01 20:17:40 +02:00
Nicola Murino
076b2f0ee0
modules: add v2 support 2021-06-26 07:31:41 +02:00
Manuel Reithuber
fd4c388b23 added vfs.ListProviders() and using it in template fsconfig.html (added a new ListFSProviders template function for that) 2021-06-19 19:27:54 +02:00
Manuel Reithuber
88b10da596 updated utils.LoadTemplate() to call template.ParseFiles() directly and added a way to specify a base template (will be used in the next commit) 2021-06-19 19:27:54 +02:00
Manuel Reithuber
0ff010cc94 added vfs.GetProviderByName(), using it in for sftpgo portable and for parsing the webadmin form field 2021-06-19 19:27:54 +02:00
Nicola Murino
c1b862394d
move other errors to utils package 2021-06-19 13:06:01 +02:00
Manuel Reithuber
f19937b715
move Filesystem config validation to vfs 2021-06-19 12:24:43 +02:00
Nicola Murino
3bb0ca1d2b
config: remove deprecated configuration keys 2021-06-19 09:47:06 +02:00
Nicola Murino
9d3d7db29c
azblob: store SAS URL as kms.Secret 2021-06-11 22:27:36 +02:00
Nicola Murino
4be6307d87
webadmin: add defender page 2021-06-08 13:24:28 +02:00
Nicola Murino
feec2118bb
improve defender and quotas REST API 2021-06-07 21:52:43 +02:00
Nicola Murino
43182fc25e
OpenAPI: add users API 
These new APIs match the web client features.

I'm aware that some API do not follow REST best practises.

I want to avoid things likes "/user/folders/<path>"

where "path" must be encoded and making it optional create issues, so
I defined resources as query parameters instead of path parameters
 2021-06-05 16:07:09 +02:00
Nicola Murino
575bcf1f03
add remote address to transfer and commands logs 2021-06-01 22:28:43 +02:00
Nicola Murino
c1239fbf59
pre-upload action: add file open flags 
Reading the flags the hook receiver can detect if the client wants to
truncate the target file
 2021-05-31 22:33:23 +02:00
Nicola Murino
c63b923ec3
cryptfs: add support for atomic uploads 2021-05-31 21:45:29 +02:00
Nicola Murino
423d8306be
webclient: allow to download multiple files as zip 2021-05-30 23:07:46 +02:00
Nicola Murino
7a85c66ee7
webclient: defer file list rendering 
combined with server side processing I can now list a directory with
about 100.000 files in less than 2 seconds without losing client side
filtering and pagination
 2021-05-27 09:40:46 +02:00
Nicola Murino
25a44030f9
actions: add pre-download and pre-upload 
Downloads and uploads can be denied based on hook response
 2021-05-26 07:48:37 +02:00
Nicola Murino
600268ebb8
httpclient: allow to set custom headers 2021-05-25 08:36:01 +02:00
Nicola Murino
1223957f91
webclient: use different icons based on the file extension 2021-05-24 19:09:03 +02:00
Nicola Murino
15cde2dd1a
improve test coverage 2021-05-23 22:29:55 +02:00
Nicola Murino
50e441849a
try to make the web admin more user friendly 
removed all the textarea with fields separated using "::".
This should, hopefully, improve user experience
 2021-05-23 22:02:01 +02:00
Nicola Murino
02bb09ec01
remove deprecated file extensions filters 
these filters were deprecated a long time ago, everyone should use
patterns filters now
 2021-05-22 12:28:05 +02:00
Nicola Murino
b9bc8d722d
try to improve web client credentials page 
I should do the same for the admin page too
 2021-05-22 09:54:27 +02:00
Nicola Murino
ecfed4dc04
Add a Getting Started Guide 2021-05-20 18:16:27 +02:00
Nicola Murino
60cfbd2989
setup: auto login after creating the first admin 2021-05-16 21:36:57 +02:00
Nicola Murino
019b0f2fd5
http cookie: add max-age and samesite 
update deps too
 2021-05-16 09:13:00 +02:00
Nicola Murino
f2b93c0402
add a setup screen to create the first admin user 
If you prefer to auto-create the first admin you can enable the
"create_default_admin" configuration key and SFTPGo will work as before.

You can also create the first admin by loading initial data: now you can
set both username and password, before you could only change the password
 2021-05-14 19:21:15 +02:00
Nicola Murino
c8f7fc9bc9
httpd/webdav: add a list of hosts allowed to send proxy headers 
X-Forwarded-For, X-Real-IP and X-Forwarded-Proto headers will be ignored
for hosts not included in this list.

This is a backward incompatible change, before the proxy headers were
always used
 2021-05-11 06:54:06 +02:00
Nicola Murino
8f6cdacd00
allow to limit the number of per-host connections 2021-05-08 19:45:21 +02:00
Nicola Murino
4ea4202b99
httpd/webdav: use a custom listener with read and write deadlines 2021-05-07 20:41:20 +02:00
Nicola Murino
d4bfc3f6b5
fix lint configuration and a warning 2021-05-06 22:06:22 +02:00
Nicola Murino
23d9ebfc91
add a basic front-end web interface for end-users 
Fixes #339 #321 #398
 2021-05-06 21:35:43 +02:00
Nicola Murino
a383957cfa
OpenAPI: document that also folder-quota-update supports partial updates 2021-04-28 19:33:32 +02:00
Nicola Murino
00f97aabb4
OpenAPI: document that quota-update support partial updates 
If the update mode is "add" and you pass only used_quota_size or only
used_quota_files the missing field will remain unchanged
 2021-04-28 19:16:15 +02:00
Nicola Murino
32db0787bb
add an example script for scheduled quota updates 2021-04-26 21:53:09 +02:00
Nicola Murino
1275328fdf
Authentication errors: try to avoid user enumeration 
Fixes #395
 2021-04-26 19:48:21 +02:00
Nicola Murino
3941255733
docs: fix a typo 2021-04-25 09:42:19 +02:00
Nicola Murino
46998252e5
use bcrypt as default password hashing algo 
argon2id has a high memory cost and, if not properly tuned, it can lead to
resource starvation.

Advanced users can still configure and use argon2id.
Passwords stored as argon2id will continue to work
 2021-04-25 09:38:33 +02:00
Nicola Murino
92638ce93d
add support for hashing password using bcrypt 
argon2id remains the default
 2021-04-20 13:55:09 +02:00
Nicola Murino
47317bed9b
make sure that Retry-After header has a value greater than zero 2021-04-19 09:16:27 +02:00
Nicola Murino
f45c89fc46
add rate limiting support for REST API/web admin too 2021-04-19 08:14:04 +02:00
Nicola Murino
112e3b2fc2
add rate limiting support 2021-04-18 12:31:06 +02:00
Nicola Murino
c844fc7477
add support for delayed quota update 
If there are a lot of close uploads, accumulating quota updates can
save you many queries to the data provider
 2021-04-11 08:38:43 +02:00
Nicola Murino
0bc4db9950
web admin: make base url configurable 2021-04-09 22:02:48 +02:00
Nicola Murino
0823928f98
allow to disable login filesystem checks 
SFTPGo requires that the user's home directory, virtual folder root,
and intermediate paths to virtual folders exist to work properly.
If you already know that the required directories exist, disabling
these checks will speed up login.
 2021-04-05 17:57:30 +02:00
Nicola Murino
fdf3f23df5
allow to disable some hooks on a per-user basis 
This way you can, for example, mix external and internal users
 2021-04-04 22:32:25 +02:00
Nicola Murino
1ee843757d
fix OpenAPI schema 2021-04-03 17:09:08 +02:00
Nicola Murino
ea26d7786c
sftpfs: add buffering support 
this way we improve performance over high latency networks
 2021-04-03 16:00:55 +02:00
Nicola Murino
6eb43baf3d
web: fix content type for folders form 
Fixes #367
 2021-04-01 19:42:18 +02:00
Nicola Murino
2f56375121
improve SFTP loop detection 2021-04-01 18:53:48 +02:00
Nicola Murino
3bfd7e4d17
sftpfs: try to detect if an SFTP user point to itself 
this will cause an infinite loop on login. The check should be improved
 2021-03-29 21:53:44 +02:00
Nicola Murino
a43854ae9b
OpenAPI: document that secrets are automatically encrypted before saving 2021-03-28 11:23:06 +02:00
Nicola Murino
9ad750da54
WebDAV: try to preserve the lock fs as much as possible 2021-03-27 19:10:27 +01:00
Nicola Murino
0e50310a66
add a test case for UID/GID limits 2021-03-25 17:30:39 +01:00
Mike Unitskyi
5939ac4801
Increase uid:gid limits (#362) 
Fixes #361
 2021-03-25 17:11:42 +01:00
Nicola Murino
8a1249878a
OpenAPI schema: remove some superfluous required definitions 
Fixes #356
 2021-03-22 19:22:41 +01:00
Nicola Murino
5e375f56dd
kms: add a lock, secrets could be modified concurrently for cached users 
also reduce the size of the JSON payload omitting empty secrets
 2021-03-22 19:03:25 +01:00
Nicola Murino
d6dc3a507e
extend virtual folders support to all storage backends 
Fixes #241
 2021-03-21 19:15:47 +01:00
Nicola Murino
67ea75cf03
improve OpenAPI schema so it is better rendered on Stoplight 2021-03-07 18:41:56 +01:00
Nicola Murino
1ab02d5891
OpenAPI: improve schema 
Fix some lint warnings
 2021-03-06 17:08:24 +01:00
Nicola Murino
055506e518
sftpfs: add an option to disable concurrent reads 2021-03-06 15:41:40 +01:00
Nicola Murino
88122ba2f8
update jwtauth to v5 2021-03-05 18:50:45 +01:00
Nicola Murino
df41f0c556
add a setting to skip natural keys validation 
Enabling the "skip_natural_keys_validation" data provider setting,
the natural keys for REST API/Web Admin as usernames, admin names,
folder names are not restricted to unreserved URI chars

Fixes #334 #308
 2021-03-04 09:48:53 +01:00
Nicola Murino
534b253c20
WebDAV: improve TLS certificate authentication 
For each user you can now configure:

- TLS certificate auth
- TLS certificate auth and password
- Password auth

For TLS certificate auth, the certificate common name is used as
username
 2021-03-01 19:28:11 +01:00
Nicola Murino
a6e36e7cad
FTP: improve TLS certificate authentication 
For each user you can now configure:

- TLS certificate auth
- TLS certificate auth and password
- Password auth

For TLS auth, the certificate common name must match the name provided
using the "USER" FTP command
 2021-02-28 12:10:40 +01:00
Nicola Murino
ca3e15578e
Use new methods in the io and os packages instead of ioutil ones 
ioutil is deprecated in Go 1.16 and SFTPGo is an application, not
a library, we have no reason to keep compatibility with old Go
versions.

Go 1.16 fix some cifs related issues too.
 2021-02-25 21:53:04 +01:00
Nicola Murino
2146b83343
data providers: add filesystem to folder ... 
... and some descriptive fields.
The filesystem support for virtual folders will be implemented in
future commits
 2021-02-24 19:40:29 +01:00
Nicola Murino
5da4f931c5
TLS: allow to configure cipher suites 
Fixes #316
 2021-02-18 20:17:16 +01:00
Nicola Murino
be9230e85b
micro optimizations spotted using the go-critic linter 2021-02-16 19:11:36 +01:00
Nicola Murino
b1ce6eb85b
web admin: allow to set an empty password for SFTPGo users 2021-02-15 19:38:53 +01:00
Nicola Murino
46176a54b4
minor doc fixes 2021-02-14 22:08:08 +01:00
Nicola Murino
a21ccad174
web hooks: add mutual TLS support 2021-02-13 14:41:37 +01:00
Nicola Murino
1bccb93fcb
rename default branch from master to main 2021-02-09 19:53:03 +01:00
Nicola Murino
a2a99f9b57
merge full and slim dockerfiles 
Fixes #232
 2021-02-07 21:49:04 +01:00
Nicola Murino
d32b195a57
httpd: reuse the same compressor among bindings 2021-02-04 22:32:55 +01:00
Nicola Murino
267d9f1831
web ui: allow to create folders from a template 2021-02-04 19:09:43 +01:00
Nicola Murino
70bb3c34ce
sftpfs: improve endpoint validation 
Validation will fail if the endpoint is not specified as host:port
 2021-02-03 11:29:04 +01:00
Nicola Murino
e9dd4ecdf0
web admin: add CSRF 2021-02-03 08:55:28 +01:00
Nicola Murino
f863530653
JWT: only accepts tokens from the expected header or cookie 2021-02-02 13:11:47 +01:00
Nicola Murino
4f609cfa30
JWT: add token audience 
a token released for API audience cannot be used for web pages and
vice-versa
 2021-02-02 09:14:10 +01:00
Nicola Murino
78bf808322
virtual folders: change dataprovider structure 
This way we no longer depend on the local file system path and so we can
add support for cloud backends in future updates
 2021-02-01 19:04:15 +01:00
Nicola Murino
afe1da92c5
web UI cookie: set the Secure flags if we are over TLS 2021-01-28 13:29:16 +01:00
Nicola Murino
c2bbd468c4
REST API: add logout and store invalidated token 2021-01-26 22:35:36 +01:00
Nicola Murino
54321c5240
web ui: allow to create multiple users from a template 2021-01-25 21:31:33 +01:00
Nicola Murino
80f5ccd357
web admin: add backup/restore 2021-01-22 19:42:18 +01:00
Nicola Murino
57976b4085
httpd: add mTLS and multiple bindings support 2021-01-19 18:59:41 +01:00
Nicola Murino
41a1af863e
OpenAPI: minor changes 2021-01-18 13:24:38 +01:00
Nicola Murino
778ec9b88f
REST API v2 
- add JWT authentication
- admins are now stored inside the data provider
- admin access can be restricted based on the source IP: both proxy
  header and connection IP are checked
- deprecate REST API CLI: it is not relevant anymore

Some other changes to the REST API can still happen before releasing
SFTPGo 2.0.0

Fixes #197
 2021-01-17 22:29:08 +01:00
Nicola Murino
daac90c4e1
fix a potential race condition for pre-login and ext auth 
hooks

doing something like this:

err = provider.updateUser(u)
...
return provider.userExists(username)

could be racy if another update happen before

provider.userExists(username)

also pass a pointer to updateUser so if the user is modified inside
"validateUser" we can just return the modified user without do a new
query
 2021-01-05 09:50:22 +01:00
Nicola Murino
684f4ba1a6
mutal TLS: add support for revocation lists 2021-01-03 17:03:04 +01:00
Nicola Murino
6d84c5b9e3
capture http servers error logs 
otherwise they will be printed to stdout
 2021-01-03 10:38:28 +01:00
Nicola Murino
4b522a2455
webdav: refactor server initialization 2021-01-03 09:51:54 +01:00
Nicola Murino
d6b3acdb62
add REST API for the defender 2021-01-02 19:33:24 +01:00
Nicola Murino
037d89a320
add support for a basic built-in defender 
It can help to prevent DoS and brute force password guessing
 2021-01-02 14:05:09 +01:00
Nicola Murino
30eb3c4a99
update OpenAPI schema 2020-12-29 19:33:04 +01:00
Nicola Murino
0966d44c0f
httpd: add support for listening over a Unix-domain socket 
Fixes #266
 2020-12-29 19:02:56 +01:00
Nicola Murino
80c06d6b59
clone: disable decrypt error test for memory provider 
This test cannot work using memory provider, we cannot change the provider
for a kms secrete without reloading it from JSON and the memory provider
will never reload users
 2020-12-26 15:57:01 +01:00
Nicola Murino
e536a638c9
web UI: improve user cloning 2020-12-26 15:11:38 +01:00
Jochen Munz
bc397002d4
Feature: Clone existing user via web admin (#259) 
UI based cloning of an existing user. The "add user" screen is prepopulated with existing user data.

Resolves drakkan/sftpgo#225
 2020-12-26 14:58:59 +01:00
Nicola Murino
1dce1eff48
improve FTP support 
- allow to disable active mode
- allow to disable SITE commands
- add optional support for calculating hash value of files
- add optional support for the non standard COMB command
 2020-12-24 18:48:06 +01:00
Nicola Murino
c69d63c1f8
add support for multiple bindings 
Fixes #253
 2020-12-23 16:12:30 +01:00
Nicola Murino
743b350fdd
httpd: add support for route undefined HEAD requests to GET handlers 
HEAD responses will not include a body but the Content-Length will be
set as the equivalent GET request

Fixes #255
 2020-12-20 10:22:16 +01:00
Nicola Murino
bcf0fa073e
telemetry server: add optional https and authentication 2020-12-18 16:04:42 +01:00
Nicola Murino
140380716d
remove unused constant 2020-12-18 10:05:08 +01:00
Nicola Murino
143df87fee
add some docs for telemetry server 
move pprof to the telemetry server only
 2020-12-18 09:47:22 +01:00
Nicola Murino
a6985075b9
add sftpfs storage backend 
Fixes #224
 2020-12-12 10:31:09 +01:00
Nicola Murino
50982229e1
REST API: add a method to get the status of the services 
added a status page to the built-in web admin
 2020-12-08 11:18:34 +01:00
Nicola Murino
4a88ea5c03
add Data At Rest Encryption support 2020-12-05 13:48:13 +01:00
Nicola Murino
a67276ccc2
add build tags to disable kms providers 2020-12-02 09:44:18 +01:00
Nicola Murino
87b51a6fd5
kms: remember if a secret was saved without a master key 
So we will be able to decrypt secret stored without a master key if a
such key is provided later
 2020-12-01 22:18:16 +01:00
Nicola Murino
634b723b5d
add KMS support 
Fixes #226
 2020-11-30 21:46:34 +01:00
Nicola Murino
4bb9d07dde
user: add a free text field 
Fixes #230
 2020-11-25 22:26:34 +01:00
Nicola Murino
dccc583b5d
add a dedicated struct to store encrypted credentials 
also gcs credentials are now encrypted, both on disk and inside the
provider.

Data provider is automatically migrated and load data will accept
old format too but you should upgrade to the new format to avoid future
issues
 2020-11-22 21:53:04 +01:00
Nicola Murino
e3eca424f1
web admin: allow both allowed and denied extensions/patterns for a dir 
this fix a regression introduced in the previous commit
 2020-11-16 19:21:50 +01:00
Nicola Murino
a6355e298e
add support for limit files using shell like patterns 
Fixes #209
 2020-11-15 22:04:48 +01:00
Nicola Murino
6ad4cc317c
cloud backends: stat and other performance improvements 2020-11-02 19:16:12 +01:00
Nicola Murino
57bec976ae
document heathz endpoint 2020-11-01 10:39:10 +01:00
Nicola Murino
fcfdd633f6
Azure Blob: update SDK and add access tier support 2020-10-30 22:17:17 +01:00
Nicola Murino
5bc1c2de2d
add a link to the heml chart 
Fixes #210
 2020-10-29 21:50:21 +01:00
Mark Sagi-Kazar
ec00613202 feat(httpd): add new healthz endpoint 
Signed-off-by: Mark Sagi-Kazar <mark.sagikazar@gmail.com>
 2020-10-29 21:37:30 +01:00
Mark Sagi-Kazar
02ec3a5f48 refactor(httpd): move every route under a new group 
Signed-off-by: Mark Sagi-Kazar <mark.sagikazar@gmail.com>
 2020-10-29 21:37:30 +01:00
Nicola Murino
14bcc6f2fc
s3, azblob: check upper limit for part size 2020-10-25 12:10:11 +01:00
Nicola Murino
5ff8f75917
add Azure Blob support 2020-10-25 08:18:48 +01:00
Sean Hildebrand
db7e81e9d0
add prefer_database_credentials configuration parameter 
When true, users' Google Cloud Storage credentials will be written to
the data provider instead of disk.
Pre-existing credentials on disk will be used as a fallback

Fixes #201
 2020-10-22 10:42:40 +02:00
Nicola Murino
bb5207ad77
Add support for loading users/folders on startup 
Fixes #161
 2020-10-20 18:42:37 +02:00
Mark Sagi-Kazar
5e2db77ef9 refactor: add an enum for filesystem providers 
Signed-off-by: Mark Sagi-Kazar <mark.sagikazar@gmail.com>
 2020-10-05 21:40:21 +02:00
Nicola Murino
2df0dd1f70 sshd: map each channel with a new connection 
Fixes #169
 2020-09-18 10:52:53 +02:00
Nicola Murino
3c1300721c add some basic how-to style documents 2020-09-13 19:43:56 +02:00
Nicola Murino
61003c8079 sftpd: add lstat support 2020-09-11 09:30:25 +02:00
Nicola Murino
01850c7399 REST API: remove status from ApiResponse 
it duplicates the header HTTP status
 2020-09-08 09:45:21 +02:00
Nicola Murino
b9c381e26f sftpd: update pkg/sftp 
The patch to open a file in read/write mode is now merged
 2020-09-06 11:40:31 +02:00
Nicola Murino
542554fb2c replace the library to verify UNIX's crypt(3) passwords 2020-09-04 21:08:09 +02:00
Nicola Murino
3925c7ff95 REST API/Web admin: add a parameter to disconnect a user after an update 
This way you can force the user to login again and so to use the updated
configuration.

A deleted user will be automatically disconnected.

Fixes #163

Improved some docs too.
 2020-09-01 16:10:26 +02:00
Nicola Murino
8b0a1817b3 add check password hook 
its main use case is to allow to easily support things like password+OTP for
protocols without keyboard interactive support such as FTP and WebDAV
 2020-08-19 19:36:12 +02:00
Nicola Murino
bbc8c091e6 portable mode: add WebDAV support 2020-08-17 14:08:08 +02:00
Nicola Murino
f3228713bc Allow individual protocols to be enabled per user 
Fixes #154
 2020-08-17 12:49:20 +02:00
Nicola Murino
fa5333784b add a maximum allowed size for a single upload 2020-08-16 20:17:02 +02:00
Nicola Murino
196a56726e FTP improvements 
- add a setting to require TLS
- add symlink support

require TLS 1.2 for all TLS connections
 2020-08-15 13:02:25 +02:00
Nicola Murino
aa0ed5dbd0 add post-login hook 
a login scope is supported too so you can get notifications for failed logins,
successful logins or both
 2020-08-12 16:15:12 +02:00
Nicola Murino
a9e21c282a add WebDAV support 
Fixes #147
 2020-08-11 23:56:10 +02:00
Nicola Murino
93ce96d011 add support for the venerable FTP protocol 
Fixes #46
 2020-07-29 21:56:56 +02:00
Nicola Murino
4e41a5583d refactoring: add common package 
The common package defines the interfaces that a protocol must implement
and contain code that can be shared among supported protocols.

This way should be easier to support new protocols
 2020-07-24 23:39:38 +02:00
Nicola Murino
da0f470f1c document FreeBSD support 
improve some tests cleanup
 2020-07-10 19:20:37 +02:00
Nicola Murino
0ea2ca3141 simplify data provider usage 
remove the obsolete SQL scripts too. They are not required since v0.9.6
 2020-07-08 19:59:31 +02:00
Nicola Murino
6ab4e9f533 add test case for concurrent logins 2020-06-27 12:36:42 +02:00
Nicola Murino
cf541d62ea recursive permissions check before renaming/copying directories 2020-06-26 23:38:29 +02:00
Nicola Murino
ddf99ab706 workflow: execute test cases on MySQL too 2020-06-22 20:02:51 +02:00
Nicola Murino
b80abe6c05 return exit code 1 on error 
Fixes #132
 2020-06-20 14:30:46 +02:00
Nicola Murino
8cb47817f6 Add API endpoint to set current quota 
Fixes #130
 2020-06-20 12:38:04 +02:00
Nicola Murino
23a80b01b6 add build tag to disable metrics 2020-06-19 17:08:51 +02:00
Nicola Murino
b30614e9d8 httpd: make the built-in web interface optional 
The built-in web admin will be disabled if both "templates_path" and
"static_files_path" are empty

Fixes #131
 2020-06-18 23:53:38 +02:00
Nicola Murino
7807fa7cc2 use os.ModePerm for files and directory creation 2020-06-08 19:40:17 +02:00
Nicola Murino
c231b663a3 add docs for virtual folders 
fix test cases on macOS
 2020-06-08 00:15:14 +02:00
Nicola Murino
8306b6bde6 refactor virtual folders 
The same virtual folder can now be shared among users and different
folder quota limits for each user are supported.

Fixes #120
 2020-06-07 23:30:18 +02:00
Nicola Murino
ad53429cf1 add support for build tag to allow to disable some features 
The following build tags are available:

- "nogcs", disable Google Cloud Storage backend
- "nos3", disable S3 Compabible Object Storage backends
- "nobolt", disable Bolt data provider
- "nomysql", disable MySQL data provider
- "nopgsql", disable PostgreSQL data provider
- "nosqlite", disable SQLite data provider
- "noportable", disable portable mode
 2020-05-23 11:58:05 +02:00
Nicola Murino
738c7ab43e sftpd: add support for SSH user certificate authentication 
This add support for PROTOCOL.certkeys vendor extension:

https://cvsweb.openbsd.org/src/usr.bin/ssh/PROTOCOL.certkeys?rev=1.8

Fixes #117

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
 2020-05-15 20:08:53 +02:00
Nicola Murino
f02e24437a add more linters 
test cases migration to testify is now complete.
Linters are enabled for test cases too
 2020-05-06 19:36:34 +02:00
Nicola Murino
e9534be1e6 travis: exclude go 1.13 for arch arm64 2020-05-03 22:46:39 +02:00
Nicola Murino
155af19aaa tests: update httpd test to use testify 2020-05-03 15:24:26 +02:00
Nicola Murino
3f75d46a16 sftpd: add support for excluding virtual folders from user quota limit 
Fixes #110
 2020-05-01 15:27:53 +02:00
Nicola Murino
d70959c34c fix some lint issues 2020-04-30 14:23:55 +02:00
Nicola Murino
d377181b25 add a new configuration section for HTTP clients 
HTTP clients are used for executing hooks such as the ones used for custom
actions, external authentication and pre-login user modifications.

This allows, for example, to use self-signed certificate without defeating the
purpose of using TLS
 2020-04-26 23:29:09 +02:00
Nicola Murino
4f668bf558 simplify some httpd related code 
and update chi, cobra and viper
 2020-04-21 19:24:38 +02:00
Nicola Murino
b1c7317cf6 add support for partial authentication 
Multi-step authentication is activated disabling all single-step
auth methods for a given user
 2020-04-09 23:32:42 +02:00
Nicola Murino
c93d8cecfc update deps 
chi 4.1.0 requires some minor code changes
 2020-04-03 22:30:30 +02:00
Nicola Murino
0a9c4914aa pre-login program: allow to create a new user too 
clarify the difference between dynamic user creation/update and external
authentication
 2020-03-27 23:26:22 +01:00
Nicola Murino
0787e3e595 bolt provider: fix error handling for get users with username filter 2020-03-22 15:37:08 +01:00
Nicola Murino
f55851bdc8 update nathanaelle password to v2 
Fixes #97
 2020-03-20 17:25:38 +01:00
Nicola Murino
81c8e8d898 add profiler support 
profiling is now available via the HTTP base URL /debug/pprof/

examples, use this URL to start and download a 30 seconds CPU profile:

/debug/pprof/profile?seconds=30

use this URL to profile used memory:

/debug/pprof/heap?gc=1

use this URL to profile allocated memory:

/debug/pprof/allocs?gc=1

Full docs here:

https://golang.org/pkg/net/http/pprof/
 2020-03-15 15:16:35 +01:00
Nicola Murino
1770da545d s3: upload concurrency is now configurable 
Please note that if the upload bandwidth between the SFTP client and
SFTPGo is greater than the upload bandwidth between SFTPGo and S3 then
the SFTP client have to wait for the upload of the last parts to S3
after it ends the file upload to SFTPGo, and it may time out.
Keep this in mind if you customize parts size and upload concurrency
 2020-03-13 19:13:58 +01:00
Nicola Murino
de3e69f846 s3: add documentation and test cases for upload part size 2020-03-13 17:28:55 +01:00
Michael Bonfils
cdf1233065 s3: export PartSize parameter 
By default AWS SDK use part_size of 5 MB. For big files,
it is not ideal case. For Hadoop, it is not uncommon to
use 512 MB.
 2020-03-13 17:28:04 +01:00
Nicola Murino
7f1946de34 improve validations for user provided file and directory paths 2020-03-03 09:09:58 +01:00
Nicola Murino
d0a81cabab log file: if the path is not absolute make it relative to config dir 
Also refuse to join invalid file name such as "."

Fixes #85
 2020-03-03 00:34:06 +01:00
Nicola Murino
ed1c7cac17 update deps 
we now use git master for pkg/sftp: it includes the performance patches
from my copy branch.
 2020-03-02 10:13:49 +01:00
Nicola Murino
3ffddcba92 web: log an error if loading a required template fails 
We used template.Must that panics if an error happen but the error is
visible only if sftpgo is started in an interactive way

Fixes #82
 2020-03-02 09:34:13 +01:00
Nicola Murino
b885d453a2 filters: we can now set allowed and denied files extensions 2020-03-01 22:10:29 +01:00
Nicola Murino
45b9366dd0 add support for virtual folders 
directories outside the user home directory can be exposed as virtual folders
 2020-02-23 11:30:26 +01:00
Nicola Murino
bc11cdd8d5 add support for per user authentication methods 
You can, for example, deny one or more authentication methods to one or
more users.
 2020-02-19 22:39:30 +01:00
Nicola Murino
ae8ed75ae5 gcs: add support for automatic credentials 
We can now also support implicit credentials using the Application
Default Credentials strategy
 2020-02-19 09:41:15 +01:00
Nicola Murino
58253968fc s3: improve credentials validation 
access secret can now be empty, so check if not empty before encrypting
the secret
 2020-02-16 10:14:44 +01:00
Nicola Murino
8eff2df39c subdir perms: allow empty perms 
empty perms will allow nothing on the specified subdir.

Non empty permissions for the "/" dir are still required.

Fixes #70
 2020-02-10 19:28:35 +01:00
Nicola Murino
5bfaae9202 httpd: allow to reload the https certificate without restarting the service 
HTTPS certificate can be reloaded on demand sending a SIGHUP signal on
Unix based systems and a "paramchange" request to the running service on
Windows
 2020-02-04 23:21:33 +01:00
Nicola Murino
8b039e0447 httpd: add support for basic auth and HTTPS 2020-02-04 00:08:00 +01:00
Nicola Murino
bcaf283c35 memory provider: load users from a dump file 
The `memory` provider can load users from a dump obtained using the
`dumpdata` REST API. This dump file can be configured using the
dataprovider `name` configuration key. It will be loaded at startup
and can be reloaded on demand using a `SIGHUP` on Unix based systems
and a `paramchange` request to the running service on Windows.

Fixes #66
 2020-02-02 22:20:39 +01:00
Nicola Murino
31a433cda2 update deps and simplify some code 2020-01-31 23:26:56 +01:00
Nicola Murino
e647f3626e loaddata: add an option that allows to not modify existing users 2020-01-31 19:10:45 +01:00
Nicola Murino
3491717c26 add support for serving Google Cloud Storage over SFTP/SCP 
Each user can be mapped with a Google Cloud Storage bucket or a bucket
virtual folder
 2020-01-31 19:04:00 +01:00
Nicola Murino
7ebbbe5c29 S3: update pipeat to the latest commit in my fork 
Here are the main improvements:

- unliked files works on windows too
- the uploads are now synced on the lower speed between the SFTP client write
and the upload speed to S3

This commit increase the external auth timeout to 60 seconds too
 2020-01-22 19:42:23 +01:00
Nicola Murino
4463421028 S3: add support for serving virtual folders 
inside the same bucket each user can be assigned to a virtual folder.
This is similar to a chroot directory for local filesystem
 2020-01-19 23:23:09 +01:00
Nicola Murino
d75f56b914 vfs: store root dir 
so we don't need to pass it over and over
 2020-01-19 13:58:55 +01:00
Nicola Murino
a4834f4a83 add basic S3-Compatible Object Storage support 
we have now an interface for filesystem backeds, this make easy to add
new filesystem backends
 2020-01-19 07:41:05 +01:00
Nicola Murino
1b1c740b29 Add support for allowed/denied IP/Mask 
Login can be restricted to specific ranges of IP address or to a specific IP
address.

Please apply the appropriate SQL upgrade script to add the filter field to your
database.

The filter database field will allow to add other filters without requiring a
new database migration
 2019-12-30 18:37:50 +01:00
Nicola Murino
80e9902324 scripts: add support for converting users from other users stores 
currently we support:

- Linux/Unix users stored in shadow/passwd files
- Pure-FTPd virtual users generated using `pure-pw` CLI
- ProFTPD users generated using `ftpasswd` CLI
 2019-12-29 17:21:25 +01:00
Nicola Murino
741e65a3a1 REST cli: allow to disable password or public_key auth 
using something like this:

update-user <user-id> <username> --public-keys ''

the public keys auth will be disabled

using something like this:

update-user <user-id> <username> --password ''

the password auth will be disabled
 2019-12-29 14:07:44 +01:00