Commit graph

353 commits

Author SHA1 Message Date
Nicola Murino
e18ad55067
S3: add support for session tokens
Fixes #736

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-02-25 15:30:04 +01:00
Nicola Murino
4e9dae6fa4
allow to cache external authentications
Fixes #733

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-02-25 11:51:10 +01:00
Nicola Murino
f5a0559be6
don't execute fs check if the user has recent activity
The check could be expensive with some backends and is generally
only required the first time that a user logs in

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-02-24 16:11:35 +01:00
Nicola Murino
670018f05e
OIDC: add profile and email scope to OAuth2 config
Fixes #728

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-02-22 10:20:14 +01:00
Nicola Murino
8bbf54d2b6
azure blobs: add support for multipart downloads
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-02-21 19:01:31 +01:00
Nicola Murino
c19b03a3f7
shares: add permission to deny sharing without password
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-02-19 13:31:58 +01:00
Nicola Murino
c6b8644828
OIDC: execute pre-login hook after IDP authentication
so the SFTPGo users can be auto-created using the hook

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-02-19 10:53:35 +01:00
Nicola Murino
f1a255aa6c
httpd: allow to restrict allowed hosts ...
... and to add security headers to the responses

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-02-17 18:22:27 +01:00
Nicola Murino
f1832d4478
shares: add an upload form for shares with write scope
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-02-15 19:19:25 +01:00
Nicola Murino
1fccd05e9e
allow to configure the minimum version of TLS to be enabled
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-02-13 15:56:07 +01:00
Nicola Murino
66945c0a02
Web UIs: add OpenID Connect support
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-02-13 14:30:20 +01:00
Nicola Murino
fa0ca8fe89
quota summary and docs improvements
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-02-08 12:43:08 +01:00
Nicola Murino
9382db751c
make HTTP shares browsable
if you share a single folder with read scope, you can now browse the share
and download single files

Fixes #674
See #677

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-02-06 16:46:43 +01:00
Nicola Murino
cd35636939
S3: add a timeout for single part uploads
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-02-01 12:15:56 +01:00
Nicola Murino
02db00d008
dataprovider: add naming rules
naming rules allow to support case insensitive usernames, trim trailing
and leading white spaces, and accept any valid UTF-8 characters in
usernames.

If you were enabling `skip_natural_keys_validation` now you need to
set `naming_rules` to `1`

Fixes #687

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-01-31 18:01:37 +01:00
Nicola Murino
1df1225eed
add support for data transfer bandwidth limits
with total limit or separate settings for uploads and downloads and
overrides based on the client's IP address.

Limits can be reset using the REST API

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-01-30 11:42:36 +01:00
Jeremy Clerc
9709aed5e6 httpd: webpath redirect using status found (302)
301 MovedPermanently is cached by the browser which can
be annoying when it is is on base path like / while one
may reuse the domain (e.g. localhost) for other apps/tests.

Fixes #695

Signed-off-by: Jeremy Clerc <jeremy@clerc.io>
2022-01-26 21:50:37 +01:00
Nicola Murino
d2a4178846
check quota usage between ongoing transfers
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-01-20 18:19:20 +01:00
Nicola Murino
d73be7aee5
remove the use of some unnecessary pointers
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-01-16 12:09:17 +01:00
Nicola Murino
c3831de94e
add hide policy to pattern filters
Disallowed files/dirs can be completly hidden. This may cause performance
issues for large directories

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-01-15 17:16:49 +01:00
Nicola Murino
1c51239da8
Admin UI: allow to create multiple users/folders from templates
the clone button is not needed anymore, you can select a user and
click on template to generate one or more similar users or you can
create users/folders from an empty template

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-01-12 19:01:19 +01:00
Nicola Murino
51c15de892
web admin: simplify user page
The page to add/edit users should be less less intimidating now.
All the advanced settings are hidden by default. Permissions are set
to any, so if you also have a users base dir set, to add a user
you have to simply set username, password or public key and save

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-01-10 19:44:16 +01:00
Nicola Murino
ec1d20f46f
sshd: improve docs about supported ciphers, KEX and MACs
also added a check to ensure that the configured values are valid

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-01-06 18:09:49 +01:00
Nicola Murino
1f619d5ea6
make the sdk a separate module
The SFTPGo SDK now is at the following URL

https://github.com/sftpgo/sdk

Fixes #657

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-01-06 11:54:43 +01:00
Nicola Murino
6d3d94a01f
move kms implementation outside the sdk package
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-01-06 10:11:47 +01:00
Nicola Murino
0a3d94f73d
log at info level the service configurations
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-01-05 13:22:49 +01:00
Nicola Murino
7c68b03d07
move plugin handling outside the sdk package
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-01-05 11:37:45 +01:00
Nicola Murino
2912b2e92e
sdk: add a logger interface
we are now ready to make the sdk a separate module

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-01-04 16:07:41 +01:00
Nicola Murino
a6fe802370
move kms definitions to the sdk package
This is the first step to make the sdk a separate module

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-01-04 12:49:30 +01:00
Nicola Murino
ad483b7581
httpd: switch back to chi Recoverer now that the required patch is merged
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-01-04 09:48:16 +01:00
Nicola Murino
df86955f28
eventsearcher plugin: add support to search for provider, bucket, endpoint
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-01-03 17:02:52 +01:00
Nicola Murino
4d7a4aa99a
check rename source and target 2021-12-28 12:03:52 +01:00
Nicola Murino
7d8823307f
defender: add provider driver
Fixes #616
2021-12-25 12:08:07 +01:00
Nicola Murino
ced73ed04e
REST API: add an option to create missing dirs 2021-12-19 12:14:53 +01:00
Nicola Murino
cc73bb811b
change log level from warn to error where appropriate
Fixes #649
2021-12-16 19:53:00 +01:00
Nicola Murino
a587228cf0
add support for metadata plugins 2021-12-16 18:18:36 +01:00
Nicola Murino
0bb141960f
add support for different bandwidth limits based on client IP 2021-12-10 18:43:26 +01:00
Nicola Murino
c153330ab8
web client: use fetch to upload files
also add REST API to upload a single file as POST body
2021-12-08 19:25:22 +01:00
Nicola Murino
9632b6ee94
events search: improve test cases 2021-12-04 18:18:59 +01:00
Nicola Murino
a7c0b07a2a
add session id to notifier plugins/hook 2021-12-04 17:27:24 +01:00
Nicola Murino
bedc8e288b
web client: add support for integrating external viewers/editors 2021-12-03 18:33:08 +01:00
Nicola Murino
6092b6628e
logs: use info level for login related messages
so enabling debug level is not required, for example only to understand
that a user exceeded the allowed sessions.

Also set the cache update frequency as documented
2021-12-02 19:36:42 +01:00
Nicola Murino
6ee51c5cc1
kms: remove support for compat secrets
also document how to activate the deprecated builtin provider
2021-12-01 17:53:19 +01:00
Nicola Murino
4df0ae82ac
web client: allow downloading of single shared files without compression
Fixes #629
2021-11-30 20:32:10 +01:00
Nicola Murino
5db31f0fb3
web client: allow to upload/delete multiple files 2021-11-30 18:40:50 +01:00
Nicola Murino
015aa36c56
loaddata: improve shares restore
usage and timestamps are now preserved
2021-11-27 11:12:51 +01:00
Nicola Murino
3f3591bae0
web client: allow to preview images and pdf
pdf depends on browser support. It does not work on mobile devices.
2021-11-25 19:24:32 +01:00
Nicola Murino
8a8298ad46
web client: improve file upload 2021-11-22 12:25:36 +01:00
Nicola Murino
3d6b09e949
REST API: expose OpenAPI schema and render it using Swagger UI
Fixes #609
2021-11-21 09:32:51 +01:00
Nicola Murino
fb8f013ea7
web: update permissions on cookie refresh 2021-11-20 10:48:39 +01:00
Nicola Murino
46157ebbb6
CI docker: remove armv7 support
CI is still unreliable if we enable armv7 support
2021-11-16 09:07:10 +01:00
Nicola Murino
24b0352eb6
GCS: add ACL support 2021-11-15 21:57:41 +01:00
Nicola Murino
52f3a98cc8
preserve GCS credentials on update if not set
credentials were not preserved if "prefer_database_credentials" was
set to true

Fixes #613
2021-11-15 19:12:58 +01:00
Nicola Murino
ca730e77a5
add separate permissions to delete and rename files and dirs
perm_delete and perm_rename still exist for backward compatibility,
now they are an alias to assign both new split permissions
2021-11-14 16:23:33 +01:00
Nicola Murino
0833b4698e
httpd service: add CORS support 2021-11-13 23:14:50 +01:00
Nicola Murino
ee5c5e033d
S3: add ACL support
Fixes #610
2021-11-13 16:05:40 +01:00
Nicola Murino
78233ff9a3
web UI/REST API: add password reset
In order to reset the password from the admin/client user interface,
an SMTP configuration must be added and the user/admin must have an email
address.
You can prohibit the reset functionality on a per-user basis by using a
specific restriction.

Fixes #597
2021-11-13 13:25:43 +01:00
Nicola Murino
094ee1522e
logger: add a flag to use UTC time for logging 2021-11-06 15:18:16 +01:00
Nicola Murino
3bc58f5988
WebClient/REST API: add sharing support 2021-11-06 14:13:20 +01:00
Nicola Murino
74fc3aaf37
REST API: add events search 2021-10-23 15:47:21 +02:00
Nicola Murino
97d0a48557
plugins: improve notifier and searcher 2021-10-20 19:39:49 +02:00
Nicola Murino
a80ac80fcd
pkgs: update nfpm to 2.7 and use xz as compression for both deb and rpm 2021-10-13 09:15:04 +02:00
Nicola Murino
4aa9686e3b
refactor custom actions
SFTPGo is now fully auditable, all fs and provider events that change
something are notified and can be collected using hooks/plugins.

There are some backward incompatible changes for command hooks
2021-10-10 13:08:05 +02:00
Nicola Murino
64e87d64bd
web client UI: allow to edit plain text files
Fixes #567
2021-10-09 14:17:28 +02:00
Nicola Murino
1b4a1fbbe5
add data retention check hook 2021-10-03 15:17:49 +02:00
Nicola Murino
cc134cad9a
data retention: allow to notify results via e-mail 2021-10-02 22:25:41 +02:00
Nicola Murino
e6f969cb04
web UI: update js and css deps 2021-09-30 10:23:25 +02:00
Nicola Murino
ba1febba73
rework user and admin profiles
users and admins can now also update their email and description
2021-09-29 18:46:15 +02:00
Maharanjan
0661876e99
Added email field for user account 2021-09-25 19:06:13 +02:00
Nicola Murino
da5a061b65
add basic REST APIs for data retention
Fixes #495
2021-09-25 12:20:31 +02:00
Nicola Murino
bf4b3e6840
httpd: move the check connection middleware before the logger middleware
Fixes #543
2021-09-19 08:14:59 +02:00
Nicola Murino
6ea38188e8
minor fixes and doc improvements 2021-09-18 10:50:17 +02:00
Nicola Murino
b5639a51fd
don't generate defender events for HTTP/WebDAV requests with no auth
it is quite common for HTTP clients to send a first request without
the Authorization header and then send the credentials after receiving
a 401 response. We don't want to generate defender events in this case
2021-09-11 18:23:11 +02:00
Nicola Murino
5c34d814d6
fix a possible nil pointer dereference
it can happen by upgrading from very old versions
2021-09-11 14:19:17 +02:00
Nicola Murino
7bad65a43e
user: add a permission to disable changing api key authentication
also implement the missing APIs to enable/disable api key authentication
2021-09-06 18:46:35 +02:00
Nicola Murino
101c2962ab
web client UI: add a permission to disable password change
Fixes #528
2021-09-05 18:49:13 +02:00
Nicola Murino
59140a6d51
add additional data to MFA secrets and fix pointers management 2021-09-05 14:10:12 +02:00
Nicola Murino
8a4c21b64a
add builtin two-factor auth support
The builtin two-factor authentication is based on time-based one time
passwords (RFC 6238) which works with Authy, Google Authenticator and
other compatible apps.
2021-09-04 12:11:04 +02:00
Nicola Murino
b903a6e46f
data provider: remove default admin
you need to load initial data or set "create_default_admin" to true
and the appropriate env vars if you don't want to use the web admin
setup screen to create the default admin
2021-08-20 10:37:51 +02:00
Nicola Murino
be3857d572
dataprovider: add timestamp fields for users and admins 2021-08-19 15:51:43 +02:00
Nicola Murino
fe953d6b38
REST API: add support for API key authentication 2021-08-17 18:08:32 +02:00
Nicola Murino
ced2e16f41
add support for password validation rules
Fixes #494
2021-08-06 18:56:07 +02:00
Nicola Murino
0503215e7a
web client: try to prevent browsers from caching requests
Fixes #493
2021-08-03 19:58:03 +02:00
Nicola Murino
90b324d707
Add a link on the login pages to switch between admin and web client login
The links are hidden if only the web admin or only thw web client is
enabled and can also be controlled using the "hide_login_url" setting

Fixes #485
2021-07-27 18:43:00 +02:00
Nicola Murino
3a22aae34f
web UI: add support for upload, create dirs, rename, delete 2021-07-26 20:55:49 +02:00
Nicola Murino
c41ae116eb
improve logging
Fixes #381
2021-07-24 20:11:17 +02:00
Nicola Murino
83c7453957
user API: allow to disable writes ...
... even if the user has permissions for these actions
2021-07-23 21:41:02 +02:00
Nicola Murino
85a47810ff
S3: expose more properties, possible backward incompatible change
Before these changes we implictly set S3ForcePathStyle if an endpoint
was provided.

This can cause issues with some S3 compatible object storages and must
be explicitly set now.

AWS is also deprecating this setting

https://aws.amazon.com/it/blogs/aws/amazon-s3-path-deprecation-plan-the-rest-of-the-story/
2021-07-23 16:56:48 +02:00
Nicola Murino
ae8ccadad2
users API: add API to create, delete, rename files and directories 2021-07-23 10:19:27 +02:00
Nicola Murino
776dffcf12
kms: improve modularity 2021-07-13 21:17:21 +02:00
Nicola Murino
e1a2451c22
s3: allow to configure the chunk download timeout 2021-07-11 18:39:45 +02:00
Nicola Murino
bd5191dfc5
add experimental plugin system 2021-07-11 15:26:51 +02:00
Nicola Murino
ff19879ffd
allow to use a persistent signing key for JWT and CSRF tokens
Fixes #466
2021-07-01 20:17:40 +02:00
Nicola Murino
076b2f0ee0
modules: add v2 support 2021-06-26 07:31:41 +02:00
Manuel Reithuber
fd4c388b23 added vfs.ListProviders() and using it in template fsconfig.html (added a new ListFSProviders template function for that) 2021-06-19 19:27:54 +02:00
Manuel Reithuber
88b10da596 updated utils.LoadTemplate() to call template.ParseFiles() directly and added a way to specify a base template (will be used in the next commit) 2021-06-19 19:27:54 +02:00
Manuel Reithuber
0ff010cc94 added vfs.GetProviderByName(), using it in for sftpgo portable and for parsing the webadmin form field 2021-06-19 19:27:54 +02:00
Nicola Murino
c1b862394d
move other errors to utils package 2021-06-19 13:06:01 +02:00
Manuel Reithuber
f19937b715
move Filesystem config validation to vfs 2021-06-19 12:24:43 +02:00
Nicola Murino
3bb0ca1d2b
config: remove deprecated configuration keys 2021-06-19 09:47:06 +02:00