beenull/sftpgo-mirror
1
0
Fork 0
mirror of https://github.com/drakkan/sftpgo.git synced 2024-11-21 23:20:24 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 6
1240 commits 8 branches 49 tags 49 MiB
Commit graph

339 commits

Author SHA1 Message Date
Nicola Murino
70e035315e
data provider: add CockroachDB support 2021-03-23 19:14:15 +01:00
Nicola Murino
5e375f56dd
kms: add a lock, secrets could be modified concurrently for cached users 
also reduce the size of the JSON payload omitting empty secrets
 2021-03-22 19:03:25 +01:00
Nicola Murino
f7c7e2951d
initialize argon params before creating the data provider 
Fixes #349
 2021-03-21 19:58:57 +01:00
Nicola Murino
d6dc3a507e
extend virtual folders support to all storage backends 
Fixes #241
 2021-03-21 19:15:47 +01:00
Nicola Murino
055506e518
sftpfs: add an option to disable concurrent reads 2021-03-06 15:41:40 +01:00
Nicola Murino
df41f0c556
add a setting to skip natural keys validation 
Enabling the "skip_natural_keys_validation" data provider setting,
the natural keys for REST API/Web Admin as usernames, admin names,
folder names are not restricted to unreserved URI chars

Fixes #334 #308
 2021-03-04 09:48:53 +01:00
Nicola Murino
534b253c20
WebDAV: improve TLS certificate authentication 
For each user you can now configure:

- TLS certificate auth
- TLS certificate auth and password
- Password auth

For TLS certificate auth, the certificate common name is used as
username
 2021-03-01 19:28:11 +01:00
Nicola Murino
a6e36e7cad
FTP: improve TLS certificate authentication 
For each user you can now configure:

- TLS certificate auth
- TLS certificate auth and password
- Password auth

For TLS auth, the certificate common name must match the name provided
using the "USER" FTP command
 2021-02-28 12:10:40 +01:00
Nicola Murino
ca3e15578e
Use new methods in the io and os packages instead of ioutil ones 
ioutil is deprecated in Go 1.16 and SFTPGo is an application, not
a library, we have no reason to keep compatibility with old Go
versions.

Go 1.16 fix some cifs related issues too.
 2021-02-25 21:53:04 +01:00
Nicola Murino
2146b83343
data providers: add filesystem to folder ... 
... and some descriptive fields.
The filesystem support for virtual folders will be implemented in
future commits
 2021-02-24 19:40:29 +01:00
Nicola Murino
8cc2dfe5c2
update pkg/sftp 
we don't need my branch anymore now that all the required features for
the sftpfs are available upstream too
 2021-02-22 16:27:45 +01:00
Nicola Murino
49830516be
squash database migrations and remove compat code 2021-02-22 08:37:50 +01:00
Nicola Murino
be9230e85b
micro optimizations spotted using the go-critic linter 2021-02-16 19:11:36 +01:00
Nicola Murino
b1ce6eb85b
web admin: allow to set an empty password for SFTPGo users 2021-02-15 19:38:53 +01:00
Nicola Murino
46176a54b4
minor doc fixes 2021-02-14 22:08:08 +01:00
Nicola Murino
1ac66d27b6
Use IEC units for byte counting everywhere 2021-02-12 22:16:35 +01:00
Nicola Murino
6a6e8fffbc
web hooks: improve resilience by adding a configurable retry 
the retryable http client is used for hooks that notify events
 2021-02-12 21:42:49 +01:00
Nicola Murino
51f110bc7b
sftpd: add statvfs@openssh.com support 2021-02-11 19:45:52 +01:00
Nicola Murino
db80781716
validation: improve error message for invalid chars 2021-02-08 21:32:59 +01:00
Nicola Murino
17e2b234a0
dataprovider: fix migration with old mysql versions 
Fixes #298
 2021-02-06 14:33:51 +01:00
Nicola Murino
267d9f1831
web ui: allow to create folders from a template 2021-02-04 19:09:43 +01:00
Nicola Murino
78bf808322
virtual folders: change dataprovider structure 
This way we no longer depend on the local file system path and so we can
add support for cloud backends in future updates
 2021-02-01 19:04:15 +01:00
Nicola Murino
46ab8f8d78
post-login hook: add the full user JSON serialized 
Fixes #284
 2021-01-26 18:05:44 +01:00
Nicola Murino
54321c5240
web ui: allow to create multiple users from a template 2021-01-25 21:31:33 +01:00
Nicola Murino
778ec9b88f
REST API v2 
- add JWT authentication
- admins are now stored inside the data provider
- admin access can be restricted based on the source IP: both proxy
  header and connection IP are checked
- deprecate REST API CLI: it is not relevant anymore

Some other changes to the REST API can still happen before releasing
SFTPGo 2.0.0

Fixes #197
 2021-01-17 22:29:08 +01:00
Nicola Murino
daac90c4e1
fix a potential race condition for pre-login and ext auth 
hooks

doing something like this:

err = provider.updateUser(u)
...
return provider.userExists(username)

could be racy if another update happen before

provider.userExists(username)

also pass a pointer to updateUser so if the user is modified inside
"validateUser" we can just return the modified user without do a new
query
 2021-01-05 09:50:22 +01:00
Nicola Murino
72b2c83392
defender: allow hot-reloading for safe and block lists 2021-01-04 17:52:14 +01:00
Nicola Murino
037d89a320
add support for a basic built-in defender 
It can help to prevent DoS and brute force password guessing
 2021-01-02 14:05:09 +01:00
Nicola Murino
e536a638c9
web UI: improve user cloning 2020-12-26 15:11:38 +01:00
Nicola Murino
72e3d464b8
sftpfs: fix fingerprints copy for memory provider 2020-12-12 10:56:02 +01:00
Nicola Murino
a6985075b9
add sftpfs storage backend 
Fixes #224
 2020-12-12 10:31:09 +01:00
Nicola Murino
50982229e1
REST API: add a method to get the status of the services 
added a status page to the built-in web admin
 2020-12-08 11:18:34 +01:00
Nicola Murino
c451f742aa
revertprovider: crypted provider was not supported in v4 
also ensure to initialize kms before the dataprovider, it could be
needed to downgrade secret from cloud kms providers
 2020-12-06 10:36:48 +01:00
Nicola Murino
034d89876d
webdav: fix proppatch handling 
also respect login delay for cached webdav users and check the home dir as
soon as the user authenticates

Fixes #239
 2020-12-06 08:19:41 +01:00
Nicola Murino
4a88ea5c03
add Data At Rest Encryption support 2020-12-05 13:48:13 +01:00
Nicola Murino
634b723b5d
add KMS support 
Fixes #226
 2020-11-30 21:46:34 +01:00
Nicola Murino
2142ef20c5
fix some typos 2020-11-26 22:18:12 +01:00
Nicola Murino
224ce5fe81
add revertprovider subcommand 
Fixes #233
 2020-11-26 22:08:33 +01:00
Nicola Murino
4bb9d07dde
user: add a free text field 
Fixes #230
 2020-11-25 22:26:34 +01:00
Nicola Murino
2054dfd83d
create the credential directory when needed 
The credentials dir is currently required only for GCS users if
prefer database credential setting is false, so defer its creation
and don't fail to start the services if this directory is missing
 2020-11-25 14:18:12 +01:00
Nicola Murino
6699f5c2cc
initial data loading: an error is no longer fatal 
therefore it does not prevent the services from starting
 2020-11-25 09:18:36 +01:00
Estel Smith
70bde8b2bc
memory provider: print a log if loading the initial dump fails 
therefore this error is no longer fatal and does not prevent the services
from starting

Fixes #229
 2020-11-25 09:15:23 +01:00
Nicola Murino
dccc583b5d
add a dedicated struct to store encrypted credentials 
also gcs credentials are now encrypted, both on disk and inside the
provider.

Data provider is automatically migrated and load data will accept
old format too but you should upgrade to the new format to avoid future
issues
 2020-11-22 21:53:04 +01:00
Nicola Murino
e3eca424f1
web admin: allow both allowed and denied extensions/patterns for a dir 
this fix a regression introduced in the previous commit
 2020-11-16 19:21:50 +01:00
Nicola Murino
a6355e298e
add support for limit files using shell like patterns 
Fixes #209
 2020-11-15 22:04:48 +01:00
Nicola Murino
0a14297b48
webdav: performance improvements and bug fixes 
we need my custom golang/x/net/webdav fork for now

https://github.com/drakkan/net/tree/sftpgo
 2020-11-04 19:11:40 +01:00
Nicola Murino
6ad4cc317c
cloud backends: stat and other performance improvements 2020-11-02 19:16:12 +01:00
Nicola Murino
ac3bae00fc
add support for SFTP subsystem mode 
Fixes #204
 2020-10-29 19:23:33 +01:00
Nicola Murino
e54828a7b8
add metrics for Azure Blob storage 2020-10-26 19:01:17 +01:00
Nicola Murino
5ff8f75917
add Azure Blob support 2020-10-25 08:18:48 +01:00
Sean Hildebrand
db7e81e9d0
add prefer_database_credentials configuration parameter 
When true, users' Google Cloud Storage credentials will be written to
the data provider instead of disk.
Pre-existing credentials on disk will be used as a fallback

Fixes #201
 2020-10-22 10:42:40 +02:00
Nicola Murino
ce9387f1ab
update dependencies and some docs 2020-10-09 20:25:42 +02:00
Mark Sagi-Kazar
5e2db77ef9 refactor: add an enum for filesystem providers 
Signed-off-by: Mark Sagi-Kazar <mark.sagikazar@gmail.com>
 2020-10-05 21:40:21 +02:00
Nicola Murino
c992072286
data provider: add a setting to prevent auto-update 2020-10-05 19:42:33 +02:00
Nicola Murino
3e2afc35ba
data provider: try to automatically initialize it if required 2020-10-05 12:55:49 +02:00
Ilias Trichopoulos
c65dd86d5e
Fix typos (#181) 2020-10-05 11:29:18 +02:00
Nicola Murino
f0c9b55036 dataprovider: improve user validation errors 
Fixes #170
 2020-09-18 19:21:24 +02:00
Nicola Murino
542554fb2c replace the library to verify UNIX's crypt(3) passwords 2020-09-04 21:08:09 +02:00
Nicola Murino
bdf18fa862 password hashing: exposes argon2 options 
So the hashing complexity can be changed depending on available
memory/CPU resources and business requirements
 2020-09-04 17:09:31 +02:00
Nicola Murino
a59163e56c multi-step auth: don't advertise password method if it is disabled 
also rename the settings to password_authentication so it is more like
OpenSSH, add some test cases and improve documentation
 2020-09-01 19:34:40 +02:00
Nicola Murino
3925c7ff95 REST API/Web admin: add a parameter to disconnect a user after an update 
This way you can force the user to login again and so to use the updated
configuration.

A deleted user will be automatically disconnected.

Fixes #163

Improved some docs too.
 2020-09-01 16:10:26 +02:00
Nicola Murino
dbed110d02 WebDAV: add caching for authenticated users 
In this way we get a big performance boost
 2020-08-31 19:25:17 +02:00
Nicola Murino
91a4c64390 fix initprovider exit code for MySQL and PostgreSQL 2020-08-30 14:00:45 +02:00
Nicola Murino
600a107699 initprovider: check if the provider is already initialized 
exit with code 0 if no initialization is required
 2020-08-30 13:50:43 +02:00
Nicola Murino
02e35ee002 sftpd: add Readlink support 2020-08-22 14:52:17 +02:00
Nicola Murino
8b0a1817b3 add check password hook 
its main use case is to allow to easily support things like password+OTP for
protocols without keyboard interactive support such as FTP and WebDAV
 2020-08-19 19:36:12 +02:00
Nicola Murino
f3228713bc Allow individual protocols to be enabled per user 
Fixes #154
 2020-08-17 12:49:20 +02:00
Nicola Murino
fa5333784b add a maximum allowed size for a single upload 2020-08-16 20:17:02 +02:00
Nicola Murino
aa0ed5dbd0 add post-login hook 
a login scope is supported too so you can get notifications for failed logins,
successful logins or both
 2020-08-12 16:15:12 +02:00
Nicola Murino
a9e21c282a add WebDAV support 
Fixes #147
 2020-08-11 23:56:10 +02:00
Nicola Murino
91dcc349de Add client IP address to external auth, pre-login and keyboard interactive hooks 2020-08-04 18:03:28 +02:00
Nicola Murino
93ce96d011 add support for the venerable FTP protocol 
Fixes #46
 2020-07-29 21:56:56 +02:00
Nicola Murino
4e41a5583d refactoring: add common package 
The common package defines the interfaces that a protocol must implement
and contain code that can be shared among supported protocols.

This way should be easier to support new protocols
 2020-07-24 23:39:38 +02:00
Nicola Murino
8fddb742df try to improve error message if the user forgot to initialize the provider 
See #138
 2020-07-09 20:01:37 +02:00
Nicola Murino
0ea2ca3141 simplify data provider usage 
remove the obsolete SQL scripts too. They are not required since v0.9.6
 2020-07-08 19:59:31 +02:00
Nicola Murino
42877dd915 sql providers: add a query timeout 2020-07-08 18:54:44 +02:00
Nicola Murino
cf541d62ea recursive permissions check before renaming/copying directories 2020-06-26 23:38:29 +02:00
Nicola Murino
b80abe6c05 return exit code 1 on error 
Fixes #132
 2020-06-20 14:30:46 +02:00
Nicola Murino
23a80b01b6 add build tag to disable metrics 2020-06-19 17:08:51 +02:00
Nicola Murino
3ceba7a147 sftpgo-copy: add quota limits check 2020-06-16 22:49:18 +02:00
Nicola Murino
37418a7630 SSH system commands: allow git and rsync inside virtual folders 2020-06-15 23:32:12 +02:00
Nicola Murino
8e22dd1b13 virtual folders: allow overlapped mapped paths if quota is disabled 
See #95
 2020-06-10 09:11:32 +02:00
Nicola Murino
01d681faa3 external auth: allow to map multiple login username to a single account 
some external auth users want to map multiple login usernames with a single
SGTPGo account.
For example an SFTP user logins using "user1" or "user2" and the external auth
returns "user" in both cases, so we use the username returned from external auth
and not the one used to login

Fixes #125
 2020-06-08 13:06:02 +02:00
Nicola Murino
8306b6bde6 refactor virtual folders 
The same virtual folder can now be shared among users and different
folder quota limits for each user are supported.

Fixes #120
 2020-06-07 23:30:18 +02:00
Nicola Murino
dc011af90d sftpd actions: add support for pre-delete action 
Fixes #121
 2020-05-24 23:31:14 +02:00
Nicola Murino
c27e3ef436 actions: add a generic hook to define external commands and HTTP URL 
We can only define a single hook now and it can be an HTTP notification
or an external command, not both
 2020-05-24 15:29:39 +02:00
Nicola Murino
ad53429cf1 add support for build tag to allow to disable some features 
The following build tags are available:

- "nogcs", disable Google Cloud Storage backend
- "nos3", disable S3 Compabible Object Storage backends
- "nobolt", disable Bolt data provider
- "nomysql", disable MySQL data provider
- "nopgsql", disable PostgreSQL data provider
- "nosqlite", disable SQLite data provider
- "noportable", disable portable mode
 2020-05-23 11:58:05 +02:00
Nicola Murino
738c7ab43e sftpd: add support for SSH user certificate authentication 
This add support for PROTOCOL.certkeys vendor extension:

https://cvsweb.openbsd.org/src/usr.bin/ssh/PROTOCOL.certkeys?rev=1.8

Fixes #117

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
 2020-05-15 20:08:53 +02:00
Nicola Murino
f02e24437a add more linters 
test cases migration to testify is now complete.
Linters are enabled for test cases too
 2020-05-06 19:36:34 +02:00
Nicola Murino
3f75d46a16 sftpd: add support for excluding virtual folders from user quota limit 
Fixes #110
 2020-05-01 15:27:53 +02:00
Enes Çakır
14196167b0
add github action workflow for code quality 2020-04-30 15:06:15 +02:00
Nicola Murino
d70959c34c fix some lint issues 2020-04-30 14:23:55 +02:00
Nicola Murino
d377181b25 add a new configuration section for HTTP clients 
HTTP clients are used for executing hooks such as the ones used for custom
actions, external authentication and pre-login user modifications.

This allows, for example, to use self-signed certificate without defeating the
purpose of using TLS
 2020-04-26 23:29:09 +02:00
Nicola Murino
37357b2d63 add support for checking pbkdf2 passwords with base64 encoded salt 
This way we can import the default passwords format used in 389ds.

See TestPasswordsHashPbkdf2Sha256_389DS test case to learn how to convert
389ds passwords
 2020-04-11 12:25:21 +02:00
Nicola Murino
9b06e0a3b7 sql providers: change password field from varchar 255 to text 
some passwords can be longer than 255 characters
 2020-04-11 11:17:40 +02:00
Nicola Murino
b1c7317cf6 add support for partial authentication 
Multi-step authentication is activated disabling all single-step
auth methods for a given user
 2020-04-09 23:32:42 +02:00
Nicola Murino
c93d8cecfc update deps 
chi 4.1.0 requires some minor code changes
 2020-04-03 22:30:30 +02:00
Nicola Murino
9046acbe68 add HTTP hooks 
external auth, pre-login user modification and keyboard interactive
authentication is now supported via HTTP requests too
 2020-04-01 23:25:23 +02:00
Nicola Murino
b52d078986 pbkdf2: fix password comparison 
the key len for the derived function must be equal to the len of the
expected key
 2020-03-28 16:09:06 +01:00
Nicola Murino
0a9c4914aa pre-login program: allow to create a new user too 
clarify the difference between dynamic user creation/update and external
authentication
 2020-03-27 23:26:22 +01:00
Nicola Murino
0787e3e595 bolt provider: fix error handling for get users with username filter 2020-03-22 15:37:08 +01:00
Nicola Murino
c1194d558c docs: minor improvements 2020-03-22 14:03:06 +01:00
Nicola Murino
f55851bdc8 update nathanaelle password to v2 
Fixes #97
 2020-03-20 17:25:38 +01:00
Nicola Murino
1770da545d s3: upload concurrency is now configurable 
Please note that if the upload bandwidth between the SFTP client and
SFTPGo is greater than the upload bandwidth between SFTPGo and S3 then
the SFTP client have to wait for the upload of the last parts to S3
after it ends the file upload to SFTPGo, and it may time out.
Keep this in mind if you customize parts size and upload concurrency
 2020-03-13 19:13:58 +01:00
Germs2004
6ad9c5ae64 minor typo 2020-03-04 08:00:34 +01:00
Nicola Murino
016abda6d7 improve docs 2020-03-03 23:25:23 +01:00
Nicola Murino
7f1946de34 improve validations for user provided file and directory paths 2020-03-03 09:09:58 +01:00
Nicola Murino
d0a81cabab log file: if the path is not absolute make it relative to config dir 
Also refuse to join invalid file name such as "."

Fixes #85
 2020-03-03 00:34:06 +01:00
Nicola Murino
b885d453a2 filters: we can now set allowed and denied files extensions 2020-03-01 22:10:29 +01:00
Mengsk
637463a068 Rename before_login_program to pre_login_program 
and some documentation update
 2020-02-25 16:34:54 +01:00
Nicola Murino
e69536f540 fixed some typos and improved a log 2020-02-25 12:46:52 +01:00
Nicola Murino
703ccc8d91 add support for dynamic users modifications 
A custom program can be executed before the users login to modify the
configurations for the user trying to login.
You can, for example, allow login based on time range.

Fixes #77
 2020-02-23 18:50:59 +01:00
Nicola Murino
45b9366dd0 add support for virtual folders 
directories outside the user home directory can be exposed as virtual folders
 2020-02-23 11:30:26 +01:00
Nicola Murino
bc11cdd8d5 add support for per user authentication methods 
You can, for example, deny one or more authentication methods to one or
more users.
 2020-02-19 22:39:30 +01:00
Nicola Murino
ae8ed75ae5 gcs: add support for automatic credentials 
We can now also support implicit credentials using the Application
Default Credentials strategy
 2020-02-19 09:41:15 +01:00
Nicola Murino
79c8b6cbc2 keyboard interactive auth: allows to automatically check the user password 
This simplify the common pattern where the user password and a one time
token is requested: now the external program can delegate password check
to SFTPGo and verify the token itself
 2020-02-16 11:43:52 +01:00
Nicola Murino
58253968fc s3: improve credentials validation 
access secret can now be empty, so check if not empty before encrypting
the secret
 2020-02-16 10:14:44 +01:00
Nicola Murino
8eff2df39c subdir perms: allow empty perms 
empty perms will allow nothing on the specified subdir.

Non empty permissions for the "/" dir are still required.

Fixes #70
 2020-02-10 19:28:35 +01:00
Nicola Murino
d6fa853a37 add support for integrated database schema migrations 
added the "initprovider" command to initialize the database structure.
If we change the database schema the required changes will be checked
at startup and automatically applyed.
 2020-02-08 14:44:25 +01:00
Nicola Murino
553cceab42 dataprovider actions: add more users fiels as env vars 2020-02-05 22:17:03 +01:00
Nicola Murino
8b039e0447 httpd: add support for basic auth and HTTPS 2020-02-04 00:08:00 +01:00
Nicola Murino
bcaf283c35 memory provider: load users from a dump file 
The `memory` provider can load users from a dump obtained using the
`dumpdata` REST API. This dump file can be configured using the
dataprovider `name` configuration key. It will be loaded at startup
and can be reloaded on demand using a `SIGHUP` on Unix based systems
and a `paramchange` request to the running service on Windows.

Fixes #66
 2020-02-02 22:20:39 +01:00
Nicola Murino
31a433cda2 update deps and simplify some code 2020-01-31 23:26:56 +01:00
Nicola Murino
3491717c26 add support for serving Google Cloud Storage over SFTP/SCP 
Each user can be mapped with a Google Cloud Storage bucket or a bucket
virtual folder
 2020-01-31 19:04:00 +01:00
Nicola Murino
7ebbbe5c29 S3: update pipeat to the latest commit in my fork 
Here are the main improvements:

- unliked files works on windows too
- the uploads are now synced on the lower speed between the SFTP client write
and the upload speed to S3

This commit increase the external auth timeout to 60 seconds too
 2020-01-22 19:42:23 +01:00
Nicola Murino
9ff303b8c0 add support for keyboard interactive authentication 
Fixes #64
 2020-01-21 10:54:05 +01:00
Nicola Murino
4463421028 S3: add support for serving virtual folders 
inside the same bucket each user can be assigned to a virtual folder.
This is similar to a chroot directory for local filesystem
 2020-01-19 23:23:09 +01:00
Nicola Murino
d75f56b914 vfs: store root dir 
so we don't need to pass it over and over
 2020-01-19 13:58:55 +01:00
Nicola Murino
a4834f4a83 add basic S3-Compatible Object Storage support 
we have now an interface for filesystem backeds, this make easy to add
new filesystem backends
 2020-01-19 07:41:05 +01:00
Nicola Murino
eec60d6309 custom actions: add env vars 
action parameters can now be readed from env vars too.
Added a timeout for the command execution
 2020-01-09 12:00:37 +01:00
Paweł K
37c602a477 Expand environment variables for external auth program 
Use os.Environ() as a base instead of empty variable. Currently the environment of executed external auth program only contains SFTPGO_AUTHD* variables and therefore the program lacks additional context when started.
 2020-01-09 08:33:40 +01:00
Nicola Murino
8e604f888a improve docs and test cases 2020-01-07 09:39:20 +01:00
Nicola Murino
531091906d add support for authentication using external programs 
Fixes #62
 2020-01-06 21:42:41 +01:00
Nicola Murino
e046b35b97 check permissions against sftp path 
instead of building filesystem paths and then checking permissions against
path relative to the home dir that is the initial sftp path
 2020-01-05 11:41:25 +01:00
Nicola Murino
1b1c740b29 Add support for allowed/denied IP/Mask 
Login can be restricted to specific ranges of IP address or to a specific IP
address.

Please apply the appropriate SQL upgrade script to add the filter field to your
database.

The filter database field will allow to add other filters without requiring a
new database migration
 2019-12-30 18:37:50 +01:00
Nicola Murino
20606a0043 sftpd test: add a debug log 
The git push test sometime fails when running on travis.
The issue cannot be replicated locally so print the logs to try to
understand what is happening
 2019-12-29 23:27:32 +01:00
Nicola Murino
80e9902324 scripts: add support for converting users from other users stores 
currently we support:

- Linux/Unix users stored in shadow/passwd files
- Pure-FTPd virtual users generated using `pure-pw` CLI
- ProFTPD users generated using `ftpasswd` CLI
 2019-12-29 17:21:25 +01:00
Nicola Murino
6aff8c2f5e add support for checking passwords in md5crypt ($1$) format 
this is an old and unsafe schema but it is still useful to import users
from legacy systems
 2019-12-29 07:43:59 +01:00
Nicola Murino
ae094d3479 add backup/restore REST API 2019-12-27 23:12:44 +01:00
Nicola Murino
489101668c add per directory permissions 
we can now have permissions such as these ones

{"/":["*"],"/somedir":["list","download"]}

The old permissions are automatically converted to the new structure,
no database migration is needed
 2019-12-25 18:20:19 +01:00
Nicola Murino
0a025aabfd add support for Git over SSH 
We use the system commands "git-receive-pack", "git-upload-pack" and
"git-upload-archive". they need to be installed and in your system's
PATH. Since we execute system commands we have no direct control on
file creation/deletion and so quota check is suboptimal: if quota is
enabled, the number of files is checked at the command begin and not
while new files are created.
The allowed size is calculated as the difference between the max quota
and the used one. The command is aborted if it uploads more bytes than
the remaining allowed size calculated at the command start. Quotas are
recalculated at the command end with a full home directory scan, this
could be heavy for big directories.
 2019-11-26 22:26:42 +01:00
Nicola Murino
ca6cb34d98 sftpd: add support for chtimes 
This improve rclone compatibility
 2019-11-16 10:23:41 +01:00
Nicola Murino
3ac5af47f2 minor fixes 2019-11-15 12:26:52 +01:00
Nicola Murino
bb37a1c1ce sftpd: add support for chmod/chown 
added matching permissions too and a new setting "setstat_mode".
Setting setstat_mode to 1 you can keep the previous behaviour that
silently ignore setstat requests
 2019-11-15 12:15:07 +01:00
Nicola Murino
206799ff1c httpd: add an API to get data provider status 2019-11-14 18:48:01 +01:00
Nicola Murino
f3de83707f improve README 2019-11-14 17:43:14 +01:00
Nicola Murino
acdf351047 dataprovider: add custom command and/or HTTP notifications on users add, update and delete 
This way custom logic can be implemented for example to create a UNIX user
as asked in #58
 2019-11-14 11:06:03 +01:00
Nicola Murino
c2ff50c917 dataprovider: add support for user status and expiration 
an user can now be disabled or expired.

If you are using an SQL database as dataprovider please remember to
execute the sql update script inside "sql" folder.

Fixes #57
 2019-11-13 11:36:21 +01:00
Nicola Murino
8cd0aec417 add memory data provider and use it for portable mode 2019-10-25 18:37:12 +02:00
Nicola Murino
a4cddf4f7f add portable mode 
Portable mode is a convenient way to share a single directory on demand
 2019-10-24 18:50:35 +02:00
Nicola Murino
afd312f26a add a basic web interface 
The builtin web interface allows to manage users and connections
 2019-10-07 18:19:01 +02:00
Nicola Murino
bb0338870a unhide public keys 
hiding public keys give no security improvement
 2019-10-03 15:29:54 +02:00
Nicola Murino
fb8ccfe824 improve doc for dataprovider Close method 
the method is used in test cases, it assumes that the dataprovider
is initialized and it is not safe Initialize/Close from different
goroutines
 2019-09-29 08:38:09 +02:00
Nicola Murino
0b4ff97a1a dataprovider: simplifly validateUser 2019-09-28 23:44:36 +02:00
Nicola Murino
00dd5db226 add support for users' default base dir 2019-09-28 22:48:52 +02:00
Nicola Murino
df96ea7e9f add a new permission for overwriting existing files 
The upload permission is required to allow file overwrite
 2019-09-17 08:53:45 +02:00
Nicola Murino
bba78763e1 add a test case and document sha512crypt passwords support 2019-09-15 12:23:27 +02:00
Seunghoon Yeon
b5211fd31b Added sha512crypt support 2019-09-15 12:08:28 +02:00
Nicola Murino
018b10808d improve SQL provider availability check adding a timeout 2019-09-14 16:18:31 +02:00
Jo Vandeginste
29aadbf3e3 log redacted passwords 
Closes #48
 2019-09-13 21:57:52 +02:00
Nicola Murino
7eb5b01169 add Prometheus support 
some basic counters and gauges are now exposed
 2019-09-13 18:45:36 +02:00
Nicola Murino
e7eb3476b7 dataprovider: remove transaction for quota update 
The update is atomic so no transaction is needed.
Addionally a transaction will ask for a new connection to the pool
and this can deadlock if the pool has a max connection limit too low.

Also make configurable the pool size instead of hard code to the cpu number.

Fixes #47
 2019-09-13 08:14:07 +02:00
Nicola Murino
bf00ca334d mysql data provider: add read and write timeout 
This should hopefully fix #47
 2019-09-12 16:26:47 +02:00
Nicola Murino
4f1c2c094f improve logging 
this partially revert #45
 2019-09-06 15:19:01 +02:00
Jo Vandeginste
abbb7f272b Refactor the logging system 
* created a "Log" function for type "Connection"
* created a "log" function for type "Provider"
* replace logger calls to Log/log where possible

I also renamed PGSSQL to PGSQL, as this seemed to be a typo

Signed-off-by: Jo Vandeginste <Jo.Vandeginste@kuleuven.be>
 2019-09-06 13:43:21 +02:00
Nicola Murino
a26343a694 improve more logs 2019-09-05 23:42:00 +02:00
Jo Vandeginste
25260297aa Show info about public key during login 
This will show the key fingerprint and the associated comment, or
"password" when password was used, during login.

Eg.:

```
message":"User id: 1, logged in with: \"public_key:SHA256:FV3+wlAKGzYy7+J02786fh8N8c06+jga/mdiSOSPT7g:jo@desktop\",
```

or

```
message":"User id: 1, logged in with: \"password\",
...`

Signed-off-by: Jo Vandeginste <Jo.Vandeginste@kuleuven.be>
 2019-09-05 22:58:14 +02:00
Jo Vandeginste
0737c672f5 Add connectionID to as many entries as possible 
Signed-off-by: Jo Vandeginste <Jo.Vandeginste@kuleuven.be>
 2019-09-05 22:58:14 +02:00
Nicola Murino
133f2e8601 add support for checking pbkdf2 passwords 2019-08-17 15:20:49 +02:00
Nicola Murino
9d342cb125 dataprovider: remove clear text password checking 
passwords must be stored using argon2id or bcrypt
 2019-08-17 08:44:43 +02:00
Nicola Murino
96a39a36bb dataprovider: add support for bbolt key/value store 
This way there is an alternative for embedded/small systems if CGO
is disabled at build time and so SQLite support cannot be compiled
 2019-08-12 18:31:31 +02:00
Nicola Murino
cb87fe811a dataprovider move db handle to provider struct 
This is needed to support non SQL providers
 2019-08-11 14:53:37 +02:00
Nicola Murino
90607d4f86 sftpd stats: add file path for active upload/download 2019-08-08 19:33:16 +02:00
Nicola Murino
2aca4479a5 rename public_key in public_keys 
remove compatibility layer to convert public keys newline delimited
in json list
 2019-08-07 23:41:10 +02:00
Nicola Murino
8f421b7d0f switch to viper for configuration and use cobra for cli 2019-08-07 22:46:52 +02:00
Nicola Murino
8d4964c16d convert public key from newline delimited string to a real array 
Added a compatibility layer that will convert newline delimited keys to array
when the user is fetched from the database.
This code will be removed in future versions please update your public keys,
you only need to resave the users using the REST API.
 2019-08-01 22:42:46 +02:00
Jo Vandeginste
170e3113e1 When path starts with slash, consider it absolute 
When SQLite path starts with a `/`, we consider this to be an absolute path.

Eg.:

```json
{
  "data_provider":{
    "name":"/var/lib/sftpgo/sftpgo.db"
  }
}
```
 2019-07-31 17:14:46 +02:00
Jo Vandeginste
c752dd8e81 Support multiple public keys 
This will parse the public key field as a newline delimited list of public keys.

Return (valid) result on first match.
 2019-07-31 15:47:51 +02:00
Nicola Murino
8058178ea0 Improve documentation 2019-07-30 20:51:29 +02:00
Nicola Murino
7dd03ee24e document bcrypt support 2019-07-29 09:20:33 +02:00
Philip Hofstetter
be9a1fb7c4 add bcrypt support 2019-07-29 08:11:21 +02:00
Nicola Murino
645c4b4fa2 fix MySQL provider initialization 
this not caused any issue since the queries are basically the same
 2019-07-28 23:47:57 +02:00
Nicola Murino
0bed768909 allow quota tracking only for users with quota restrictions 
Setting "track_quota" to 2 in sftpgo.conf will avoid a database update
after each file upload/deletion if the user has no quota restrictions
 2019-07-28 22:04:50 +02:00
Nicola Murino
9987821003 rename last_quota_scan to last_quota_update 
Existing databases must be updated using the script 20190728.sql
 2019-07-28 19:29:32 +02:00
Nicola Murino
c547c5c9ee quota tracking: fix concurrent updates 
added a test case to check quota size for upload that replace an existing file
 2019-07-28 13:24:46 +02:00
Tim Sedlmeyer
eb18e30a65 Fix some spelling errors 2019-07-26 16:20:40 +02:00
Nicola Murino
417b173c78 Improve test cases and logging 2019-07-26 11:35:12 +02:00
Nicola Murino
22b8dc7f1d fix misspells and ineffectual assignments 2019-07-20 21:17:53 +02:00
Nicola Murino
31cd4d7139 first version 2019-07-20 12:26:52 +02:00