Commit graph

130 commits

Author SHA1 Message Date
Nicola Murino
738c7ab43e sftpd: add support for SSH user certificate authentication
This add support for PROTOCOL.certkeys vendor extension:

https://cvsweb.openbsd.org/src/usr.bin/ssh/PROTOCOL.certkeys?rev=1.8

Fixes #117

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2020-05-15 20:08:53 +02:00
Nicola Murino
f02e24437a add more linters
test cases migration to testify is now complete.
Linters are enabled for test cases too
2020-05-06 19:36:34 +02:00
Nicola Murino
3f75d46a16 sftpd: add support for excluding virtual folders from user quota limit
Fixes #110
2020-05-01 15:27:53 +02:00
Enes Çakır
14196167b0
add github action workflow for code quality 2020-04-30 15:06:15 +02:00
Nicola Murino
d70959c34c fix some lint issues 2020-04-30 14:23:55 +02:00
Nicola Murino
d377181b25 add a new configuration section for HTTP clients
HTTP clients are used for executing hooks such as the ones used for custom
actions, external authentication and pre-login user modifications.

This allows, for example, to use self-signed certificate without defeating the
purpose of using TLS
2020-04-26 23:29:09 +02:00
Nicola Murino
37357b2d63 add support for checking pbkdf2 passwords with base64 encoded salt
This way we can import the default passwords format used in 389ds.

See TestPasswordsHashPbkdf2Sha256_389DS test case to learn how to convert
389ds passwords
2020-04-11 12:25:21 +02:00
Nicola Murino
b1c7317cf6 add support for partial authentication
Multi-step authentication is activated disabling all single-step
auth methods for a given user
2020-04-09 23:32:42 +02:00
Nicola Murino
c93d8cecfc update deps
chi 4.1.0 requires some minor code changes
2020-04-03 22:30:30 +02:00
Nicola Murino
9046acbe68 add HTTP hooks
external auth, pre-login user modification and keyboard interactive
authentication is now supported via HTTP requests too
2020-04-01 23:25:23 +02:00
Nicola Murino
b52d078986 pbkdf2: fix password comparison
the key len for the derived function must be equal to the len of the
expected key
2020-03-28 16:09:06 +01:00
Nicola Murino
0a9c4914aa pre-login program: allow to create a new user too
clarify the difference between dynamic user creation/update and external
authentication
2020-03-27 23:26:22 +01:00
Nicola Murino
c1194d558c docs: minor improvements 2020-03-22 14:03:06 +01:00
Nicola Murino
f55851bdc8 update nathanaelle password to v2
Fixes #97
2020-03-20 17:25:38 +01:00
Nicola Murino
016abda6d7 improve docs 2020-03-03 23:25:23 +01:00
Nicola Murino
b885d453a2 filters: we can now set allowed and denied files extensions 2020-03-01 22:10:29 +01:00
Mengsk
637463a068 Rename before_login_program to pre_login_program
and some documentation update
2020-02-25 16:34:54 +01:00
Nicola Murino
e69536f540 fixed some typos and improved a log 2020-02-25 12:46:52 +01:00
Nicola Murino
703ccc8d91 add support for dynamic users modifications
A custom program can be executed before the users login to modify the
configurations for the user trying to login.
You can, for example, allow login based on time range.

Fixes #77
2020-02-23 18:50:59 +01:00
Nicola Murino
45b9366dd0 add support for virtual folders
directories outside the user home directory can be exposed as virtual folders
2020-02-23 11:30:26 +01:00
Nicola Murino
bc11cdd8d5 add support for per user authentication methods
You can, for example, deny one or more authentication methods to one or
more users.
2020-02-19 22:39:30 +01:00
Nicola Murino
ae8ed75ae5 gcs: add support for automatic credentials
We can now also support implicit credentials using the Application
Default Credentials strategy
2020-02-19 09:41:15 +01:00
Nicola Murino
79c8b6cbc2 keyboard interactive auth: allows to automatically check the user password
This simplify the common pattern where the user password and a one time
token is requested: now the external program can delegate password check
to SFTPGo and verify the token itself
2020-02-16 11:43:52 +01:00
Nicola Murino
58253968fc s3: improve credentials validation
access secret can now be empty, so check if not empty before encrypting
the secret
2020-02-16 10:14:44 +01:00
Nicola Murino
8eff2df39c subdir perms: allow empty perms
empty perms will allow nothing on the specified subdir.

Non empty permissions for the "/" dir are still required.

Fixes #70
2020-02-10 19:28:35 +01:00
Nicola Murino
d6fa853a37 add support for integrated database schema migrations
added the "initprovider" command to initialize the database structure.
If we change the database schema the required changes will be checked
at startup and automatically applyed.
2020-02-08 14:44:25 +01:00
Nicola Murino
553cceab42 dataprovider actions: add more users fiels as env vars 2020-02-05 22:17:03 +01:00
Nicola Murino
8b039e0447 httpd: add support for basic auth and HTTPS 2020-02-04 00:08:00 +01:00
Nicola Murino
bcaf283c35 memory provider: load users from a dump file
The `memory` provider can load users from a dump obtained using the
`dumpdata` REST API. This dump file can be configured using the
dataprovider `name` configuration key. It will be loaded at startup
and can be reloaded on demand using a `SIGHUP` on Unix based systems
and a `paramchange` request to the running service on Windows.

Fixes #66
2020-02-02 22:20:39 +01:00
Nicola Murino
31a433cda2 update deps and simplify some code 2020-01-31 23:26:56 +01:00
Nicola Murino
3491717c26 add support for serving Google Cloud Storage over SFTP/SCP
Each user can be mapped with a Google Cloud Storage bucket or a bucket
virtual folder
2020-01-31 19:04:00 +01:00
Nicola Murino
7ebbbe5c29 S3: update pipeat to the latest commit in my fork
Here are the main improvements:

- unliked files works on windows too
- the uploads are now synced on the lower speed between the SFTP client write
and the upload speed to S3

This commit increase the external auth timeout to 60 seconds too
2020-01-22 19:42:23 +01:00
Nicola Murino
9ff303b8c0 add support for keyboard interactive authentication
Fixes #64
2020-01-21 10:54:05 +01:00
Nicola Murino
4463421028 S3: add support for serving virtual folders
inside the same bucket each user can be assigned to a virtual folder.
This is similar to a chroot directory for local filesystem
2020-01-19 23:23:09 +01:00
Nicola Murino
a4834f4a83 add basic S3-Compatible Object Storage support
we have now an interface for filesystem backeds, this make easy to add
new filesystem backends
2020-01-19 07:41:05 +01:00
Nicola Murino
eec60d6309 custom actions: add env vars
action parameters can now be readed from env vars too.
Added a timeout for the command execution
2020-01-09 12:00:37 +01:00
Paweł K
37c602a477 Expand environment variables for external auth program
Use os.Environ() as a base instead of empty variable. Currently the environment of executed external auth program only contains SFTPGO_AUTHD* variables and therefore the program lacks additional context when started.
2020-01-09 08:33:40 +01:00
Nicola Murino
8e604f888a improve docs and test cases 2020-01-07 09:39:20 +01:00
Nicola Murino
531091906d add support for authentication using external programs
Fixes #62
2020-01-06 21:42:41 +01:00
Nicola Murino
1b1c740b29 Add support for allowed/denied IP/Mask
Login can be restricted to specific ranges of IP address or to a specific IP
address.

Please apply the appropriate SQL upgrade script to add the filter field to your
database.

The filter database field will allow to add other filters without requiring a
new database migration
2019-12-30 18:37:50 +01:00
Nicola Murino
20606a0043 sftpd test: add a debug log
The git push test sometime fails when running on travis.
The issue cannot be replicated locally so print the logs to try to
understand what is happening
2019-12-29 23:27:32 +01:00
Nicola Murino
80e9902324 scripts: add support for converting users from other users stores
currently we support:

- Linux/Unix users stored in shadow/passwd files
- Pure-FTPd virtual users generated using `pure-pw` CLI
- ProFTPD users generated using `ftpasswd` CLI
2019-12-29 17:21:25 +01:00
Nicola Murino
6aff8c2f5e add support for checking passwords in md5crypt ($1$) format
this is an old and unsafe schema but it is still useful to import users
from legacy systems
2019-12-29 07:43:59 +01:00
Nicola Murino
ae094d3479 add backup/restore REST API 2019-12-27 23:12:44 +01:00
Nicola Murino
489101668c add per directory permissions
we can now have permissions such as these ones

{"/":["*"],"/somedir":["list","download"]}

The old permissions are automatically converted to the new structure,
no database migration is needed
2019-12-25 18:20:19 +01:00
Nicola Murino
0a025aabfd add support for Git over SSH
We use the system commands "git-receive-pack", "git-upload-pack" and
"git-upload-archive". they need to be installed and in your system's
PATH. Since we execute system commands we have no direct control on
file creation/deletion and so quota check is suboptimal: if quota is
enabled, the number of files is checked at the command begin and not
while new files are created.
The allowed size is calculated as the difference between the max quota
and the used one. The command is aborted if it uploads more bytes than
the remaining allowed size calculated at the command start. Quotas are
recalculated at the command end with a full home directory scan, this
could be heavy for big directories.
2019-11-26 22:26:42 +01:00
Nicola Murino
ca6cb34d98 sftpd: add support for chtimes
This improve rclone compatibility
2019-11-16 10:23:41 +01:00
Nicola Murino
bb37a1c1ce sftpd: add support for chmod/chown
added matching permissions too and a new setting "setstat_mode".
Setting setstat_mode to 1 you can keep the previous behaviour that
silently ignore setstat requests
2019-11-15 12:15:07 +01:00
Nicola Murino
206799ff1c httpd: add an API to get data provider status 2019-11-14 18:48:01 +01:00
Nicola Murino
f3de83707f improve README 2019-11-14 17:43:14 +01:00
Nicola Murino
acdf351047 dataprovider: add custom command and/or HTTP notifications on users add, update and delete
This way custom logic can be implemented for example to create a UNIX user
as asked in #58
2019-11-14 11:06:03 +01:00
Nicola Murino
c2ff50c917 dataprovider: add support for user status and expiration
an user can now be disabled or expired.

If you are using an SQL database as dataprovider please remember to
execute the sql update script inside "sql" folder.

Fixes #57
2019-11-13 11:36:21 +01:00
Nicola Murino
8cd0aec417 add memory data provider and use it for portable mode 2019-10-25 18:37:12 +02:00
Nicola Murino
a4cddf4f7f add portable mode
Portable mode is a convenient way to share a single directory on demand
2019-10-24 18:50:35 +02:00
Nicola Murino
afd312f26a add a basic web interface
The builtin web interface allows to manage users and connections
2019-10-07 18:19:01 +02:00
Nicola Murino
fb8ccfe824 improve doc for dataprovider Close method
the method is used in test cases, it assumes that the dataprovider
is initialized and it is not safe Initialize/Close from different
goroutines
2019-09-29 08:38:09 +02:00
Nicola Murino
0b4ff97a1a dataprovider: simplifly validateUser 2019-09-28 23:44:36 +02:00
Nicola Murino
00dd5db226 add support for users' default base dir 2019-09-28 22:48:52 +02:00
Nicola Murino
df96ea7e9f add a new permission for overwriting existing files
The upload permission is required to allow file overwrite
2019-09-17 08:53:45 +02:00
Nicola Murino
bba78763e1 add a test case and document sha512crypt passwords support 2019-09-15 12:23:27 +02:00
Seunghoon Yeon
b5211fd31b Added sha512crypt support 2019-09-15 12:08:28 +02:00
Nicola Murino
7eb5b01169 add Prometheus support
some basic counters and gauges are now exposed
2019-09-13 18:45:36 +02:00
Nicola Murino
e7eb3476b7 dataprovider: remove transaction for quota update
The update is atomic so no transaction is needed.
Addionally a transaction will ask for a new connection to the pool
and this can deadlock if the pool has a max connection limit too low.

Also make configurable the pool size instead of hard code to the cpu number.

Fixes #47
2019-09-13 08:14:07 +02:00
Nicola Murino
4f1c2c094f improve logging
this partially revert #45
2019-09-06 15:19:01 +02:00
Jo Vandeginste
abbb7f272b Refactor the logging system
* created a "Log" function for type "Connection"
* created a "log" function for type "Provider"
* replace logger calls to Log/log where possible

I also renamed PGSSQL to PGSQL, as this seemed to be a typo

Signed-off-by: Jo Vandeginste <Jo.Vandeginste@kuleuven.be>
2019-09-06 13:43:21 +02:00
Jo Vandeginste
25260297aa Show info about public key during login
This will show the key fingerprint and the associated comment, or
"password" when password was used, during login.

Eg.:

```
message":"User id: 1, logged in with: \"public_key:SHA256:FV3+wlAKGzYy7+J02786fh8N8c06+jga/mdiSOSPT7g:jo@desktop\",
```

or

```
message":"User id: 1, logged in with: \"password\",
...`

Signed-off-by: Jo Vandeginste <Jo.Vandeginste@kuleuven.be>
2019-09-05 22:58:14 +02:00
Jo Vandeginste
0737c672f5 Add connectionID to as many entries as possible
Signed-off-by: Jo Vandeginste <Jo.Vandeginste@kuleuven.be>
2019-09-05 22:58:14 +02:00
Nicola Murino
133f2e8601 add support for checking pbkdf2 passwords 2019-08-17 15:20:49 +02:00
Nicola Murino
9d342cb125 dataprovider: remove clear text password checking
passwords must be stored using argon2id or bcrypt
2019-08-17 08:44:43 +02:00
Nicola Murino
96a39a36bb dataprovider: add support for bbolt key/value store
This way there is an alternative for embedded/small systems if CGO
is disabled at build time and so SQLite support cannot be compiled
2019-08-12 18:31:31 +02:00
Nicola Murino
cb87fe811a dataprovider move db handle to provider struct
This is needed to support non SQL providers
2019-08-11 14:53:37 +02:00
Nicola Murino
2aca4479a5 rename public_key in public_keys
remove compatibility layer to convert public keys newline delimited
in json list
2019-08-07 23:41:10 +02:00
Nicola Murino
8f421b7d0f switch to viper for configuration and use cobra for cli 2019-08-07 22:46:52 +02:00
Nicola Murino
8d4964c16d convert public key from newline delimited string to a real array
Added a compatibility layer that will convert newline delimited keys to array
when the user is fetched from the database.
This code will be removed in future versions please update your public keys,
you only need to resave the users using the REST API.
2019-08-01 22:42:46 +02:00
Jo Vandeginste
c752dd8e81 Support multiple public keys
This will parse the public key field as a newline delimited list of public keys.

Return (valid) result on first match.
2019-07-31 15:47:51 +02:00
Nicola Murino
8058178ea0 Improve documentation 2019-07-30 20:51:29 +02:00
Philip Hofstetter
be9a1fb7c4 add bcrypt support 2019-07-29 08:11:21 +02:00
Nicola Murino
645c4b4fa2 fix MySQL provider initialization
this not caused any issue since the queries are basically the same
2019-07-28 23:47:57 +02:00
Nicola Murino
0bed768909 allow quota tracking only for users with quota restrictions
Setting "track_quota" to 2 in sftpgo.conf will avoid a database update
after each file upload/deletion if the user has no quota restrictions
2019-07-28 22:04:50 +02:00
Nicola Murino
31cd4d7139 first version 2019-07-20 12:26:52 +02:00