beenull/sftpgo
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
1081 commits 8 branches 48 tags 35 MiB
Commit graph

333 commits

Author SHA1 Message Date
Nicola Murino
acdf351047 dataprovider: add custom command and/or HTTP notifications on users add, update and delete 
This way custom logic can be implemented for example to create a UNIX user
as asked in #58
 2019-11-14 11:06:03 +01:00
Nicola Murino
c2ff50c917 dataprovider: add support for user status and expiration 
an user can now be disabled or expired.

If you are using an SQL database as dataprovider please remember to
execute the sql update script inside "sql" folder.

Fixes #57
 2019-11-13 11:36:21 +01:00
Nicola Murino
363b9ccc7f sftpd: explicitly configure supported SFTP extensions 
update pkg/sftp to a git revision that includes the needed patch

https://github.com/pkg/sftp/pull/315
 2019-11-12 07:37:47 +01:00
Nicola Murino
4ff34b3e53 logger: add specific logs for failed attempts to initialize a connection 
This should allow for better integration in tools like fail2ban.

Hopefully fix #59
 2019-11-11 15:20:00 +01:00
Nicola Murino
77db2bd3d1 fix posix rename 
update pkg/sftp to a git revision that includes:

https://github.com/pkg/sftp/pull/316

add a test case here too and update other deps
 2019-11-08 08:43:27 +01:00
Nicola Murino
8cd0aec417 add memory data provider and use it for portable mode 2019-10-25 18:37:12 +02:00
Nicola Murino
a4cddf4f7f add portable mode 
Portable mode is a convenient way to share a single directory on demand
 2019-10-24 18:50:35 +02:00
Nicola Murino
44d403cf9c sftpd_test: use path.Join for SFTP/SCP path 
filepath.Join could use an OS dependent separator
 2019-10-16 12:57:06 +02:00
Nicola Murino
8682ae4a54 sftpd: make file/dir removal and creation more standard 
- remove a non empty directory. Before: the directory contents were
removed recursively. Now: removing a non empty directory fails.

- make a directory in a non existent path: Before: any necessary parents
were created. Now: it fails.

- remove a file. Before: files, directories and symlinks were removed.
Now: only files and symlink are removed, removing a directory using "Remove"
instead of "Rmdir" fails.

Upload a file in a non existent directory. Before: any necessary parents
were created. Now: it fails.

Now SFTPGo behaves as OpenSSH.
 2019-10-16 07:48:22 +02:00
Nicola Murino
f98a29a1e0 sftpd: port to non deprecated pkg/sftp error types 
see https://github.com/pkg/sftp/pull/304
 2019-10-14 22:44:57 +02:00
Nicola Murino
62224debd2 simplify closing a connection 
get the connection from the map instead of cycling all the open connections
 2019-10-12 15:12:43 +02:00
Nicola Murino
871e2ccbbf sftpd: refactor connection closing 
we have not known bugs with the previous implementation anyway this one
is cleaner: the underlying network connection is directly related with
SFTP/SCP connections.
This should better protect us against buggy clients and edge cases
 2019-10-10 09:04:17 +02:00
Nicola Murino
4b5ce3913e ssh handshake: add a deadline for handshake to complete 
we use a 2 minutes timeout as OpenSSH
 2019-10-09 19:07:35 +02:00
Nicola Murino
1d917561fe sftpd: add support for upload resume 
we support resume only if the client sets the correct offset while resuming
the upload.
Based on the specs the offset is optional for resume, but all the tested
clients sets a right offset.
If an invalid offset is given we interrupt the transfer with the error
"Invalid write offset ..."

See https://github.com/pkg/sftp/issues/295

This commit add a new upload mode: "atomic with resume support", this acts
as atomic but if there is an upload error the temporary file is renamed
to the requested path and not deleted, this way a client can reconnect
and resume the upload
 2019-10-09 17:33:30 +02:00
Nicola Murino
afd312f26a add a basic web interface 
The builtin web interface allows to manage users and connections
 2019-10-07 18:19:01 +02:00
Nicola Murino
df96ea7e9f add a new permission for overwriting existing files 
The upload permission is required to allow file overwrite
 2019-09-17 08:53:45 +02:00
Nicola Murino
bba78763e1 add a test case and document sha512crypt passwords support 2019-09-15 12:23:27 +02:00
Nicola Murino
7eb5b01169 add Prometheus support 
some basic counters and gauges are now exposed
 2019-09-13 18:45:36 +02:00
Nicola Murino
fd59f35108 update pkg/sftp to the latest master 
Our pull request to handle transfer errors is now merged, so updating
pkg/sftp should fix #36
 2019-09-13 08:30:22 +02:00
Nicola Murino
3d13fe15c3 connections: close the ssh channel before the network connection 
This way if pkg/sftp is stuck in Serve() method should be unlocked.
 2019-09-11 16:29:56 +02:00
Nicola Murino
9794ca7ee0 connection: add a close method 
and use it instead of netConn directly
 2019-09-11 12:46:21 +02:00
Nicola Murino
784f75f45b use net.Conn instead of ssh.Conn to close connections 2019-09-11 09:41:46 +02:00
Nicola Murino
c1effdf701 atomic upload mode: remove temporary file on error 
If a transfer error is detected, then the partial temporary file will
be removed and not renamed to requested path
 2019-09-10 18:47:21 +02:00
Nicola Murino
e79f7010b4 docker alpine: use the latest stable docker tag for both run and build env 
so we don't need to update our Dockerfile each time a new alpine or golang
version is released
 2019-09-09 12:58:07 +02:00
Nicola Murino
f4507aeec2 improve transfer error log and TestSCPErrors 2019-09-09 08:57:11 +02:00
Nicola Murino
e09ea00d8b improve transfer error log 2019-09-07 23:27:37 +02:00
Nicola Murino
bc5779e26f actions: don't execute actions on errors 
detect upload/download errors and don't execute actions if a transfer
error happen.

To detect SFTP errors this patch is needed:

https://github.com/pkg/sftp/pull/307
 2019-09-07 23:10:20 +02:00
Nicola Murino
2a7e56ed29 docker: minor fixes 2019-09-07 18:21:03 +02:00
Nicola Murino
4f1c2c094f improve logging 
this partially revert #45
 2019-09-06 15:19:01 +02:00
Jo Vandeginste
abbb7f272b Refactor the logging system 
* created a "Log" function for type "Connection"
* created a "log" function for type "Provider"
* replace logger calls to Log/log where possible

I also renamed PGSSQL to PGSQL, as this seemed to be a typo

Signed-off-by: Jo Vandeginste <Jo.Vandeginste@kuleuven.be>
 2019-09-06 13:43:21 +02:00
Nicola Murino
a26343a694 improve more logs 2019-09-05 23:42:00 +02:00
Jo Vandeginste
25260297aa Show info about public key during login 
This will show the key fingerprint and the associated comment, or
"password" when password was used, during login.

Eg.:

```
message":"User id: 1, logged in with: \"public_key:SHA256:FV3+wlAKGzYy7+J02786fh8N8c06+jga/mdiSOSPT7g:jo@desktop\",
```

or

```
message":"User id: 1, logged in with: \"password\",
...`

Signed-off-by: Jo Vandeginste <Jo.Vandeginste@kuleuven.be>
 2019-09-05 22:58:14 +02:00
Jo Vandeginste
bb589c6fc8 Show user info, remove some extraneous connection ids 
Signed-off-by: Jo Vandeginste <Jo.Vandeginste@kuleuven.be>
 2019-09-05 22:58:14 +02:00
Jo Vandeginste
0737c672f5 Add connectionID to as many entries as possible 
Signed-off-by: Jo Vandeginste <Jo.Vandeginste@kuleuven.be>
 2019-09-05 22:58:14 +02:00
Jo Vandeginste
795ffc4c5f Add quotes to certain strings 
Signed-off-by: Jo Vandeginste <Jo.Vandeginste@kuleuven.be>
 2019-09-05 16:14:31 +02:00
Nicola Murino
49a40f7a0b sftpd: add configuration options for allowed ciphers, MACs and KEX algorithms 
add support for login banner too

Fixes #32
 2019-09-03 12:08:09 +02:00
Nicola Murino
dc5eeb54fd scp: fix quota update after file overwrite 
added a test case too
 2019-09-02 23:12:41 +02:00
Nicola Murino
25fb639465 actions: properly release resources once the external command exits 2019-08-30 03:58:54 +02:00
Nicola Murino
d4969cde03 scp: add test case and document remote to remote transfers 2019-08-25 13:51:54 +02:00
Nicola Murino
1c5aac0dc4 simplify scp upload code and add some test cases 2019-08-24 22:44:01 +02:00
Nicola Murino
e50c521c33 add SCP support 
SCP is an experimental feature, we have our own SCP implementation
since we can't rely on scp system command to proper handle permissions,
quota and user's home dir restrictions. The SCP protocol is quite simple
but there is no official docs about it, so we need more testing and
feedbacks before enabling it by default.
We may not handle some borderline cases or have sneaky bugs.

This commit contains some breaking changes to the REST API.
SFTPGo API should be stable now and I hope no more breaking changes
before the first stable release.
 2019-08-24 14:41:15 +02:00
Nicola Murino
8dc3501a1f add test cases for password stored using bcrypt and pbkdf2 2019-08-17 16:54:02 +02:00
Nicola Murino
cb87fe811a dataprovider move db handle to provider struct 
This is needed to support non SQL providers
 2019-08-11 14:53:37 +02:00
Nicola Murino
51aacae3c5 version: only git commit and build date must be modifiable 
Improved some test cases too
 2019-08-08 21:42:07 +02:00
Nicola Murino
90607d4f86 sftpd stats: add file path for active upload/download 2019-08-08 19:33:16 +02:00
Nicola Murino
4f4489d3f1 add version info 2019-08-08 10:01:33 +02:00
Nicola Murino
2aca4479a5 rename public_key in public_keys 
remove compatibility layer to convert public keys newline delimited
in json list
 2019-08-07 23:41:10 +02:00
Nicola Murino
8f421b7d0f switch to viper for configuration and use cobra for cli 2019-08-07 22:46:52 +02:00
Nicola Murino
4dc52ee3e9 api_utils: return response body too 
useful for debug and to build external API, for example a CLI interface
 2019-08-04 21:37:31 +02:00
Nicola Murino
4d3c1fbad6 add more test cases 2019-08-04 12:35:33 +02:00
Nicola Murino
61b5a1d9a3 simplify FileOpenFlags to os.Flags conversion 2019-08-04 11:02:38 +02:00
Nicola Murino
80b9c40489 add support for atomic upload 
Atomic uploads are now configurable. The default upload mode remains
non atomic
 2019-08-04 09:37:58 +02:00
Nicola Murino
b8dafecdd9 transfer: initialize lastActivity 
This avoid a random value for transfer of empty file where no reads or writes happen
 2019-08-02 15:49:51 +02:00
Nicola Murino
8c40684989 sftpd: simplify initialization code 
now gocyclo is happy again
 2019-08-02 11:17:23 +02:00
Nicola Murino
8d4964c16d convert public key from newline delimited string to a real array 
Added a compatibility layer that will convert newline delimited keys to array
when the user is fetched from the database.
This code will be removed in future versions please update your public keys,
you only need to resave the users using the REST API.
 2019-08-01 22:42:46 +02:00
Nicola Murino
788e068e13 Improve documentation 2019-08-01 15:39:30 +02:00
Jo Vandeginste
bd7b6a785e Support multiple private (host) keys 
With this patch, one can configure one or more private (host) keys in
the configuration file.

I made it a nested struct, so we can add more options later. Eg. host
certificates might be a useful addition if I can figure out how this is
done in golang's crypto/ssh...

Signed-off-by: Jo Vandeginste <Jo.Vandeginste@kuleuven.be>
 2019-08-01 14:57:15 +02:00
Nicola Murino
1566e43cd7 set track_quota to 2 as default 2019-08-01 09:24:35 +02:00
Nicola Murino
a7a3d533e7 Add documentation and tests for multiple public keys support 2019-07-31 17:06:12 +02:00
Nicola Murino
34543fd861 simplify test case for quota checking after file replace 2019-07-31 14:21:56 +02:00
Nicola Murino
89986b9305 add new test cases 2019-07-31 14:11:44 +02:00
Nicola Murino
fab21dcf51 Print initialization errors to standard output 
This way debugging configuration and initialization issues is simpler
 2019-07-31 08:14:31 +02:00
Nicola Murino
8058178ea0 Improve documentation 2019-07-30 20:51:29 +02:00
Philip Hofstetter
88a288ccfe make the server banner configurable 2019-07-29 08:07:59 +02:00
Nicola Murino
0bed768909 allow quota tracking only for users with quota restrictions 
Setting "track_quota" to 2 in sftpgo.conf will avoid a database update
after each file upload/deletion if the user has no quota restrictions
 2019-07-28 22:04:50 +02:00
Nicola Murino
88fedd577d sftpd handler: remove eccessive logging and prepare the code to support upload resume 
Upload resume is currently disabled for this upstream bug:

https://github.com/pkg/sftp/issues/295
 2019-07-28 13:40:12 +02:00
Nicola Murino
c547c5c9ee quota tracking: fix concurrent updates 
added a test case to check quota size for upload that replace an existing file
 2019-07-28 13:24:46 +02:00
Nicola Murino
e177366be9 windows: fix test cases 2019-07-27 21:19:30 +02:00
Nicola Murino
4a46b84dd5 Improve http action notification 
Send the notification using a goroutine and set the request timeout to 15 seconds
 2019-07-27 19:29:33 +02:00
Nicola Murino
25f7043fb3 sftpd transfer: close the file before executing any action ... 
... that could potentially move or delete it
 2019-07-27 11:17:32 +02:00
Nicola Murino
48451a9924 add support for custom actions 
Configurable custom commands and/or HTTP notifications on SFTP upload, download, delete or rename
 2019-07-27 09:38:09 +02:00
Nicola Murino
aaa4c22911 sftpd: test case for SetStat 
SetStat is silently ignored. Ownership and permissions are configured globally
or per account
 2019-07-26 17:18:51 +02:00
Nicola Murino
5d3a93baae sftpd: improve test cases 2019-07-26 16:15:42 +02:00
Nicola Murino
9f61415832 add more test cases 2019-07-26 15:08:08 +02:00
Nicola Murino
ff8fb80e3c fix check for used quota 
remove unused data provider from sftpd configuration struct
 2019-07-26 13:07:08 +02:00
Nicola Murino
7360b0f73c test cases: wait 100 milliseconds before recheck if the test server is up 
this avoid a lot of useless warnings in travis logs
 2019-07-26 11:41:58 +02:00
Nicola Murino
417b173c78 Improve test cases and logging 2019-07-26 11:35:12 +02:00
Vincent Castellano
69e941483a
Fix typo in ActiveQuotaScan json annotations 2019-07-25 21:55:26 -07:00
Nicola Murino
5c861a7c46 sftpd config: MaxAuthTries is now configurable 2019-07-21 12:02:24 +02:00
Nicola Murino
14f919df47 simplify some code 2019-07-21 00:19:17 +02:00
Nicola Murino
fea9ae1de2 fix another misspell 2019-07-20 21:20:16 +02:00
Nicola Murino
22b8dc7f1d fix misspells and ineffectual assignments 2019-07-20 21:17:53 +02:00
Nicola Murino
31cd4d7139 first version 2019-07-20 12:26:52 +02:00