Commit graph

23 commits

Author SHA1 Message Date
Nicola Murino
d481294519 S3: fix quota update after an upload error
S3 uploads are atomic, if the upload fails we have no partial file so we
have to update the user quota only if the upload succeed
2020-01-23 10:19:56 +01:00
Nicola Murino
4463421028 S3: add support for serving virtual folders
inside the same bucket each user can be assigned to a virtual folder.
This is similar to a chroot directory for local filesystem
2020-01-19 23:23:09 +01:00
Nicola Murino
d75f56b914 vfs: store root dir
so we don't need to pass it over and over
2020-01-19 13:58:55 +01:00
Nicola Murino
a4834f4a83 add basic S3-Compatible Object Storage support
we have now an interface for filesystem backeds, this make easy to add
new filesystem backends
2020-01-19 07:41:05 +01:00
Nicola Murino
1d9bb54073 transfers: improve errors detection
We can now properly report write errors if for example no space left on
device.

For downloads we check the downloaded size with the expected one
2020-01-10 19:20:22 +01:00
Nicola Murino
e046b35b97 check permissions against sftp path
instead of building filesystem paths and then checking permissions against
path relative to the home dir that is the initial sftp path
2020-01-05 11:41:25 +01:00
Nicola Murino
489101668c add per directory permissions
we can now have permissions such as these ones

{"/":["*"],"/somedir":["list","download"]}

The old permissions are automatically converted to the new structure,
no database migration is needed
2019-12-25 18:20:19 +01:00
Nicola Murino
9c4dbbc3f8 sftpd: add support for some SSH commands
md5sum, sha1sum are used by rclone.
cd, pwd improve the support for RemoteFiles mobile app.

These commands are all implemented inside SFTPGo so they work even
if the matching system commands are not available, for example on Windows
2019-11-18 23:30:37 +01:00
Nicola Murino
bb37a1c1ce sftpd: add support for chmod/chown
added matching permissions too and a new setting "setstat_mode".
Setting setstat_mode to 1 you can keep the previous behaviour that
silently ignore setstat requests
2019-11-15 12:15:07 +01:00
Nicola Murino
5be1d1be69 sftpd: send exit-status message on close
this fix restic compatibility
2019-11-14 16:49:42 +01:00
Nicola Murino
871e2ccbbf sftpd: refactor connection closing
we have not known bugs with the previous implementation anyway this one
is cleaner: the underlying network connection is directly related with
SFTP/SCP connections.
This should better protect us against buggy clients and edge cases
2019-10-10 09:04:17 +02:00
Nicola Murino
1d917561fe sftpd: add support for upload resume
we support resume only if the client sets the correct offset while resuming
the upload.
Based on the specs the offset is optional for resume, but all the tested
clients sets a right offset.
If an invalid offset is given we interrupt the transfer with the error
"Invalid write offset ..."

See https://github.com/pkg/sftp/issues/295

This commit add a new upload mode: "atomic with resume support", this acts
as atomic but if there is an upload error the temporary file is renamed
to the requested path and not deleted, this way a client can reconnect
and resume the upload
2019-10-09 17:33:30 +02:00
Nicola Murino
df96ea7e9f add a new permission for overwriting existing files
The upload permission is required to allow file overwrite
2019-09-17 08:53:45 +02:00
Nicola Murino
3d13fe15c3 connections: close the ssh channel before the network connection
This way if pkg/sftp is stuck in Serve() method should be unlocked.
2019-09-11 16:29:56 +02:00
Nicola Murino
bc5779e26f actions: don't execute actions on errors
detect upload/download errors and don't execute actions if a transfer
error happen.

To detect SFTP errors this patch is needed:

https://github.com/pkg/sftp/pull/307
2019-09-07 23:10:20 +02:00
Nicola Murino
4f1c2c094f improve logging
this partially revert #45
2019-09-06 15:19:01 +02:00
Jo Vandeginste
abbb7f272b Refactor the logging system
* created a "Log" function for type "Connection"
* created a "log" function for type "Provider"
* replace logger calls to Log/log where possible

I also renamed PGSSQL to PGSQL, as this seemed to be a typo

Signed-off-by: Jo Vandeginste <Jo.Vandeginste@kuleuven.be>
2019-09-06 13:43:21 +02:00
Nicola Murino
a26343a694 improve more logs 2019-09-05 23:42:00 +02:00
Jo Vandeginste
0737c672f5 Add connectionID to as many entries as possible
Signed-off-by: Jo Vandeginste <Jo.Vandeginste@kuleuven.be>
2019-09-05 22:58:14 +02:00
Nicola Murino
dc5eeb54fd scp: fix quota update after file overwrite
added a test case too
2019-09-02 23:12:41 +02:00
Nicola Murino
d4969cde03 scp: add test case and document remote to remote transfers 2019-08-25 13:51:54 +02:00
Nicola Murino
1c5aac0dc4 simplify scp upload code and add some test cases 2019-08-24 22:44:01 +02:00
Nicola Murino
e50c521c33 add SCP support
SCP is an experimental feature, we have our own SCP implementation
since we can't rely on scp system command to proper handle permissions,
quota and user's home dir restrictions. The SCP protocol is quite simple
but there is no official docs about it, so we need more testing and
feedbacks before enabling it by default.
We may not handle some borderline cases or have sneaky bugs.

This commit contains some breaking changes to the REST API.
SFTPGo API should be stable now and I hope no more breaking changes
before the first stable release.
2019-08-24 14:41:15 +02:00