HTTP clients are used for executing hooks such as the ones used for custom
actions, external authentication and pre-login user modifications.
This allows, for example, to use self-signed certificate without defeating the
purpose of using TLS
The underlying filesystem errors for permissions and non-existing files
can contain the real storage path.
Map these errors to more generic ones to avoid to leak this info
Fixes#109
This way we can import the default passwords format used in 389ds.
See TestPasswordsHashPbkdf2Sha256_389DS test case to learn how to convert
389ds passwords
The HTTP notifications are now invoked as POST and the notification is
a JSON inside the POST body.
This is a backward incompatible change but this way the actions can be
extended more easily, sorry for the trouble
Fixes#101
profiling is now available via the HTTP base URL /debug/pprof/
examples, use this URL to start and download a 30 seconds CPU profile:
/debug/pprof/profile?seconds=30
use this URL to profile used memory:
/debug/pprof/heap?gc=1
use this URL to profile allocated memory:
/debug/pprof/allocs?gc=1
Full docs here:
https://golang.org/pkg/net/http/pprof/
"proxy_allowed" setting allows to specify the allowed IP address and IP
ranges that can send the proxy header. This setting combined with
"proxy_protocol" allows to ignore the header or to reject connections
that send the proxy header from a non listed IP
scp now properly handles virtual folders.
rsync is disabled for users with virtual folders: we execute a system
command and it is not aware about virtual folders.
git is not allowed if the repo path is inside a virtual folder
A custom program can be executed before the users login to modify the
configurations for the user trying to login.
You can, for example, allow login based on time range.
Fixes#77
some clients, for example rclone can request only part of a file, we have
no way to detect this so we haven't return an error if the downloaded size
does not match the file size
We have to rework TestRelativePaths and TestResolvePaths if we want to run
them for Cloud Storage on Windows too: we use filesystem path while Cloud
Storage providers expect Unix paths.
On Windows is important to check the local filesystem so skip Cloud Storage
providers test cases for now
The `memory` provider can load users from a dump obtained using the
`dumpdata` REST API. This dump file can be configured using the
dataprovider `name` configuration key. It will be loaded at startup
and can be reloaded on demand using a `SIGHUP` on Unix based systems
and a `paramchange` request to the running service on Windows.
Fixes#66
... now that it contains all the needed patches.
Remove an hack for setstat with empty attrs, it is now handled in pkg/sftp.
Update other dependencies too.
Login can be restricted to specific ranges of IP address or to a specific IP
address.
Please apply the appropriate SQL upgrade script to add the filter field to your
database.
The filter database field will allow to add other filters without requiring a
new database migration
Sometime we can have this error:
read |0: file already closed
reading from the command standard error, this means that the command is
already finished so we don't need to do nothing.
This happen randomically while running the test cases on travis.
The git push test sometime fails when running on travis.
The issue cannot be replicated locally so print the logs to try to
understand what is happening
SysProcAttr.Credential is not available on Windows we need to move the
WrapCmd test in a separate file to be able to build test cases on Windows,
skipping the test is not enough
It was possible to remove an empty root dir or create a symlink to it.
We now return a Permission Denied error if we detect an attempt to remove,
renaming or symlinking the root directory
we can now have permissions such as these ones
{"/":["*"],"/somedir":["list","download"]}
The old permissions are automatically converted to the new structure,
no database migration is needed
It seems that there are some clients that sends Setstat requests with
no attrs:
https://github.com/pkg/sftp/issues/325
I haven't never seen this myself, anyway we now return ErrSSHFxBadMessage
and log the client version in such cases
and better document quota management issues for system commands.
rsync and git are not enabled in the default config so don't install
them in sample Dockerfiles, simply add a comment to facilitate their
installation if needed
Fixes#44
we only need to wait for the write from the local command to
the ssh channel. There is no need to wait for the write from ssh
channel to the local command stdin
We use the system commands "git-receive-pack", "git-upload-pack" and
"git-upload-archive". they need to be installed and in your system's
PATH. Since we execute system commands we have no direct control on
file creation/deletion and so quota check is suboptimal: if quota is
enabled, the number of files is checked at the command begin and not
while new files are created.
The allowed size is calculated as the difference between the max quota
and the used one. The command is aborted if it uploads more bytes than
the remaining allowed size calculated at the command start. Quotas are
recalculated at the command end with a full home directory scan, this
could be heavy for big directories.
md5sum, sha1sum are used by rclone.
cd, pwd improve the support for RemoteFiles mobile app.
These commands are all implemented inside SFTPGo so they work even
if the matching system commands are not available, for example on Windows
added matching permissions too and a new setting "setstat_mode".
Setting setstat_mode to 1 you can keep the previous behaviour that
silently ignore setstat requests
an user can now be disabled or expired.
If you are using an SQL database as dataprovider please remember to
execute the sql update script inside "sql" folder.
Fixes#57
- remove a non empty directory. Before: the directory contents were
removed recursively. Now: removing a non empty directory fails.
- make a directory in a non existent path: Before: any necessary parents
were created. Now: it fails.
- remove a file. Before: files, directories and symlinks were removed.
Now: only files and symlink are removed, removing a directory using "Remove"
instead of "Rmdir" fails.
Upload a file in a non existent directory. Before: any necessary parents
were created. Now: it fails.
Now SFTPGo behaves as OpenSSH.
we have not known bugs with the previous implementation anyway this one
is cleaner: the underlying network connection is directly related with
SFTP/SCP connections.
This should better protect us against buggy clients and edge cases
we support resume only if the client sets the correct offset while resuming
the upload.
Based on the specs the offset is optional for resume, but all the tested
clients sets a right offset.
If an invalid offset is given we interrupt the transfer with the error
"Invalid write offset ..."
See https://github.com/pkg/sftp/issues/295
This commit add a new upload mode: "atomic with resume support", this acts
as atomic but if there is an upload error the temporary file is renamed
to the requested path and not deleted, this way a client can reconnect
and resume the upload
detect upload/download errors and don't execute actions if a transfer
error happen.
To detect SFTP errors this patch is needed:
https://github.com/pkg/sftp/pull/307
* created a "Log" function for type "Connection"
* created a "log" function for type "Provider"
* replace logger calls to Log/log where possible
I also renamed PGSSQL to PGSQL, as this seemed to be a typo
Signed-off-by: Jo Vandeginste <Jo.Vandeginste@kuleuven.be>
This will show the key fingerprint and the associated comment, or
"password" when password was used, during login.
Eg.:
```
message":"User id: 1, logged in with: \"public_key:SHA256:FV3+wlAKGzYy7+J02786fh8N8c06+jga/mdiSOSPT7g:jo@desktop\",
```
or
```
message":"User id: 1, logged in with: \"password\",
...`
Signed-off-by: Jo Vandeginste <Jo.Vandeginste@kuleuven.be>
SCP is an experimental feature, we have our own SCP implementation
since we can't rely on scp system command to proper handle permissions,
quota and user's home dir restrictions. The SCP protocol is quite simple
but there is no official docs about it, so we need more testing and
feedbacks before enabling it by default.
We may not handle some borderline cases or have sneaky bugs.
This commit contains some breaking changes to the REST API.
SFTPGo API should be stable now and I hope no more breaking changes
before the first stable release.
Added a compatibility layer that will convert newline delimited keys to array
when the user is fetched from the database.
This code will be removed in future versions please update your public keys,
you only need to resave the users using the REST API.
With this patch, one can configure one or more private (host) keys in
the configuration file.
I made it a nested struct, so we can add more options later. Eg. host
certificates might be a useful addition if I can figure out how this is
done in golang's crypto/ssh...
Signed-off-by: Jo Vandeginste <Jo.Vandeginste@kuleuven.be>
