Commit graph

299 commits

Author SHA1 Message Date
Nicola Murino
fa5333784b add a maximum allowed size for a single upload 2020-08-16 20:17:02 +02:00
Nicola Murino
0dbf0cc81f WebDAV: add CORS support 2020-08-15 15:55:20 +02:00
Nicola Murino
aa0ed5dbd0 add post-login hook
a login scope is supported too so you can get notifications for failed logins,
successful logins or both
2020-08-12 16:15:12 +02:00
Nicola Murino
a9e21c282a add WebDAV support
Fixes #147
2020-08-11 23:56:10 +02:00
Antoine Deschênes
9a15a54885
sftpd: set failed connection loglevel to debug (#152) 2020-08-06 21:20:31 +02:00
Nicola Murino
91dcc349de Add client IP address to external auth, pre-login and keyboard interactive hooks 2020-08-04 18:03:28 +02:00
Nicola Murino
fa41bfd06a Cloud backends: add support for FTP REST command
So partial downloads are now supported as for local fs
2020-08-03 18:03:09 +02:00
Nicola Murino
22338ed478 add post connect hook
Fixes #144
2020-07-30 22:33:49 +02:00
Nicola Murino
93ce96d011 add support for the venerable FTP protocol
Fixes #46
2020-07-29 21:56:56 +02:00
Nicola Murino
cc2f04b0e4 fix concurrency test case on go 1.13
a sleep seems required, needs investigation
2020-07-25 08:55:17 +02:00
Nicola Murino
4e41a5583d refactoring: add common package
The common package defines the interfaces that a protocol must implement
and contain code that can be shared among supported protocols.

This way should be easier to support new protocols
2020-07-24 23:39:38 +02:00
Nicola Murino
da0f470f1c document FreeBSD support
improve some tests cleanup
2020-07-10 19:20:37 +02:00
Nicola Murino
0ea2ca3141 simplify data provider usage
remove the obsolete SQL scripts too. They are not required since v0.9.6
2020-07-08 19:59:31 +02:00
Nicola Murino
ee1131f254 enable SCP test cases on Windows 2020-06-30 23:25:25 +02:00
Nicola Murino
dd593b1035 ssh commands: send a generic error for unexpected failures
and log the real error, it could leak a filesystem path
2020-06-29 18:53:33 +02:00
Nicola Murino
165110872b add release workflow
for each tag a new release, including binaries, is automatically created
2020-06-28 15:57:33 +02:00
Nicola Murino
6ab4e9f533 add test case for concurrent logins 2020-06-27 12:36:42 +02:00
Nicola Murino
cf541d62ea recursive permissions check before renaming/copying directories 2020-06-26 23:38:29 +02:00
Nicola Murino
e86089a9f3 quota: improve size check
get the remaining allowed size when an upload starts and check it against the
uploaded bytes

Fixes #128
2020-06-18 22:38:03 +02:00
Nicola Murino
3ceba7a147 sftpgo-copy: add quota limits check 2020-06-16 22:49:18 +02:00
Nicola Murino
37418a7630 SSH system commands: allow git and rsync inside virtual folders 2020-06-15 23:32:12 +02:00
Nicola Murino
73a9c002e0 permissions: improve rename
Allow to enable rename permission in a more controlled way granting "delete"
permission on source directory and "upload" permission on target directory
2020-06-13 23:49:28 +02:00
Nicola Murino
3d48fa7382 ssh commands: add sftpgo-copy and sftpgo-remove
Fixes #122
2020-06-13 22:48:51 +02:00
Nicola Murino
8e22dd1b13 virtual folders: allow overlapped mapped paths if quota is disabled
See #95
2020-06-10 09:11:32 +02:00
Nicola Murino
7807fa7cc2 use os.ModePerm for files and directory creation 2020-06-08 19:40:17 +02:00
Nicola Murino
cd380973df allows host keys auto generation inside a user configured directory
Fixes #124
2020-06-08 18:45:04 +02:00
Nicola Murino
01d681faa3 external auth: allow to map multiple login username to a single account
some external auth users want to map multiple login usernames with a single
SGTPGo account.
For example an SFTP user logins using "user1" or "user2" and the external auth
returns "user" in both cases, so we use the username returned from external auth
and not the one used to login

Fixes #125
2020-06-08 13:06:02 +02:00
Nicola Murino
c231b663a3 add docs for virtual folders
fix test cases on macOS
2020-06-08 00:15:14 +02:00
Nicola Murino
8306b6bde6 refactor virtual folders
The same virtual folder can now be shared among users and different
folder quota limits for each user are supported.

Fixes #120
2020-06-07 23:30:18 +02:00
Nicola Murino
dc011af90d sftpd actions: add support for pre-delete action
Fixes #121
2020-05-24 23:31:14 +02:00
Nicola Murino
c27e3ef436 actions: add a generic hook to define external commands and HTTP URL
We can only define a single hook now and it can be an HTTP notification
or an external command, not both
2020-05-24 15:29:39 +02:00
Nicola Murino
15298b0409 sftpd: remove unused expectedSize field from Transfer struct
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2020-05-20 20:17:59 +02:00
Nicola Murino
cfa710037c cloud backends: fix SFTP error message for some write failures
Fixes #119

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2020-05-19 19:17:43 +02:00
Nicola Murino
a08dd85efd sftpd: deprecate keys and add a new host_keys config param
host_key defines the private host keys as plain list of strings.

Remove the other deprecated config params from the default config too.

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2020-05-16 23:26:44 +02:00
Nicola Murino
469d36d979 certificate auth: fix source address checking inside crypto/ssh
So we can avoid to check source address ourself

81aafe6d26

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2020-05-16 15:15:32 +02:00
Nicola Murino
cf148db75d add test case for expired SSH certificate
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2020-05-15 23:23:49 +02:00
Nicola Murino
738c7ab43e sftpd: add support for SSH user certificate authentication
This add support for PROTOCOL.certkeys vendor extension:

https://cvsweb.openbsd.org/src/usr.bin/ssh/PROTOCOL.certkeys?rev=1.8

Fixes #117

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2020-05-15 20:08:53 +02:00
Nicola Murino
e0f2ab9c01 test cases: minor improvements
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2020-05-10 12:37:29 +02:00
Nicola Murino
e0183217b6 test cases: simplify TestLoginInvalidFs
we can simulate an invalid filesystem config using a GCS user without a
credentials file
2020-05-07 19:47:46 +02:00
Nicola Murino
0c6e2b566b fix test cases on Windows 2020-05-06 23:16:08 +02:00
Nicola Murino
f02e24437a add more linters
test cases migration to testify is now complete.
Linters are enabled for test cases too
2020-05-06 19:36:34 +02:00
Nicola Murino
b006c5f914 NewOsFs: return an interface and not a pointer 2020-05-02 15:01:56 +02:00
Nicola Murino
3f75d46a16 sftpd: add support for excluding virtual folders from user quota limit
Fixes #110
2020-05-01 15:27:53 +02:00
Nicola Murino
d70959c34c fix some lint issues 2020-04-30 14:23:55 +02:00
Nicola Murino
d377181b25 add a new configuration section for HTTP clients
HTTP clients are used for executing hooks such as the ones used for custom
actions, external authentication and pre-login user modifications.

This allows, for example, to use self-signed certificate without defeating the
purpose of using TLS
2020-04-26 23:29:09 +02:00
Nicola Murino
0a47412e8c scp, ssh commands: hide the real fs path on errors
The underlying filesystem errors for permissions and non-existing files
can contain the real storage path.
Map these errors to more generic ones to avoid to leak this info

Fixes #109
2020-04-22 12:26:18 +02:00
Nicola Murino
37357b2d63 add support for checking pbkdf2 passwords with base64 encoded salt
This way we can import the default passwords format used in 389ds.

See TestPasswordsHashPbkdf2Sha256_389DS test case to learn how to convert
389ds passwords
2020-04-11 12:25:21 +02:00
Nicola Murino
5a5912ea66 switch to my pkg/sftp branch and enable the request-server allocator
This way we have performance comparable to OpenSSH if the cipher
isn't the bottleneck
2020-04-10 23:35:57 +02:00
Nicola Murino
b1c7317cf6 add support for partial authentication
Multi-step authentication is activated disabling all single-step
auth methods for a given user
2020-04-09 23:32:42 +02:00
Nicola Murino
94b46e57f1 sftpd actions: execute defined command on error too
add a new field inside the notification to indicate if an error is
detected
2020-04-03 19:25:38 +02:00
Nicola Murino
9046acbe68 add HTTP hooks
external auth, pre-login user modification and keyboard interactive
authentication is now supported via HTTP requests too
2020-04-01 23:25:23 +02:00
Nicola Murino
075bbe2aef added test case that checks quota for files inside virtual folders 2020-03-29 11:10:03 +02:00
Nicola Murino
b52d078986 pbkdf2: fix password comparison
the key len for the derived function must be equal to the len of the
expected key
2020-03-28 16:09:06 +01:00
Nicola Murino
0a9c4914aa pre-login program: allow to create a new user too
clarify the difference between dynamic user creation/update and external
authentication
2020-03-27 23:26:22 +01:00
Nicola Murino
f284008fb5 enable scp in default configuration
remove the deprecated enable_scp setting
2020-03-26 23:38:24 +01:00
Nicola Murino
4759254e10 file actions: add bucket and endpoint to notifications
The HTTP notifications are now invoked as POST and the notification is
a JSON inside the POST body.

This is a backward incompatible change but this way the actions can be
extended more easily, sorry for the trouble

Fixes #101
2020-03-25 18:36:33 +01:00
Nicola Murino
81c8e8d898 add profiler support
profiling is now available via the HTTP base URL /debug/pprof/

examples, use this URL to start and download a 30 seconds CPU profile:

/debug/pprof/profile?seconds=30

use this URL to profile used memory:

/debug/pprof/heap?gc=1

use this URL to profile allocated memory:

/debug/pprof/allocs?gc=1

Full docs here:

https://golang.org/pkg/net/http/pprof/
2020-03-15 15:16:35 +01:00
Nicola Murino
9b119765fc docs: minor improvements 2020-03-04 23:51:16 +01:00
Nicola Murino
016abda6d7 improve docs 2020-03-03 23:25:23 +01:00
Nicola Murino
ed1c7cac17 update deps
we now use git master for pkg/sftp: it includes the performance patches
from my copy branch.
2020-03-02 10:13:49 +01:00
Nicola Murino
833b702b90 proxy protocol: add list of allowed IP addresses and IP ranges
"proxy_allowed" setting allows to specify the allowed IP address and IP
ranges that can send the proxy header. This setting combined with
"proxy_protocol" allows to ignore the header or to reject connections
that send the proxy header from a non listed IP
2020-03-01 23:12:28 +01:00
Nicola Murino
b885d453a2 filters: we can now set allowed and denied files extensions 2020-03-01 22:10:29 +01:00
Nicola Murino
7163fde724 proxy protocol: added an option to make the proxy header required
now we can configure SFTPGo to accept or reject requests without the proxy
header when the proxy protocol is enabled
2020-02-29 00:02:06 +01:00
Nicola Murino
830e3d1f64 Support for HAProxy PROXY protocol
you can proxy and/or load balance the SFTP/SCP service without losing
the information about the client's address.
2020-02-27 09:21:30 +01:00
Mengsk
637463a068 Rename before_login_program to pre_login_program
and some documentation update
2020-02-25 16:34:54 +01:00
Nicola Murino
e69536f540 fixed some typos and improved a log 2020-02-25 12:46:52 +01:00
Nicola Murino
eb1b869b73 virtual folders fixes
scp now properly handles virtual folders.

rsync is disabled for users with virtual folders: we execute a system
command and it is not aware about virtual folders.

git is not allowed if the repo path is inside a virtual folder
2020-02-24 18:54:35 +01:00
Nicola Murino
703ccc8d91 add support for dynamic users modifications
A custom program can be executed before the users login to modify the
configurations for the user trying to login.
You can, for example, allow login based on time range.

Fixes #77
2020-02-23 18:50:59 +01:00
Nicola Murino
45b9366dd0 add support for virtual folders
directories outside the user home directory can be exposed as virtual folders
2020-02-23 11:30:26 +01:00
Nicola Murino
bc11cdd8d5 add support for per user authentication methods
You can, for example, deny one or more authentication methods to one or
more users.
2020-02-19 22:39:30 +01:00
Nicola Murino
62b20cd884 scp: check for write errors
exits as soon as there is a write error instead of get the same error when
the transfer is closed
2020-02-19 11:26:40 +01:00
Nicola Murino
c8cc81cf4a sftpd: autogenerate ecdsa key
With default configuration we now generate RSA and ECDSA server keys.
2020-02-16 18:17:39 +01:00
Nicola Murino
0b7be1175d parse ssh commands with shlex
instead of use our bugged home made method.

Fixes #72
2020-02-14 16:17:32 +01:00
Nicola Murino
b99495ebbb sftpd download: remove check for download size
some clients, for example rclone can request only part of a file, we have
no way to detect this so we haven't return an error if the downloaded size
does not match the file size
2020-02-14 11:09:16 +01:00
Nicola Murino
5b47292366 sftpd: improve error logs
Now logs for cloud filesystems are more readable.

Also use standard output as default for debian Dockerfile
2020-02-13 08:26:45 +01:00
Nicola Murino
8eff2df39c subdir perms: allow empty perms
empty perms will allow nothing on the specified subdir.

Non empty permissions for the "/" dir are still required.

Fixes #70
2020-02-10 19:28:35 +01:00
Nicola Murino
7bfe0ddf80 ssh commands: fix parsing commands with space
For now we support "\" escaping style
2020-02-08 23:33:06 +01:00
Nicola Murino
8b039e0447 httpd: add support for basic auth and HTTPS 2020-02-04 00:08:00 +01:00
Nicola Murino
c64c080159 fix test cases on Windows
We have to rework TestRelativePaths and TestResolvePaths if we want to run
them for Cloud Storage on Windows too: we use filesystem path while Cloud
Storage providers expect Unix paths.
On Windows is important to check the local filesystem so skip Cloud Storage
providers test cases for now
2020-02-02 22:40:10 +01:00
Nicola Murino
bcaf283c35 memory provider: load users from a dump file
The `memory` provider can load users from a dump obtained using the
`dumpdata` REST API. This dump file can be configured using the
dataprovider `name` configuration key. It will be loaded at startup
and can be reloaded on demand using a `SIGHUP` on Unix based systems
and a `paramchange` request to the running service on Windows.

Fixes #66
2020-02-02 22:20:39 +01:00
Nicola Murino
31a433cda2 update deps and simplify some code 2020-01-31 23:26:56 +01:00
Nicola Murino
3491717c26 add support for serving Google Cloud Storage over SFTP/SCP
Each user can be mapped with a Google Cloud Storage bucket or a bucket
virtual folder
2020-01-31 19:04:00 +01:00
Nicola Murino
6884ce3f3e sftp actions: add a parameter to distinguish local and remote files 2020-01-23 23:23:28 +01:00
Nicola Murino
5f4efc9148 S3: add metrics 2020-01-23 23:17:00 +01:00
Nicola Murino
d481294519 S3: fix quota update after an upload error
S3 uploads are atomic, if the upload fails we have no partial file so we
have to update the user quota only if the upload succeed
2020-01-23 10:19:56 +01:00
Nicola Murino
9ff303b8c0 add support for keyboard interactive authentication
Fixes #64
2020-01-21 10:54:05 +01:00
Nicola Murino
4463421028 S3: add support for serving virtual folders
inside the same bucket each user can be assigned to a virtual folder.
This is similar to a chroot directory for local filesystem
2020-01-19 23:23:09 +01:00
Nicola Murino
d75f56b914 vfs: store root dir
so we don't need to pass it over and over
2020-01-19 13:58:55 +01:00
Nicola Murino
a4834f4a83 add basic S3-Compatible Object Storage support
we have now an interface for filesystem backeds, this make easy to add
new filesystem backends
2020-01-19 07:41:05 +01:00
Nicola Murino
27dbcf0066 fix test cases on Windows 2020-01-12 08:25:08 +01:00
Nicola Murino
ec194d73d2 update pkg/sftp to a stable version ...
... now that it contains all the needed patches.
Remove an hack for setstat with empty attrs, it is now handled in pkg/sftp.
Update other dependencies too.
2020-01-11 14:29:30 +01:00
Nicola Murino
1d9bb54073 transfers: improve errors detection
We can now properly report write errors if for example no space left on
device.

For downloads we check the downloaded size with the expected one
2020-01-10 19:20:22 +01:00
Nicola Murino
eec60d6309 custom actions: add env vars
action parameters can now be readed from env vars too.
Added a timeout for the command execution
2020-01-09 12:00:37 +01:00
Nicola Murino
8e604f888a improve docs and test cases 2020-01-07 09:39:20 +01:00
Nicola Murino
531091906d add support for authentication using external programs
Fixes #62
2020-01-06 21:42:41 +01:00
Nicola Murino
e046b35b97 check permissions against sftp path
instead of building filesystem paths and then checking permissions against
path relative to the home dir that is the initial sftp path
2020-01-05 11:41:25 +01:00
Nicola Murino
1b1c740b29 Add support for allowed/denied IP/Mask
Login can be restricted to specific ranges of IP address or to a specific IP
address.

Please apply the appropriate SQL upgrade script to add the filter field to your
database.

The filter database field will allow to add other filters without requiring a
new database migration
2019-12-30 18:37:50 +01:00
Nicola Murino
ad5436e3f6 ssh commands: improve command ended detection
Sometime we can have this error:

read |0: file already closed

reading from the command standard error, this means that the command is
already finished so we don't need to do nothing.
This happen randomically while running the test cases on travis.
2019-12-30 00:10:03 +01:00
Nicola Murino
20606a0043 sftpd test: add a debug log
The git push test sometime fails when running on travis.
The issue cannot be replicated locally so print the logs to try to
understand what is happening
2019-12-29 23:27:32 +01:00
Nicola Murino
6aff8c2f5e add support for checking passwords in md5crypt ($1$) format
this is an old and unsafe schema but it is still useful to import users
from legacy systems
2019-12-29 07:43:59 +01:00
Nicola Murino
ae094d3479 add backup/restore REST API 2019-12-27 23:12:44 +01:00
Nicola Murino
f49c280a7f fix test cases on Windows
SysProcAttr.Credential is not available on Windows we need to move the
WrapCmd test in a separate file to be able to build test cases on Windows,
skipping the test is not enough
2019-12-26 08:29:38 +01:00
Nicola Murino
ae812e55af sftpd: explicitly disallow some commands on root directory
It was possible to remove an empty root dir or create a symlink to it.
We now return a Permission Denied error if we detect an attempt to remove,
renaming or symlinking the root directory
2019-12-25 23:37:37 +01:00
Nicola Murino
489101668c add per directory permissions
we can now have permissions such as these ones

{"/":["*"],"/somedir":["list","download"]}

The old permissions are automatically converted to the new structure,
no database migration is needed
2019-12-25 18:20:19 +01:00
Nicola Murino
39fc9b73e9 sftp setstat: guard against empty attrs
It seems that there are some clients that sends Setstat requests with
no attrs:

https://github.com/pkg/sftp/issues/325

I haven't never seen this myself, anyway we now return ErrSSHFxBadMessage
and log the client version in such cases
2019-12-04 08:31:47 +01:00
Nicola Murino
80a5138115 add rsync support ...
and better document quota management issues for system commands.

rsync and git are not enabled in the default config so don't install
them in sample Dockerfiles, simply add a comment to facilitate their
installation if needed

Fixes #44
2019-11-29 15:24:56 +01:00
Nicola Murino
bc844105b2 ssh system commands: wait on write from command out to network
we only need to wait for the write from the local command to
the ssh channel. There is no need to wait for the write from ssh
channel to the local command stdin
2019-11-27 22:33:17 +01:00
Nicola Murino
0a025aabfd add support for Git over SSH
We use the system commands "git-receive-pack", "git-upload-pack" and
"git-upload-archive". they need to be installed and in your system's
PATH. Since we execute system commands we have no direct control on
file creation/deletion and so quota check is suboptimal: if quota is
enabled, the number of files is checked at the command begin and not
while new files are created.
The allowed size is calculated as the difference between the max quota
and the used one. The command is aborted if it uploads more bytes than
the remaining allowed size calculated at the command start. Quotas are
recalculated at the command end with a full home directory scan, this
could be heavy for big directories.
2019-11-26 22:26:42 +01:00
Nicola Murino
b3729e4666 log ssh commands in "command logs" category 2019-11-19 11:38:39 +01:00
Nicola Murino
9c4dbbc3f8 sftpd: add support for some SSH commands
md5sum, sha1sum are used by rclone.
cd, pwd improve the support for RemoteFiles mobile app.

These commands are all implemented inside SFTPGo so they work even
if the matching system commands are not available, for example on Windows
2019-11-18 23:30:37 +01:00
Nicola Murino
ca6cb34d98 sftpd: add support for chtimes
This improve rclone compatibility
2019-11-16 10:23:41 +01:00
Nicola Murino
fc442d7862 sftpd: document chmod/chown on Windows
chmod is partially supported and chown is not supported on Windows.

Skip unsupported test cases on Windows
2019-11-15 17:09:00 +01:00
Nicola Murino
3ac5af47f2 minor fixes 2019-11-15 12:26:52 +01:00
Nicola Murino
bb37a1c1ce sftpd: add support for chmod/chown
added matching permissions too and a new setting "setstat_mode".
Setting setstat_mode to 1 you can keep the previous behaviour that
silently ignore setstat requests
2019-11-15 12:15:07 +01:00
Nicola Murino
5be1d1be69 sftpd: send exit-status message on close
this fix restic compatibility
2019-11-14 16:49:42 +01:00
Nicola Murino
08e85f6be9 sftpd: return sftp.ErrSSHFxNoSuchFile if the client ask the file for a missing path
some clients expected this error and not the generic one if the path is missing
2019-11-14 14:18:43 +01:00
Nicola Murino
acdf351047 dataprovider: add custom command and/or HTTP notifications on users add, update and delete
This way custom logic can be implemented for example to create a UNIX user
as asked in #58
2019-11-14 11:06:03 +01:00
Nicola Murino
c2ff50c917 dataprovider: add support for user status and expiration
an user can now be disabled or expired.

If you are using an SQL database as dataprovider please remember to
execute the sql update script inside "sql" folder.

Fixes #57
2019-11-13 11:36:21 +01:00
Nicola Murino
363b9ccc7f sftpd: explicitly configure supported SFTP extensions
update pkg/sftp to a git revision that includes the needed patch

https://github.com/pkg/sftp/pull/315
2019-11-12 07:37:47 +01:00
Nicola Murino
4ff34b3e53 logger: add specific logs for failed attempts to initialize a connection
This should allow for better integration in tools like fail2ban.

Hopefully fix #59
2019-11-11 15:20:00 +01:00
Nicola Murino
77db2bd3d1 fix posix rename
update pkg/sftp to a git revision that includes:

https://github.com/pkg/sftp/pull/316

add a test case here too and update other deps
2019-11-08 08:43:27 +01:00
Nicola Murino
8cd0aec417 add memory data provider and use it for portable mode 2019-10-25 18:37:12 +02:00
Nicola Murino
a4cddf4f7f add portable mode
Portable mode is a convenient way to share a single directory on demand
2019-10-24 18:50:35 +02:00
Nicola Murino
44d403cf9c sftpd_test: use path.Join for SFTP/SCP path
filepath.Join could use an OS dependent separator
2019-10-16 12:57:06 +02:00
Nicola Murino
8682ae4a54 sftpd: make file/dir removal and creation more standard
- remove a non empty directory. Before: the directory contents were
removed recursively. Now: removing a non empty directory fails.

- make a directory in a non existent path: Before: any necessary parents
were created. Now: it fails.

- remove a file. Before: files, directories and symlinks were removed.
Now: only files and symlink are removed, removing a directory using "Remove"
instead of "Rmdir" fails.

Upload a file in a non existent directory. Before: any necessary parents
were created. Now: it fails.

Now SFTPGo behaves as OpenSSH.
2019-10-16 07:48:22 +02:00
Nicola Murino
f98a29a1e0 sftpd: port to non deprecated pkg/sftp error types
see https://github.com/pkg/sftp/pull/304
2019-10-14 22:44:57 +02:00
Nicola Murino
62224debd2 simplify closing a connection
get the connection from the map instead of cycling all the open connections
2019-10-12 15:12:43 +02:00
Nicola Murino
871e2ccbbf sftpd: refactor connection closing
we have not known bugs with the previous implementation anyway this one
is cleaner: the underlying network connection is directly related with
SFTP/SCP connections.
This should better protect us against buggy clients and edge cases
2019-10-10 09:04:17 +02:00
Nicola Murino
4b5ce3913e ssh handshake: add a deadline for handshake to complete
we use a 2 minutes timeout as OpenSSH
2019-10-09 19:07:35 +02:00
Nicola Murino
1d917561fe sftpd: add support for upload resume
we support resume only if the client sets the correct offset while resuming
the upload.
Based on the specs the offset is optional for resume, but all the tested
clients sets a right offset.
If an invalid offset is given we interrupt the transfer with the error
"Invalid write offset ..."

See https://github.com/pkg/sftp/issues/295

This commit add a new upload mode: "atomic with resume support", this acts
as atomic but if there is an upload error the temporary file is renamed
to the requested path and not deleted, this way a client can reconnect
and resume the upload
2019-10-09 17:33:30 +02:00
Nicola Murino
afd312f26a add a basic web interface
The builtin web interface allows to manage users and connections
2019-10-07 18:19:01 +02:00
Nicola Murino
df96ea7e9f add a new permission for overwriting existing files
The upload permission is required to allow file overwrite
2019-09-17 08:53:45 +02:00
Nicola Murino
bba78763e1 add a test case and document sha512crypt passwords support 2019-09-15 12:23:27 +02:00
Nicola Murino
7eb5b01169 add Prometheus support
some basic counters and gauges are now exposed
2019-09-13 18:45:36 +02:00
Nicola Murino
fd59f35108 update pkg/sftp to the latest master
Our pull request to handle transfer errors is now merged, so updating
pkg/sftp should fix #36
2019-09-13 08:30:22 +02:00
Nicola Murino
3d13fe15c3 connections: close the ssh channel before the network connection
This way if pkg/sftp is stuck in Serve() method should be unlocked.
2019-09-11 16:29:56 +02:00
Nicola Murino
9794ca7ee0 connection: add a close method
and use it instead of netConn directly
2019-09-11 12:46:21 +02:00
Nicola Murino
784f75f45b use net.Conn instead of ssh.Conn to close connections 2019-09-11 09:41:46 +02:00
Nicola Murino
c1effdf701 atomic upload mode: remove temporary file on error
If a transfer error is detected, then the partial temporary file will
be removed and not renamed to requested path
2019-09-10 18:47:21 +02:00
Nicola Murino
e79f7010b4 docker alpine: use the latest stable docker tag for both run and build env
so we don't need to update our Dockerfile each time a new alpine or golang
version is released
2019-09-09 12:58:07 +02:00
Nicola Murino
f4507aeec2 improve transfer error log and TestSCPErrors 2019-09-09 08:57:11 +02:00
Nicola Murino
e09ea00d8b improve transfer error log 2019-09-07 23:27:37 +02:00
Nicola Murino
bc5779e26f actions: don't execute actions on errors
detect upload/download errors and don't execute actions if a transfer
error happen.

To detect SFTP errors this patch is needed:

https://github.com/pkg/sftp/pull/307
2019-09-07 23:10:20 +02:00
Nicola Murino
2a7e56ed29 docker: minor fixes 2019-09-07 18:21:03 +02:00
Nicola Murino
4f1c2c094f improve logging
this partially revert #45
2019-09-06 15:19:01 +02:00
Jo Vandeginste
abbb7f272b Refactor the logging system
* created a "Log" function for type "Connection"
* created a "log" function for type "Provider"
* replace logger calls to Log/log where possible

I also renamed PGSSQL to PGSQL, as this seemed to be a typo

Signed-off-by: Jo Vandeginste <Jo.Vandeginste@kuleuven.be>
2019-09-06 13:43:21 +02:00
Nicola Murino
a26343a694 improve more logs 2019-09-05 23:42:00 +02:00
Jo Vandeginste
25260297aa Show info about public key during login
This will show the key fingerprint and the associated comment, or
"password" when password was used, during login.

Eg.:

```
message":"User id: 1, logged in with: \"public_key:SHA256:FV3+wlAKGzYy7+J02786fh8N8c06+jga/mdiSOSPT7g:jo@desktop\",
```

or

```
message":"User id: 1, logged in with: \"password\",
...`

Signed-off-by: Jo Vandeginste <Jo.Vandeginste@kuleuven.be>
2019-09-05 22:58:14 +02:00
Jo Vandeginste
bb589c6fc8 Show user info, remove some extraneous connection ids
Signed-off-by: Jo Vandeginste <Jo.Vandeginste@kuleuven.be>
2019-09-05 22:58:14 +02:00
Jo Vandeginste
0737c672f5 Add connectionID to as many entries as possible
Signed-off-by: Jo Vandeginste <Jo.Vandeginste@kuleuven.be>
2019-09-05 22:58:14 +02:00