Commit graph

135 commits

Author SHA1 Message Date
Nicola Murino
4edb9cd6b9
simplify some code 2020-11-07 18:05:47 +01:00
Nicola Murino
1d5d184720
webdav file: ensure to close the reader only once 2020-11-05 09:30:38 +01:00
Nicola Murino
0a14297b48
webdav: performance improvements and bug fixes
we need my custom golang/x/net/webdav fork for now

https://github.com/drakkan/net/tree/sftpgo
2020-11-04 19:11:40 +01:00
Nicola Murino
57bec976ae
document heathz endpoint 2020-11-01 10:39:10 +01:00
Nicola Murino
fcfdd633f6
Azure Blob: update SDK and add access tier support 2020-10-30 22:17:17 +01:00
Nicola Murino
ac3bae00fc
add support for SFTP subsystem mode
Fixes #204
2020-10-29 19:23:33 +01:00
Nicola Murino
5ff8f75917
add Azure Blob support 2020-10-25 08:18:48 +01:00
Nicola Murino
d12819932a
update cobra to v1.1.1
this version fix the man page generation so we don't need to use
our branch anymore
2020-10-18 21:52:42 +02:00
Nicola Murino
756c3d0503
fix man page generation
other minor changes
2020-10-17 22:14:04 +02:00
Nicola Murino
ce9387f1ab
update dependencies and some docs 2020-10-09 20:25:42 +02:00
Nicola Murino
2d6c0388af
update deps 2020-10-04 18:29:42 +02:00
Nicola Murino
4d19d87720
pkgs: use glob notation to include static folder 2020-10-02 18:16:49 +02:00
Nicola Murino
55515fee95
update deps, GCS can no finally use attribute selection
See https://github.com/googleapis/google-cloud-go/pull/2661
2020-09-28 12:51:19 +02:00
Nicola Murino
38f06ab373 ftpd: fix TLS for active connections
See https://github.com/fclairamb/ftpserverlib/issues/177

Some minor doc improvements
2020-09-17 09:45:40 +02:00
Nicola Murino
61003c8079 sftpd: add lstat support 2020-09-11 09:30:25 +02:00
Nicola Murino
b9c381e26f sftpd: update pkg/sftp
The patch to open a file in read/write mode is now merged
2020-09-06 11:40:31 +02:00
Nicola Murino
542554fb2c replace the library to verify UNIX's crypt(3) passwords 2020-09-04 21:08:09 +02:00
Nicola Murino
afc411c51b adjust runtime.GOMAXPROCS to match the container CPU quota, if any 2020-09-03 18:09:45 +02:00
Nicola Murino
dbed110d02 WebDAV: add caching for authenticated users
In this way we get a big performance boost
2020-08-31 19:25:17 +02:00
Nicola Murino
4748e6f54d
sftpd: handle read and write from the same handle (#158)
Fixes #155
2020-08-31 06:45:22 +02:00
Nicola Murino
2746c0b0f1 move stat to base connection and differentiate between Stat and Lstat
we will use Lstat once it will be exposed in pkg/sftp
2020-08-25 18:23:00 +02:00
Nicola Murino
701a6115f8 ftpd: use ftpserverlib master, the tls patch is now merged 2020-08-24 23:06:10 +02:00
Nicola Murino
56b00addc4 docker: try to improve the docs
See #159
2020-08-24 15:46:31 +02:00
Nicola Murino
5208e4a4ca sftpd: improve truncate
quota usage and max allowed write size are now properly updated after a
truncate
2020-08-22 10:12:00 +02:00
Nicola Murino
f41ce6619f sftpd: add SSH_FXP_FSETSTAT support
This change will fix file editing from sshfs, we need this patch

https://github.com/pkg/sftp/pull/373

for pkg/sftp to support this feature
2020-08-20 13:54:36 +02:00
Nicola Murino
8b0a1817b3 add check password hook
its main use case is to allow to easily support things like password+OTP for
protocols without keyboard interactive support such as FTP and WebDAV
2020-08-19 19:36:12 +02:00
Nicola Murino
0dbf0cc81f WebDAV: add CORS support 2020-08-15 15:55:20 +02:00
Nicola Murino
196a56726e FTP improvements
- add a setting to require TLS
- add symlink support

require TLS 1.2 for all TLS connections
2020-08-15 13:02:25 +02:00
Nicola Murino
a9e21c282a add WebDAV support
Fixes #147
2020-08-11 23:56:10 +02:00
Nicola Murino
fa41bfd06a Cloud backends: add support for FTP REST command
So partial downloads are now supported as for local fs
2020-08-03 18:03:09 +02:00
Nicola Murino
8839c34d53 FTP: implements ClientDriverExtensionRemoveDir
Fixes #149 for FTP too
2020-08-03 17:36:43 +02:00
Nicola Murino
2a9f7db1e2 Cloud FS: don't propagate the error if removing a folder returns not found
for Cloud FS the folders are virtual and they, generally, disappear when the
last file is removed.

This fix doesn't work for FTP protocol for now.

Fixes #149
2020-07-31 19:24:57 +02:00
Nicola Murino
59a21158a6 fix FTP quota limits test case
It failed sometime due to a bug in the ftp client library used in test
cases. The failure was more frequent on FreeBSD but it could happen in
any supported OS. It was not systematic since we use small files in
test cases.

See https://github.com/jlaffaye/ftp/pull/192
2020-07-30 19:52:29 +02:00
Nicola Murino
93ce96d011 add support for the venerable FTP protocol
Fixes #46
2020-07-29 21:56:56 +02:00
Nicola Murino
4e41a5583d refactoring: add common package
The common package defines the interfaces that a protocol must implement
and contain code that can be shared among supported protocols.

This way should be easier to support new protocols
2020-07-24 23:39:38 +02:00
Nicola Murino
96cbce52f9 cmd: add shell completion and man pages generators 2020-07-08 23:21:33 +02:00
Nicola Murino
fc32286045 update deps 2020-07-05 22:54:00 +02:00
Nicola Murino
403b9a8310 replace Travis with GitHub actions 2020-06-20 21:57:51 +02:00
Nicola Murino
33436488e2 update deps 2020-06-20 16:09:55 +02:00
Nicola Murino
3d48fa7382 ssh commands: add sftpgo-copy and sftpgo-remove
Fixes #122
2020-06-13 22:48:51 +02:00
Nicola Murino
760cc9ba5a partial auth: fix public key query response
more details here:

https://github.com/golang/crypto/pull/130#issuecomment-633191423
2020-05-24 12:13:14 +02:00
Nicola Murino
469d36d979 certificate auth: fix source address checking inside crypto/ssh
So we can avoid to check source address ourself

81aafe6d26

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2020-05-16 15:15:32 +02:00
Nicola Murino
82fb7f8cf0 update proxyproto to v0.1.3
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2020-05-14 20:10:33 +02:00
Nicola Murino
e0183217b6 test cases: simplify TestLoginInvalidFs
we can simulate an invalid filesystem config using a GCS user without a
credentials file
2020-05-07 19:47:46 +02:00
Nicola Murino
f066b7fb9c use upstream pipeat
my patches are now merged
2020-05-07 00:05:40 +02:00
Nicola Murino
da90dbe645 tests: update config to use testify
we should port the other test cases to testify too
2020-05-02 15:47:23 +02:00
Nicola Murino
4f668bf558 simplify some httpd related code
and update chi, cobra and viper
2020-04-21 19:24:38 +02:00
Nicola Murino
5a5912ea66 switch to my pkg/sftp branch and enable the request-server allocator
This way we have performance comparable to OpenSSH if the cipher
isn't the bottleneck
2020-04-10 23:35:57 +02:00
Nicola Murino
b1c7317cf6 add support for partial authentication
Multi-step authentication is activated disabling all single-step
auth methods for a given user
2020-04-09 23:32:42 +02:00
Nicola Murino
fb9e188e36 systemd service: add ExecReload 2020-04-05 11:36:29 +02:00
Nicola Murino
c93d8cecfc update deps
chi 4.1.0 requires some minor code changes
2020-04-03 22:30:30 +02:00
Nicola Murino
952b10a9f6 update boltdb to v1.3.4
update other deps too
2020-03-21 10:12:30 +01:00
Nicola Murino
f55851bdc8 update nathanaelle password to v2
Fixes #97
2020-03-20 17:25:38 +01:00
Nicola Murino
ddcb500c51 update pipeat
it contains my latest performance patch that remove extraneous
allocation.

This improve performance for S3 and GCS
2020-03-15 01:36:19 +01:00
Nicola Murino
e8664c0ce4 docker: update docs
update dependencies too
2020-03-14 15:27:03 +01:00
Nicola Murino
3b002ddc86 improve performance
- use latest pkg/sftp that contains my latest performance patch
- replace default crypto with my branch that use minio sha256-simd
instead of Golang SHA256 implementation, this improve performance on
some hardware
2020-03-13 19:37:51 +01:00
Nicola Murino
ed1c7cac17 update deps
we now use git master for pkg/sftp: it includes the performance patches
from my copy branch.
2020-03-02 10:13:49 +01:00
Nicola Murino
830e3d1f64 Support for HAProxy PROXY protocol
you can proxy and/or load balance the SFTP/SCP service without losing
the information about the client's address.
2020-02-27 09:21:30 +01:00
Nicola Murino
382c6fda89 updated dependencies
fix CVE-2020-9283
2020-02-21 00:23:00 +01:00
Nicola Murino
0b7be1175d parse ssh commands with shlex
instead of use our bugged home made method.

Fixes #72
2020-02-14 16:17:32 +01:00
Nicola Murino
8b039e0447 httpd: add support for basic auth and HTTPS 2020-02-04 00:08:00 +01:00
Nicola Murino
31a433cda2 update deps and simplify some code 2020-01-31 23:26:56 +01:00
Nicola Murino
3491717c26 add support for serving Google Cloud Storage over SFTP/SCP
Each user can be mapped with a Google Cloud Storage bucket or a bucket
virtual folder
2020-01-31 19:04:00 +01:00
Nicola Murino
5f4efc9148 S3: add metrics 2020-01-23 23:17:00 +01:00
Nicola Murino
7ebbbe5c29 S3: update pipeat to the latest commit in my fork
Here are the main improvements:

- unliked files works on windows too
- the uploads are now synced on the lower speed between the SFTP client write
and the upload speed to S3

This commit increase the external auth timeout to 60 seconds too
2020-01-22 19:42:23 +01:00
Nicola Murino
a4834f4a83 add basic S3-Compatible Object Storage support
we have now an interface for filesystem backeds, this make easy to add
new filesystem backends
2020-01-19 07:41:05 +01:00
Nicola Murino
ec194d73d2 update pkg/sftp to a stable version ...
... now that it contains all the needed patches.
Remove an hack for setstat with empty attrs, it is now handled in pkg/sftp.
Update other dependencies too.
2020-01-11 14:29:30 +01:00
Nicola Murino
c7e55db4e0 update dependencies 2019-12-30 19:27:50 +01:00
Nicola Murino
363b9ccc7f sftpd: explicitly configure supported SFTP extensions
update pkg/sftp to a git revision that includes the needed patch

https://github.com/pkg/sftp/pull/315
2019-11-12 07:37:47 +01:00
Nicola Murino
77db2bd3d1 fix posix rename
update pkg/sftp to a git revision that includes:

https://github.com/pkg/sftp/pull/316

add a test case here too and update other deps
2019-11-08 08:43:27 +01:00
Nicola Murino
c5a6ca5650 portable mode: advertise service via multicast dns
Fixes #51
2019-10-26 18:25:53 +02:00
Nicola Murino
2932dba5cc update modules deps 2019-10-14 08:36:39 +02:00
Nicola Murino
bba78763e1 add a test case and document sha512crypt passwords support 2019-09-15 12:23:27 +02:00
Nicola Murino
7eb5b01169 add Prometheus support
some basic counters and gauges are now exposed
2019-09-13 18:45:36 +02:00
Nicola Murino
fd59f35108 update pkg/sftp to the latest master
Our pull request to handle transfer errors is now merged, so updating
pkg/sftp should fix #36
2019-09-13 08:30:22 +02:00
Nicola Murino
cc2ccb3dd7 go.sum cleanup 2019-09-03 12:18:19 +02:00
Nicola Murino
4a1b67454e set version to 0.9.1 2019-09-02 09:36:02 +02:00
Nicola Murino
0e6113eef6 remove uneeded deps using go mod tidy 2019-08-27 15:59:00 +02:00
Nicola Murino
2785a1c7c7 update modules deps 2019-08-27 10:26:04 +02:00
Nicola Murino
96a39a36bb dataprovider: add support for bbolt key/value store
This way there is an alternative for embedded/small systems if CGO
is disabled at build time and so SQLite support cannot be compiled
2019-08-12 18:31:31 +02:00
Nicola Murino
8f421b7d0f switch to viper for configuration and use cobra for cli 2019-08-07 22:46:52 +02:00
Nicola Murino
80b9c40489 add support for atomic upload
Atomic uploads are now configurable. The default upload mode remains
non atomic
2019-08-04 09:37:58 +02:00
Nicola Murino
14f919df47 simplify some code 2019-07-21 00:19:17 +02:00
Nicola Murino
22b8dc7f1d fix misspells and ineffectual assignments 2019-07-20 21:17:53 +02:00
Nicola Murino
31cd4d7139 first version 2019-07-20 12:26:52 +02:00