beenull/sftpgo
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
1243 commits 8 branches 48 tags 35 MiB
Commit graph

329 commits

Author SHA1 Message Date
Nicola Murino
a43854ae9b
OpenAPI: document that secrets are automatically encrypted before saving 2021-03-28 11:23:06 +02:00
Nicola Murino
5f49af1780
external auth: allow to inspect and preserve an existing user 2021-03-26 15:19:01 +01:00
Nicola Murino
6bc5c64a3a
webdav: ignore path, perm and not exist errors in PROPFIND 
Fixes #340
 2021-03-24 13:32:20 +01:00
Nicola Murino
70e035315e
data provider: add CockroachDB support 2021-03-23 19:14:15 +01:00
Nicola Murino
800e64404b
update deps 2021-03-22 08:55:35 +01:00
Nicola Murino
67ea75cf03
improve OpenAPI schema so it is better rendered on Stoplight 2021-03-07 18:41:56 +01:00
Nicola Murino
055506e518
sftpfs: add an option to disable concurrent reads 2021-03-06 15:41:40 +01:00
Nicola Murino
88122ba2f8
update jwtauth to v5 2021-03-05 18:50:45 +01:00
Nicola Murino
ad07fc78eb
update nfpm and deps 2021-03-03 18:39:58 +01:00
Nicola Murino
534b253c20
WebDAV: improve TLS certificate authentication 
For each user you can now configure:

- TLS certificate auth
- TLS certificate auth and password
- Password auth

For TLS certificate auth, the certificate common name is used as
username
 2021-03-01 19:28:11 +01:00
Nicola Murino
a6e36e7cad
FTP: improve TLS certificate authentication 
For each user you can now configure:

- TLS certificate auth
- TLS certificate auth and password
- Password auth

For TLS auth, the certificate common name must match the name provided
using the "USER" FTP command
 2021-02-28 12:10:40 +01:00
Nicola Murino
4b2edff6dd
update deps 2021-02-24 22:27:52 +01:00
Nicola Murino
8cc2dfe5c2
update pkg/sftp 
we don't need my branch anymore now that all the required features for
the sftpfs are available upstream too
 2021-02-22 16:27:45 +01:00
Nicola Murino
49830516be
squash database migrations and remove compat code 2021-02-22 08:37:50 +01:00
Nicola Murino
1129a868a5
Improve powershell completion 
cobra 1.1.3 has much better powershell support
 2021-02-13 09:10:35 +01:00
Nicola Murino
51f110bc7b
sftpd: add statvfs@openssh.com support 2021-02-11 19:45:52 +01:00
Nicola Murino
05ae0ea5f2
config: fix bindings backward compatibility 2021-02-06 09:53:31 +01:00
Nicola Murino
fc9082c422
webdav: try to handle HEAD for collection too 
The underlying golang webdav library returns Method Not Allowed for
HEAD requests on directories:

https://github.com/golang/net/blob/master/webdav/webdav.go#L210

let's see if we can workaround this inside SFTPGo itself in a similar
way as we do for GET.

The HEAD response will not return a Content-Length, we cannot handle
this inside SFTPGo.

Fixes #294
 2021-02-03 22:36:13 +01:00
Nicola Murino
78bf808322
virtual folders: change dataprovider structure 
This way we no longer depend on the local file system path and so we can
add support for cloud backends in future updates
 2021-02-01 19:04:15 +01:00
Nicola Murino
80f5ccd357
web admin: add backup/restore 2021-01-22 19:42:18 +01:00
Nicola Murino
820169c5c6
windows service: simplify code 
update testify to 1.7.0 too
 2021-01-21 19:07:13 +01:00
Nicola Murino
778ec9b88f
REST API v2 
- add JWT authentication
- admins are now stored inside the data provider
- admin access can be restricted based on the source IP: both proxy
  header and connection IP are checked
- deprecate REST API CLI: it is not relevant anymore

Some other changes to the REST API can still happen before releasing
SFTPGo 2.0.0

Fixes #197
 2021-01-17 22:29:08 +01:00
Nicola Murino
aa40b04576
update deps 2021-01-05 12:40:49 +01:00
Nicola Murino
72b2c83392
defender: allow hot-reloading for safe and block lists 2021-01-04 17:52:14 +01:00
Nicola Murino
037d89a320
add support for a basic built-in defender 
It can help to prevent DoS and brute force password guessing
 2021-01-02 14:05:09 +01:00
Nicola Murino
3c16a19269
FTP: update ftpserverlib 
fixes another sneaky bug
 2020-12-28 09:22:52 +01:00
Nicola Murino
b3c6d79f51
FTP: add support for ASCII transfer mode 
the default remain binary, a client have to explicitly request an
ASCII transfer
 2020-12-27 09:48:56 +01:00
Nicola Murino
2a95d031ea
FTP: add support for AVBL command 2020-12-25 11:14:08 +01:00
Nicola Murino
1dce1eff48
improve FTP support 
- allow to disable active mode
- allow to disable SITE commands
- add optional support for calculating hash value of files
- add optional support for the non standard COMB command
 2020-12-24 18:48:06 +01:00
Nicola Murino
c69d63c1f8
add support for multiple bindings 
Fixes #253
 2020-12-23 16:12:30 +01:00
Nicola Murino
65e6d5475f
update ftpserverlib to include the latest fixes and features 2020-12-18 08:49:32 +01:00
Nicola Murino
15609cdbc7
fix build on FreeBSD 
see https://github.com/otiai10/copy/pull/36
 2020-12-17 14:46:31 +01:00
Nicola Murino
f876c728ad
add support for the latest ftpserverlib and azblob versions 2020-12-17 13:40:36 +01:00
Nicola Murino
a6985075b9
add sftpfs storage backend 
Fixes #224
 2020-12-12 10:31:09 +01:00
Nicola Murino
ab1bf2ad44
update deps 2020-12-06 22:20:53 +01:00
Nicola Murino
4a88ea5c03
add Data At Rest Encryption support 2020-12-05 13:48:13 +01:00
Nicola Murino
634b723b5d
add KMS support 
Fixes #226
 2020-11-30 21:46:34 +01:00
Nicola Murino
5d789a01b7
update pkg/sftp 
These patches are now merged upstream:

https://github.com/pkg/sftp/pull/392
https://github.com/pkg/sftp/pull/393
 2020-11-18 19:06:12 +01:00
Nicola Murino
ca0ff0d630
add a File interface so we can avoid to use os.File directly 2020-11-17 19:36:39 +01:00
Nicola Murino
969b38586e
update pkg/sftp to fix requests accumulation 
Include this patch:

https://github.com/pkg/sftp/pull/393

to avoid request accumulation (no underlying fd) if we return an error.
Before this patch the accumulated requests are released only when the
client disconnects.

We use our fork for now to include

https://github.com/pkg/sftp/pull/392

too
 2020-11-16 19:49:26 +01:00
Nicola Murino
e3eca424f1
web admin: allow both allowed and denied extensions/patterns for a dir 
this fix a regression introduced in the previous commit
 2020-11-16 19:21:50 +01:00
Nicola Murino
dc845fa2f4
webdav: fix permission errors if the client try to read multiple times 2020-11-14 19:19:41 +01:00
Nicola Murino
19a619ff65
Linux pkgs: use python3 for API CLI inside generated deb 2020-11-14 09:10:45 +01:00
Nicola Murino
14035f407e
set version to 1.2.0 2020-11-08 06:14:03 +01:00
Nicola Murino
4edb9cd6b9
simplify some code 2020-11-07 18:05:47 +01:00
Nicola Murino
1d5d184720
webdav file: ensure to close the reader only once 2020-11-05 09:30:38 +01:00
Nicola Murino
0a14297b48
webdav: performance improvements and bug fixes 
we need my custom golang/x/net/webdav fork for now

https://github.com/drakkan/net/tree/sftpgo
 2020-11-04 19:11:40 +01:00
Nicola Murino
57bec976ae
document heathz endpoint 2020-11-01 10:39:10 +01:00
Nicola Murino
fcfdd633f6
Azure Blob: update SDK and add access tier support 2020-10-30 22:17:17 +01:00
Nicola Murino
ac3bae00fc
add support for SFTP subsystem mode 
Fixes #204
 2020-10-29 19:23:33 +01:00
Nicola Murino
5ff8f75917
add Azure Blob support 2020-10-25 08:18:48 +01:00
Nicola Murino
d12819932a
update cobra to v1.1.1 
this version fix the man page generation so we don't need to use
our branch anymore
 2020-10-18 21:52:42 +02:00
Nicola Murino
756c3d0503
fix man page generation 
other minor changes
 2020-10-17 22:14:04 +02:00
Nicola Murino
ce9387f1ab
update dependencies and some docs 2020-10-09 20:25:42 +02:00
Nicola Murino
2d6c0388af
update deps 2020-10-04 18:29:42 +02:00
Nicola Murino
4d19d87720
pkgs: use glob notation to include static folder 2020-10-02 18:16:49 +02:00
Nicola Murino
55515fee95
update deps, GCS can no finally use attribute selection 
See https://github.com/googleapis/google-cloud-go/pull/2661
 2020-09-28 12:51:19 +02:00
Nicola Murino
38f06ab373 ftpd: fix TLS for active connections 
See https://github.com/fclairamb/ftpserverlib/issues/177

Some minor doc improvements
 2020-09-17 09:45:40 +02:00
Nicola Murino
61003c8079 sftpd: add lstat support 2020-09-11 09:30:25 +02:00
Nicola Murino
b9c381e26f sftpd: update pkg/sftp 
The patch to open a file in read/write mode is now merged
 2020-09-06 11:40:31 +02:00
Nicola Murino
542554fb2c replace the library to verify UNIX's crypt(3) passwords 2020-09-04 21:08:09 +02:00
Nicola Murino
afc411c51b adjust runtime.GOMAXPROCS to match the container CPU quota, if any 2020-09-03 18:09:45 +02:00
Nicola Murino
dbed110d02 WebDAV: add caching for authenticated users 
In this way we get a big performance boost
 2020-08-31 19:25:17 +02:00
Nicola Murino
4748e6f54d
sftpd: handle read and write from the same handle (#158) 
Fixes #155
 2020-08-31 06:45:22 +02:00
Nicola Murino
2746c0b0f1 move stat to base connection and differentiate between Stat and Lstat 
we will use Lstat once it will be exposed in pkg/sftp
 2020-08-25 18:23:00 +02:00
Nicola Murino
701a6115f8 ftpd: use ftpserverlib master, the tls patch is now merged 2020-08-24 23:06:10 +02:00
Nicola Murino
56b00addc4 docker: try to improve the docs 
See #159
 2020-08-24 15:46:31 +02:00
Nicola Murino
5208e4a4ca sftpd: improve truncate 
quota usage and max allowed write size are now properly updated after a
truncate
 2020-08-22 10:12:00 +02:00
Nicola Murino
f41ce6619f sftpd: add SSH_FXP_FSETSTAT support 
This change will fix file editing from sshfs, we need this patch

https://github.com/pkg/sftp/pull/373

for pkg/sftp to support this feature
 2020-08-20 13:54:36 +02:00
Nicola Murino
8b0a1817b3 add check password hook 
its main use case is to allow to easily support things like password+OTP for
protocols without keyboard interactive support such as FTP and WebDAV
 2020-08-19 19:36:12 +02:00
Nicola Murino
0dbf0cc81f WebDAV: add CORS support 2020-08-15 15:55:20 +02:00
Nicola Murino
196a56726e FTP improvements 
- add a setting to require TLS
- add symlink support

require TLS 1.2 for all TLS connections
 2020-08-15 13:02:25 +02:00
Nicola Murino
a9e21c282a add WebDAV support 
Fixes #147
 2020-08-11 23:56:10 +02:00
Nicola Murino
fa41bfd06a Cloud backends: add support for FTP REST command 
So partial downloads are now supported as for local fs
 2020-08-03 18:03:09 +02:00
Nicola Murino
8839c34d53 FTP: implements ClientDriverExtensionRemoveDir 
Fixes #149 for FTP too
 2020-08-03 17:36:43 +02:00
Nicola Murino
2a9f7db1e2 Cloud FS: don't propagate the error if removing a folder returns not found 
for Cloud FS the folders are virtual and they, generally, disappear when the
last file is removed.

This fix doesn't work for FTP protocol for now.

Fixes #149
 2020-07-31 19:24:57 +02:00
Nicola Murino
59a21158a6 fix FTP quota limits test case 
It failed sometime due to a bug in the ftp client library used in test
cases. The failure was more frequent on FreeBSD but it could happen in
any supported OS. It was not systematic since we use small files in
test cases.

See https://github.com/jlaffaye/ftp/pull/192
 2020-07-30 19:52:29 +02:00
Nicola Murino
93ce96d011 add support for the venerable FTP protocol 
Fixes #46
 2020-07-29 21:56:56 +02:00
Nicola Murino
4e41a5583d refactoring: add common package 
The common package defines the interfaces that a protocol must implement
and contain code that can be shared among supported protocols.

This way should be easier to support new protocols
 2020-07-24 23:39:38 +02:00
Nicola Murino
96cbce52f9 cmd: add shell completion and man pages generators 2020-07-08 23:21:33 +02:00
Nicola Murino
fc32286045 update deps 2020-07-05 22:54:00 +02:00
Nicola Murino
403b9a8310 replace Travis with GitHub actions 2020-06-20 21:57:51 +02:00
Nicola Murino
33436488e2 update deps 2020-06-20 16:09:55 +02:00
Nicola Murino
3d48fa7382 ssh commands: add sftpgo-copy and sftpgo-remove 
Fixes #122
 2020-06-13 22:48:51 +02:00
Nicola Murino
760cc9ba5a partial auth: fix public key query response 
more details here:

https://github.com/golang/crypto/pull/130#issuecomment-633191423
 2020-05-24 12:13:14 +02:00
Nicola Murino
469d36d979 certificate auth: fix source address checking inside crypto/ssh 
So we can avoid to check source address ourself

81aafe6d26

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
 2020-05-16 15:15:32 +02:00
Nicola Murino
82fb7f8cf0 update proxyproto to v0.1.3 
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
 2020-05-14 20:10:33 +02:00
Nicola Murino
e0183217b6 test cases: simplify TestLoginInvalidFs 
we can simulate an invalid filesystem config using a GCS user without a
credentials file
 2020-05-07 19:47:46 +02:00
Nicola Murino
f066b7fb9c use upstream pipeat 
my patches are now merged
 2020-05-07 00:05:40 +02:00
Nicola Murino
da90dbe645 tests: update config to use testify 
we should port the other test cases to testify too
 2020-05-02 15:47:23 +02:00
Nicola Murino
4f668bf558 simplify some httpd related code 
and update chi, cobra and viper
 2020-04-21 19:24:38 +02:00
Nicola Murino
5a5912ea66 switch to my pkg/sftp branch and enable the request-server allocator 
This way we have performance comparable to OpenSSH if the cipher
isn't the bottleneck
 2020-04-10 23:35:57 +02:00
Nicola Murino
b1c7317cf6 add support for partial authentication 
Multi-step authentication is activated disabling all single-step
auth methods for a given user
 2020-04-09 23:32:42 +02:00
Nicola Murino
fb9e188e36 systemd service: add ExecReload 2020-04-05 11:36:29 +02:00
Nicola Murino
c93d8cecfc update deps 
chi 4.1.0 requires some minor code changes
 2020-04-03 22:30:30 +02:00
Nicola Murino
952b10a9f6 update boltdb to v1.3.4 
update other deps too
 2020-03-21 10:12:30 +01:00
Nicola Murino
f55851bdc8 update nathanaelle password to v2 
Fixes #97
 2020-03-20 17:25:38 +01:00
Nicola Murino
ddcb500c51 update pipeat 
it contains my latest performance patch that remove extraneous
allocation.

This improve performance for S3 and GCS
 2020-03-15 01:36:19 +01:00
Nicola Murino
e8664c0ce4 docker: update docs 
update dependencies too
 2020-03-14 15:27:03 +01:00
Nicola Murino
3b002ddc86 improve performance 
- use latest pkg/sftp that contains my latest performance patch
- replace default crypto with my branch that use minio sha256-simd
instead of Golang SHA256 implementation, this improve performance on
some hardware
 2020-03-13 19:37:51 +01:00
Nicola Murino
ed1c7cac17 update deps 
we now use git master for pkg/sftp: it includes the performance patches
from my copy branch.
 2020-03-02 10:13:49 +01:00
Nicola Murino
830e3d1f64 Support for HAProxy PROXY protocol 
you can proxy and/or load balance the SFTP/SCP service without losing
the information about the client's address.
 2020-02-27 09:21:30 +01:00
Nicola Murino
382c6fda89 updated dependencies 
fix CVE-2020-9283
 2020-02-21 00:23:00 +01:00
Nicola Murino
0b7be1175d parse ssh commands with shlex 
instead of use our bugged home made method.

Fixes #72
 2020-02-14 16:17:32 +01:00
Nicola Murino
8b039e0447 httpd: add support for basic auth and HTTPS 2020-02-04 00:08:00 +01:00
Nicola Murino
31a433cda2 update deps and simplify some code 2020-01-31 23:26:56 +01:00
Nicola Murino
3491717c26 add support for serving Google Cloud Storage over SFTP/SCP 
Each user can be mapped with a Google Cloud Storage bucket or a bucket
virtual folder
 2020-01-31 19:04:00 +01:00
Nicola Murino
5f4efc9148 S3: add metrics 2020-01-23 23:17:00 +01:00
Nicola Murino
7ebbbe5c29 S3: update pipeat to the latest commit in my fork 
Here are the main improvements:

- unliked files works on windows too
- the uploads are now synced on the lower speed between the SFTP client write
and the upload speed to S3

This commit increase the external auth timeout to 60 seconds too
 2020-01-22 19:42:23 +01:00
Nicola Murino
a4834f4a83 add basic S3-Compatible Object Storage support 
we have now an interface for filesystem backeds, this make easy to add
new filesystem backends
 2020-01-19 07:41:05 +01:00
Nicola Murino
ec194d73d2 update pkg/sftp to a stable version ... 
... now that it contains all the needed patches.
Remove an hack for setstat with empty attrs, it is now handled in pkg/sftp.
Update other dependencies too.
 2020-01-11 14:29:30 +01:00
Nicola Murino
c7e55db4e0 update dependencies 2019-12-30 19:27:50 +01:00
Nicola Murino
363b9ccc7f sftpd: explicitly configure supported SFTP extensions 
update pkg/sftp to a git revision that includes the needed patch

https://github.com/pkg/sftp/pull/315
 2019-11-12 07:37:47 +01:00
Nicola Murino
77db2bd3d1 fix posix rename 
update pkg/sftp to a git revision that includes:

https://github.com/pkg/sftp/pull/316

add a test case here too and update other deps
 2019-11-08 08:43:27 +01:00
Nicola Murino
c5a6ca5650 portable mode: advertise service via multicast dns 
Fixes #51
 2019-10-26 18:25:53 +02:00
Nicola Murino
2932dba5cc update modules deps 2019-10-14 08:36:39 +02:00
Nicola Murino
bba78763e1 add a test case and document sha512crypt passwords support 2019-09-15 12:23:27 +02:00
Nicola Murino
7eb5b01169 add Prometheus support 
some basic counters and gauges are now exposed
 2019-09-13 18:45:36 +02:00
Nicola Murino
fd59f35108 update pkg/sftp to the latest master 
Our pull request to handle transfer errors is now merged, so updating
pkg/sftp should fix #36
 2019-09-13 08:30:22 +02:00
Nicola Murino
cc2ccb3dd7 go.sum cleanup 2019-09-03 12:18:19 +02:00
Nicola Murino
4a1b67454e set version to 0.9.1 2019-09-02 09:36:02 +02:00
Nicola Murino
0e6113eef6 remove uneeded deps using go mod tidy 2019-08-27 15:59:00 +02:00
Nicola Murino
2785a1c7c7 update modules deps 2019-08-27 10:26:04 +02:00
Nicola Murino
96a39a36bb dataprovider: add support for bbolt key/value store 
This way there is an alternative for embedded/small systems if CGO
is disabled at build time and so SQLite support cannot be compiled
 2019-08-12 18:31:31 +02:00
Nicola Murino
8f421b7d0f switch to viper for configuration and use cobra for cli 2019-08-07 22:46:52 +02:00
Nicola Murino
80b9c40489 add support for atomic upload 
Atomic uploads are now configurable. The default upload mode remains
non atomic
 2019-08-04 09:37:58 +02:00
Nicola Murino
14f919df47 simplify some code 2019-07-21 00:19:17 +02:00
Nicola Murino
22b8dc7f1d fix misspells and ineffectual assignments 2019-07-20 21:17:53 +02:00
Nicola Murino
31cd4d7139 first version 2019-07-20 12:26:52 +02:00