Commit graph

515 commits

Author SHA1 Message Date
Nicola Murino
9046acbe68 add HTTP hooks
external auth, pre-login user modification and keyboard interactive
authentication is now supported via HTTP requests too
2020-04-01 23:25:23 +02:00
Nicola Murino
075bbe2aef added test case that checks quota for files inside virtual folders 2020-03-29 11:10:03 +02:00
Nicola Murino
b52d078986 pbkdf2: fix password comparison
the key len for the derived function must be equal to the len of the
expected key
2020-03-28 16:09:06 +01:00
Nicola Murino
0a9c4914aa pre-login program: allow to create a new user too
clarify the difference between dynamic user creation/update and external
authentication
2020-03-27 23:26:22 +01:00
Nicola Murino
f284008fb5 enable scp in default configuration
remove the deprecated enable_scp setting
2020-03-26 23:38:24 +01:00
Nicola Murino
4759254e10 file actions: add bucket and endpoint to notifications
The HTTP notifications are now invoked as POST and the notification is
a JSON inside the POST body.

This is a backward incompatible change but this way the actions can be
extended more easily, sorry for the trouble

Fixes #101
2020-03-25 18:36:33 +01:00
Nicola Murino
e22d377203 docs: clarify "ca-certificates" requirement
Fixes #98
2020-03-22 20:17:36 +01:00
Nicola Murino
0787e3e595 bolt provider: fix error handling for get users with username filter 2020-03-22 15:37:08 +01:00
Nicola Murino
c1194d558c docs: minor improvements 2020-03-22 14:03:06 +01:00
Nicola Murino
952b10a9f6 update boltdb to v1.3.4
update other deps too
2020-03-21 10:12:30 +01:00
Nicola Murino
f55851bdc8 update nathanaelle password to v2
Fixes #97
2020-03-20 17:25:38 +01:00
Nicola Murino
76bb361393 docs: add built-in profiler 2020-03-15 23:33:12 +01:00
Nicola Murino
81c8e8d898 add profiler support
profiling is now available via the HTTP base URL /debug/pprof/

examples, use this URL to start and download a 30 seconds CPU profile:

/debug/pprof/profile?seconds=30

use this URL to profile used memory:

/debug/pprof/heap?gc=1

use this URL to profile allocated memory:

/debug/pprof/allocs?gc=1

Full docs here:

https://golang.org/pkg/net/http/pprof/
2020-03-15 15:16:35 +01:00
Nicola Murino
f4e872c782 portable mode: add flags for s3 upload part size and concurrency 2020-03-15 11:40:06 +01:00
Nicola Murino
ddcb500c51 update pipeat
it contains my latest performance patch that remove extraneous
allocation.

This improve performance for S3 and GCS
2020-03-15 01:36:19 +01:00
Nicola Murino
e8664c0ce4 docker: update docs
update dependencies too
2020-03-14 15:27:03 +01:00
Nicola Murino
3b002ddc86 improve performance
- use latest pkg/sftp that contains my latest performance patch
- replace default crypto with my branch that use minio sha256-simd
instead of Golang SHA256 implementation, this improve performance on
some hardware
2020-03-13 19:37:51 +01:00
Nicola Murino
1770da545d s3: upload concurrency is now configurable
Please note that if the upload bandwidth between the SFTP client and
SFTPGo is greater than the upload bandwidth between SFTPGo and S3 then
the SFTP client have to wait for the upload of the last parts to S3
after it ends the file upload to SFTPGo, and it may time out.
Keep this in mind if you customize parts size and upload concurrency
2020-03-13 19:13:58 +01:00
Nicola Murino
de3e69f846 s3: add documentation and test cases for upload part size 2020-03-13 17:28:55 +01:00
Michael Bonfils
cdf1233065 s3: export PartSize parameter
By default AWS SDK use part_size of 5 MB. For big files,
it is not ideal case. For Hadoop, it is not uncommon to
use 512 MB.
2020-03-13 17:28:04 +01:00
Nicola Murino
6b70f0b25f back to development 2020-03-07 18:06:46 +01:00
Nicola Murino
4fe51f7cce set version to 0.9.6 2020-03-07 13:36:46 +01:00
HiFiPhile
7221bf9b25 Add performance summary
Pull request #92
2020-03-06 22:48:55 +01:00
Nicola Murino
61f20f5449 Linux: add basic instructions to run SFTPGo as service 2020-03-06 09:24:55 +01:00
Nicola Murino
5dafbb54de macOS: add basic instructions to run SFTPGo as service 2020-03-05 23:26:47 +01:00
Nicola Murino
ec8ab28a22 portable mode: add support for file extensions filters 2020-03-05 15:37:10 +01:00
Nicola Murino
aaa6d0c71f docs: fix some typos 2020-03-05 09:32:29 +01:00
Nicola Murino
ea74aca165 doc: some other minor improvements 2020-03-05 00:01:40 +01:00
Nicola Murino
9b119765fc docs: minor improvements 2020-03-04 23:51:16 +01:00
Jo Vandeginste
df02496145 Refactor docs 2020-03-04 23:10:58 +01:00
Nicola Murino
31d285813e windows: try to escape trailing double quote in user input
we try to remove the trailing double quote for user input such as this one

sftpgo.exe serve -c "C:\ProgramData\SFTPGO\"

the value for the -c flag is parsed as:

C:\ProgramData\SFTPGO"

this is what the user specified, but the user want this value:

C:\ProgramData\SFTPGO

so we try to remove the trailing double quote.

Please note that we cannot do anything for something like this:

-c "C:\ProgramData\SFTPGO\" -l "sftpgo.log"

in this case the -l flag will be ignored and the value for the c flag is:

C:\ProgramData\SFTPGO" -l sftpgo.log

and so probably it is invalid. This is definitely a bad user input
2020-03-04 09:27:14 +01:00
Germs2004
f9fc5792fd fixed minor typos and edits 2020-03-04 08:01:02 +01:00
Germs2004
6ad9c5ae64 minor typo 2020-03-04 08:00:34 +01:00
Nicola Murino
016abda6d7 improve docs 2020-03-03 23:25:23 +01:00
Nicola Murino
2eea6c95b9 windows setup: use ProgramData folder to store application data
this is a backward incompatible change, but it is the way to do the
things on Windows.
2020-03-03 22:31:13 +01:00
Nicola Murino
7f1946de34 improve validations for user provided file and directory paths 2020-03-03 09:09:58 +01:00
Nicola Murino
d0a81cabab log file: if the path is not absolute make it relative to config dir
Also refuse to join invalid file name such as "."

Fixes #85
2020-03-03 00:34:06 +01:00
Nicola Murino
df67f4ef34 clean config dir
Fixes #80
2020-03-02 22:56:14 +01:00
Nicola Murino
ed11e1128a docs: clarify the initprovider command is required for SQLite too 2020-03-02 10:34:31 +01:00
Nicola Murino
ed1c7cac17 update deps
we now use git master for pkg/sftp: it includes the performance patches
from my copy branch.
2020-03-02 10:13:49 +01:00
Nicola Murino
7c115aa9c8 windows service: only restart twice
reset the service failure count to zero after 3600 seconds.

Fixes #83
2020-03-02 09:58:36 +01:00
Nicola Murino
3ffddcba92 web: log an error if loading a required template fails
We used template.Must that panics if an error happen but the error is
visible only if sftpgo is started in an interactive way

Fixes #82
2020-03-02 09:34:13 +01:00
Nicola Murino
833b702b90 proxy protocol: add list of allowed IP addresses and IP ranges
"proxy_allowed" setting allows to specify the allowed IP address and IP
ranges that can send the proxy header. This setting combined with
"proxy_protocol" allows to ignore the header or to reject connections
that send the proxy header from a non listed IP
2020-03-01 23:12:28 +01:00
Nicola Murino
b885d453a2 filters: we can now set allowed and denied files extensions 2020-03-01 22:10:29 +01:00
Nicola Murino
7163fde724 proxy protocol: added an option to make the proxy header required
now we can configure SFTPGo to accept or reject requests without the proxy
header when the proxy protocol is enabled
2020-02-29 00:02:06 +01:00
Nicola Murino
830e3d1f64 Support for HAProxy PROXY protocol
you can proxy and/or load balance the SFTP/SCP service without losing
the information about the client's address.
2020-02-27 09:21:30 +01:00
Mengsk
637463a068 Rename before_login_program to pre_login_program
and some documentation update
2020-02-25 16:34:54 +01:00
Nicola Murino
e69536f540 fixed some typos and improved a log 2020-02-25 12:46:52 +01:00
Mengsk
c516780289 Documentation update 1 2020-02-25 12:41:28 +01:00
Nicola Murino
eb1b869b73 virtual folders fixes
scp now properly handles virtual folders.

rsync is disabled for users with virtual folders: we execute a system
command and it is not aware about virtual folders.

git is not allowed if the repo path is inside a virtual folder
2020-02-24 18:54:35 +01:00