Nicola Murino
778ec9b88f
REST API v2
...
- add JWT authentication
- admins are now stored inside the data provider
- admin access can be restricted based on the source IP: both proxy
header and connection IP are checked
- deprecate REST API CLI: it is not relevant anymore
Some other changes to the REST API can still happen before releasing
SFTPGo 2.0.0
Fixes #197
2021-01-17 22:29:08 +01:00
Nicola Murino
173c1820e1
Go 1.15 is now required
...
VerifyConnection is not available in 1.14
2021-01-03 17:25:24 +01:00
Nicola Murino
037d89a320
add support for a basic built-in defender
...
It can help to prevent DoS and brute force password guessing
2021-01-02 14:05:09 +01:00
Nicola Murino
40e759c983
FTP: add support for client certificate authentication
2020-12-29 09:20:09 +01:00
Nicola Murino
14d1b82f6b
minor README improvements
2020-12-14 07:54:27 +01:00
Nicola Murino
a6985075b9
add sftpfs storage backend
...
Fixes #224
2020-12-12 10:31:09 +01:00
Nicola Murino
4a88ea5c03
add Data At Rest Encryption support
2020-12-05 13:48:13 +01:00
Nicola Murino
2142ef20c5
fix some typos
2020-11-26 22:18:12 +01:00
Nicola Murino
224ce5fe81
add revertprovider subcommand
...
Fixes #233
2020-11-26 22:08:33 +01:00
Nicola Murino
a6355e298e
add support for limit files using shell like patterns
...
Fixes #209
2020-11-15 22:04:48 +01:00
Nicola Murino
957430e675
back to development
2020-11-08 12:56:37 +01:00
Nicola Murino
ac3bae00fc
add support for SFTP subsystem mode
...
Fixes #204
2020-10-29 19:23:33 +01:00
Nicola Murino
5ff8f75917
add Azure Blob support
2020-10-25 08:18:48 +01:00
Nicola Murino
bb5207ad77
Add support for loading users/folders on startup
...
Fixes #161
2020-10-20 18:42:37 +02:00
Nicola Murino
1625cd5a9f
back to development
2020-10-18 11:09:16 +02:00
Nicola Murino
6c0839e197
Improve docker images
2020-10-14 07:46:36 +02:00
Nicola Murino
80d695f3a2
back to development
2020-10-11 09:29:17 +02:00
Nicola Murino
5048d54d32
PPA: add source files used to build the packages
2020-10-08 18:20:15 +02:00
Nicola Murino
f22fe6af76
remove py extension from REST API CLI
2020-10-08 16:02:04 +02:00
Nicola Murino
87f78b07b3
docker: add some docs and build for arm64 too
2020-10-06 13:59:31 +02:00
Nicola Murino
c992072286
data provider: add a setting to prevent auto-update
2020-10-05 19:42:33 +02:00
Nicola Murino
3e2afc35ba
data provider: try to automatically initialize it if required
2020-10-05 12:55:49 +02:00
Nicola Murino
13d43a2d31
improve some docs
2020-09-27 09:24:10 +02:00
Nicola Murino
3c1300721c
add some basic how-to style documents
2020-09-13 19:43:56 +02:00
Giorgio Pellero
f978355520
Fix "compatible" typo in README.md ( #162 )
2020-08-31 13:43:24 +02:00
Nicola Murino
8b0a1817b3
add check password hook
...
its main use case is to allow to easily support things like password+OTP for
protocols without keyboard interactive support such as FTP and WebDAV
2020-08-19 19:36:12 +02:00
Nicola Murino
f3228713bc
Allow individual protocols to be enabled per user
...
Fixes #154
2020-08-17 12:49:20 +02:00
Nicola Murino
a9e21c282a
add WebDAV support
...
Fixes #147
2020-08-11 23:56:10 +02:00
Nicola Murino
93ce96d011
add support for the venerable FTP protocol
...
Fixes #46
2020-07-29 21:56:56 +02:00
Nicola Murino
da0f470f1c
document FreeBSD support
...
improve some tests cleanup
2020-07-10 19:20:37 +02:00
Nicola Murino
0ea2ca3141
simplify data provider usage
...
remove the obsolete SQL scripts too. They are not required since v0.9.6
2020-07-08 19:59:31 +02:00
Nicola Murino
d2a739f8f6
add workflow status badge
2020-06-28 21:01:03 +02:00
Nicola Murino
ddf99ab706
workflow: execute test cases on MySQL too
2020-06-22 20:02:51 +02:00
Nicola Murino
0056984d4b
Allow to rotate logs on demand
...
Log file can be rotated sending a SIGUSR1 signal on Unix based systems and
using "sftpgo service rotatelogs" on Windows
Fixes #133
2020-06-22 19:11:53 +02:00
Nicola Murino
8f934f2648
run test cases against bolt provider too
2020-06-20 23:49:27 +02:00
Nicola Murino
403b9a8310
replace Travis with GitHub actions
2020-06-20 21:57:51 +02:00
Nicola Murino
3d48fa7382
ssh commands: add sftpgo-copy and sftpgo-remove
...
Fixes #122
2020-06-13 22:48:51 +02:00
Nicola Murino
8e22dd1b13
virtual folders: allow overlapped mapped paths if quota is disabled
...
See #95
2020-06-10 09:11:32 +02:00
Nicola Murino
8306b6bde6
refactor virtual folders
...
The same virtual folder can now be shared among users and different
folder quota limits for each user are supported.
Fixes #120
2020-06-07 23:30:18 +02:00
Nicola Murino
5665e9c0e7
improve some docs
2020-05-23 12:47:44 +02:00
Nicola Murino
7ae8b2cdeb
move REST API CLI in examples directory
...
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2020-05-16 14:02:46 +02:00
Nicola Murino
738c7ab43e
sftpd: add support for SSH user certificate authentication
...
This add support for PROTOCOL.certkeys vendor extension:
https://cvsweb.openbsd.org/src/usr.bin/ssh/PROTOCOL.certkeys?rev=1.8
Fixes #117
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2020-05-15 20:08:53 +02:00
Nicola Murino
d70959c34c
fix some lint issues
2020-04-30 14:23:55 +02:00
Nicola Murino
b1c7317cf6
add support for partial authentication
...
Multi-step authentication is activated disabling all single-step
auth methods for a given user
2020-04-09 23:32:42 +02:00
Nicola Murino
0a9c4914aa
pre-login program: allow to create a new user too
...
clarify the difference between dynamic user creation/update and external
authentication
2020-03-27 23:26:22 +01:00
Nicola Murino
76bb361393
docs: add built-in profiler
2020-03-15 23:33:12 +01:00
HiFiPhile
7221bf9b25
Add performance summary
...
Pull request #92
2020-03-06 22:48:55 +01:00
Nicola Murino
aaa6d0c71f
docs: fix some typos
2020-03-05 09:32:29 +01:00
Nicola Murino
ea74aca165
doc: some other minor improvements
2020-03-05 00:01:40 +01:00
Nicola Murino
9b119765fc
docs: minor improvements
2020-03-04 23:51:16 +01:00
Jo Vandeginste
df02496145
Refactor docs
2020-03-04 23:10:58 +01:00
Germs2004
f9fc5792fd
fixed minor typos and edits
2020-03-04 08:01:02 +01:00
Nicola Murino
016abda6d7
improve docs
2020-03-03 23:25:23 +01:00
Nicola Murino
ed11e1128a
docs: clarify the initprovider command is required for SQLite too
2020-03-02 10:34:31 +01:00
Nicola Murino
833b702b90
proxy protocol: add list of allowed IP addresses and IP ranges
...
"proxy_allowed" setting allows to specify the allowed IP address and IP
ranges that can send the proxy header. This setting combined with
"proxy_protocol" allows to ignore the header or to reject connections
that send the proxy header from a non listed IP
2020-03-01 23:12:28 +01:00
Nicola Murino
b885d453a2
filters: we can now set allowed and denied files extensions
2020-03-01 22:10:29 +01:00
Nicola Murino
7163fde724
proxy protocol: added an option to make the proxy header required
...
now we can configure SFTPGo to accept or reject requests without the proxy
header when the proxy protocol is enabled
2020-02-29 00:02:06 +01:00
Nicola Murino
830e3d1f64
Support for HAProxy PROXY protocol
...
you can proxy and/or load balance the SFTP/SCP service without losing
the information about the client's address.
2020-02-27 09:21:30 +01:00
Mengsk
637463a068
Rename before_login_program to pre_login_program
...
and some documentation update
2020-02-25 16:34:54 +01:00
Nicola Murino
e69536f540
fixed some typos and improved a log
2020-02-25 12:46:52 +01:00
Mengsk
c516780289
Documentation update 1
2020-02-25 12:41:28 +01:00
Nicola Murino
eb1b869b73
virtual folders fixes
...
scp now properly handles virtual folders.
rsync is disabled for users with virtual folders: we execute a system
command and it is not aware about virtual folders.
git is not allowed if the repo path is inside a virtual folder
2020-02-24 18:54:35 +01:00
Nicola Murino
703ccc8d91
add support for dynamic users modifications
...
A custom program can be executed before the users login to modify the
configurations for the user trying to login.
You can, for example, allow login based on time range.
Fixes #77
2020-02-23 18:50:59 +01:00
Nicola Murino
45b9366dd0
add support for virtual folders
...
directories outside the user home directory can be exposed as virtual folders
2020-02-23 11:30:26 +01:00
Nicola Murino
bc11cdd8d5
add support for per user authentication methods
...
You can, for example, deny one or more authentication methods to one or
more users.
2020-02-19 22:39:30 +01:00
Nicola Murino
ae8ed75ae5
gcs: add support for automatic credentials
...
We can now also support implicit credentials using the Application
Default Credentials strategy
2020-02-19 09:41:15 +01:00
Nicola Murino
c8cc81cf4a
sftpd: autogenerate ecdsa key
...
With default configuration we now generate RSA and ECDSA server keys.
2020-02-16 18:17:39 +01:00
Nicola Murino
79c8b6cbc2
keyboard interactive auth: allows to automatically check the user password
...
This simplify the common pattern where the user password and a one time
token is requested: now the external program can delegate password check
to SFTPGo and verify the token itself
2020-02-16 11:43:52 +01:00
Nicola Murino
58253968fc
s3: improve credentials validation
...
access secret can now be empty, so check if not empty before encrypting
the secret
2020-02-16 10:14:44 +01:00
Enes Çakır
dbd75209df
s3: add auth options to README
2020-02-16 09:06:25 +01:00
Nicola Murino
0b7be1175d
parse ssh commands with shlex
...
instead of use our bugged home made method.
Fixes #72
2020-02-14 16:17:32 +01:00
Nicola Murino
5b47292366
sftpd: improve error logs
...
Now logs for cloud filesystems are more readable.
Also use standard output as default for debian Dockerfile
2020-02-13 08:26:45 +01:00
Nicola Murino
7bfe0ddf80
ssh commands: fix parsing commands with space
...
For now we support "\" escaping style
2020-02-08 23:33:06 +01:00
Nicola Murino
d6fa853a37
add support for integrated database schema migrations
...
added the "initprovider" command to initialize the database structure.
If we change the database schema the required changes will be checked
at startup and automatically applyed.
2020-02-08 14:44:25 +01:00
Nicola Murino
553cceab42
dataprovider actions: add more users fiels as env vars
2020-02-05 22:17:03 +01:00
Nicola Murino
5bfaae9202
httpd: allow to reload the https certificate without restarting the service
...
HTTPS certificate can be reloaded on demand sending a SIGHUP signal on
Unix based systems and a "paramchange" request to the running service on
Windows
2020-02-04 23:21:33 +01:00
Nicola Murino
8b039e0447
httpd: add support for basic auth and HTTPS
2020-02-04 00:08:00 +01:00
Nicola Murino
bcaf283c35
memory provider: load users from a dump file
...
The `memory` provider can load users from a dump obtained using the
`dumpdata` REST API. This dump file can be configured using the
dataprovider `name` configuration key. It will be loaded at startup
and can be reloaded on demand using a `SIGHUP` on Unix based systems
and a `paramchange` request to the running service on Windows.
Fixes #66
2020-02-02 22:20:39 +01:00
Nicola Murino
3491717c26
add support for serving Google Cloud Storage over SFTP/SCP
...
Each user can be mapped with a Google Cloud Storage bucket or a bucket
virtual folder
2020-01-31 19:04:00 +01:00
thomsh
45a13f5f4e
Update README add section for dataprovider init
2020-01-31 08:56:15 +01:00
Nicola Murino
6884ce3f3e
sftp actions: add a parameter to distinguish local and remote files
2020-01-23 23:23:28 +01:00
Nicola Murino
5f4efc9148
S3: add metrics
2020-01-23 23:17:00 +01:00
Nicola Murino
7ebbbe5c29
S3: update pipeat to the latest commit in my fork
...
Here are the main improvements:
- unliked files works on windows too
- the uploads are now synced on the lower speed between the SFTP client write
and the upload speed to S3
This commit increase the external auth timeout to 60 seconds too
2020-01-22 19:42:23 +01:00
Nicola Murino
9ff303b8c0
add support for keyboard interactive authentication
...
Fixes #64
2020-01-21 10:54:05 +01:00
Nicola Murino
4463421028
S3: add support for serving virtual folders
...
inside the same bucket each user can be assigned to a virtual folder.
This is similar to a chroot directory for local filesystem
2020-01-19 23:23:09 +01:00
Nicola Murino
d75f56b914
vfs: store root dir
...
so we don't need to pass it over and over
2020-01-19 13:58:55 +01:00
Nicola Murino
a4834f4a83
add basic S3-Compatible Object Storage support
...
we have now an interface for filesystem backeds, this make easy to add
new filesystem backends
2020-01-19 07:41:05 +01:00
Nicola Murino
eec60d6309
custom actions: add env vars
...
action parameters can now be readed from env vars too.
Added a timeout for the command execution
2020-01-09 12:00:37 +01:00
Nicola Murino
8e604f888a
improve docs and test cases
2020-01-07 09:39:20 +01:00
Nicola Murino
531091906d
add support for authentication using external programs
...
Fixes #62
2020-01-06 21:42:41 +01:00
Nicola Murino
eb2ddc4798
small README improvements
2020-01-01 23:44:33 +01:00
Nicola Murino
aee9312cea
better document how to reuse existing users
2019-12-31 14:22:42 +01:00
Nicola Murino
6a99a5cb9f
Improve README
2019-12-31 11:11:07 +01:00
RIVIERE Fabien
8e0ca88421
Add systemD/journalD sftpgo Fail2ban configuration
2019-12-31 10:08:51 +01:00
Nicola Murino
1b1c740b29
Add support for allowed/denied IP/Mask
...
Login can be restricted to specific ranges of IP address or to a specific IP
address.
Please apply the appropriate SQL upgrade script to add the filter field to your
database.
The filter database field will allow to add other filters without requiring a
new database migration
2019-12-30 18:37:50 +01:00
Nicola Murino
ad5436e3f6
ssh commands: improve command ended detection
...
Sometime we can have this error:
read |0: file already closed
reading from the command standard error, this means that the command is
already finished so we don't need to do nothing.
This happen randomically while running the test cases on travis.
2019-12-30 00:10:03 +01:00
Nicola Murino
6aff8c2f5e
add support for checking passwords in md5crypt ($1$) format
...
this is an old and unsafe schema but it is still useful to import users
from legacy systems
2019-12-29 07:43:59 +01:00
Nicola Murino
ae094d3479
add backup/restore REST API
2019-12-27 23:12:44 +01:00
Nicola Murino
489101668c
add per directory permissions
...
we can now have permissions such as these ones
{"/":["*"],"/somedir":["list","download"]}
The old permissions are automatically converted to the new structure,
no database migration is needed
2019-12-25 18:20:19 +01:00
Nicola Murino
f8fd5c067c
docker alpine: fix example
...
execute 20191112.sql too
2019-12-06 21:57:29 +01:00
Nicola Murino
80a5138115
add rsync support ...
...
and better document quota management issues for system commands.
rsync and git are not enabled in the default config so don't install
them in sample Dockerfiles, simply add a comment to facilitate their
installation if needed
Fixes #44
2019-11-29 15:24:56 +01:00
Nicola Murino
0a025aabfd
add support for Git over SSH
...
We use the system commands "git-receive-pack", "git-upload-pack" and
"git-upload-archive". they need to be installed and in your system's
PATH. Since we execute system commands we have no direct control on
file creation/deletion and so quota check is suboptimal: if quota is
enabled, the number of files is checked at the command begin and not
while new files are created.
The allowed size is calculated as the difference between the max quota
and the used one. The command is aborted if it uploads more bytes than
the remaining allowed size calculated at the command start. Quotas are
recalculated at the command end with a full home directory scan, this
could be heavy for big directories.
2019-11-26 22:26:42 +01:00
Nicola Murino
7a8b1645ef
set version to 0.9.4
2019-11-22 21:27:56 +01:00
Nicola Murino
b3729e4666
log ssh commands in "command logs" category
2019-11-19 11:38:39 +01:00
Nicola Murino
9c4dbbc3f8
sftpd: add support for some SSH commands
...
md5sum, sha1sum are used by rclone.
cd, pwd improve the support for RemoteFiles mobile app.
These commands are all implemented inside SFTPGo so they work even
if the matching system commands are not available, for example on Windows
2019-11-18 23:30:37 +01:00
Nicola Murino
ca6cb34d98
sftpd: add support for chtimes
...
This improve rclone compatibility
2019-11-16 10:23:41 +01:00
Nicola Murino
fc442d7862
sftpd: document chmod/chown on Windows
...
chmod is partially supported and chown is not supported on Windows.
Skip unsupported test cases on Windows
2019-11-15 17:09:00 +01:00
Nicola Murino
bb37a1c1ce
sftpd: add support for chmod/chown
...
added matching permissions too and a new setting "setstat_mode".
Setting setstat_mode to 1 you can keep the previous behaviour that
silently ignore setstat requests
2019-11-15 12:15:07 +01:00
Nicola Murino
f3de83707f
improve README
2019-11-14 17:43:14 +01:00
Nicola Murino
08e85f6be9
sftpd: return sftp.ErrSSHFxNoSuchFile if the client ask the file for a missing path
...
some clients expected this error and not the generic one if the path is missing
2019-11-14 14:18:43 +01:00
Nicola Murino
acdf351047
dataprovider: add custom command and/or HTTP notifications on users add, update and delete
...
This way custom logic can be implemented for example to create a UNIX user
as asked in #58
2019-11-14 11:06:03 +01:00
Nicola Murino
c2ff50c917
dataprovider: add support for user status and expiration
...
an user can now be disabled or expired.
If you are using an SQL database as dataprovider please remember to
execute the sql update script inside "sql" folder.
Fixes #57
2019-11-13 11:36:21 +01:00
Nicola Murino
74367a65cc
failed connection logs: rename host to client_ip
2019-11-11 19:53:27 +01:00
Nicola Murino
2221d3307a
Improve README
2019-11-11 18:28:18 +01:00
Nicola Murino
4ff34b3e53
logger: add specific logs for failed attempts to initialize a connection
...
This should allow for better integration in tools like fail2ban.
Hopefully fix #59
2019-11-11 15:20:00 +01:00
Nicola Murino
77db2bd3d1
fix posix rename
...
update pkg/sftp to a git revision that includes:
https://github.com/pkg/sftp/pull/316
add a test case here too and update other deps
2019-11-08 08:43:27 +01:00
Nicola Murino
758f2ee834
improve README
2019-10-26 20:42:46 +02:00
Nicola Murino
c5a6ca5650
portable mode: advertise service via multicast dns
...
Fixes #51
2019-10-26 18:25:53 +02:00
Nicola Murino
b409523d5c
document memory data provider
2019-10-25 19:01:01 +02:00
Nicola Murino
a4cddf4f7f
add portable mode
...
Portable mode is a convenient way to share a single directory on demand
2019-10-24 18:50:35 +02:00
Roman Isko
d970e757eb
Use relative to the current branch links in readme
2019-10-22 10:00:09 +02:00
Nicola Murino
083d9f76c6
add commit hash and build date to the version logged when starting the daemon
2019-10-21 21:18:45 +02:00
Nicola Murino
9cf4653425
improve docs for available Arch Linux packages
2019-10-20 09:23:04 +02:00
Nicola Murino
1d917561fe
sftpd: add support for upload resume
...
we support resume only if the client sets the correct offset while resuming
the upload.
Based on the specs the offset is optional for resume, but all the tested
clients sets a right offset.
If an invalid offset is given we interrupt the transfer with the error
"Invalid write offset ..."
See https://github.com/pkg/sftp/issues/295
This commit add a new upload mode: "atomic with resume support", this acts
as atomic but if there is an upload error the temporary file is renamed
to the requested path and not deleted, this way a client can reconnect
and resume the upload
2019-10-09 17:33:30 +02:00
Nicola Murino
60d4a3e1b5
improve docs
2019-10-08 08:20:26 +02:00
Nicola Murino
3e0558c0e9
add web interface support to windows setup ...
...
... and other small improvements
2019-10-07 22:37:28 +02:00
Nicola Murino
afd312f26a
add a basic web interface
...
The builtin web interface allows to manage users and connections
2019-10-07 18:19:01 +02:00
Nicola Murino
00dd5db226
add support for users' default base dir
2019-09-28 22:48:52 +02:00
Nicola Murino
0d8506c93d
set version to 0.9.2
2019-09-18 22:19:34 +02:00
Nicola Murino
df96ea7e9f
add a new permission for overwriting existing files
...
The upload permission is required to allow file overwrite
2019-09-17 08:53:45 +02:00
Nicola Murino
580fae7a8f
minor improvements
2019-09-16 18:11:35 +02:00
Nicola Murino
4a1baaee69
windows service: improve doc
2019-09-16 09:22:27 +02:00
Nicola Murino
f3f38f5f09
add Windows Service support
2019-09-16 08:52:58 +02:00
Nicola Murino
bba78763e1
add a test case and document sha512crypt passwords support
2019-09-15 12:23:27 +02:00
Nicola Murino
018b10808d
improve SQL provider availability check adding a timeout
2019-09-14 16:18:31 +02:00
Jo Vandeginste
29aadbf3e3
log redacted passwords
...
Closes #48
2019-09-13 21:57:52 +02:00
Nicola Murino
7eb5b01169
add Prometheus support
...
some basic counters and gauges are now exposed
2019-09-13 18:45:36 +02:00
Nicola Murino
e7eb3476b7
dataprovider: remove transaction for quota update
...
The update is atomic so no transaction is needed.
Addionally a transaction will ask for a new connection to the pool
and this can deadlock if the pool has a max connection limit too low.
Also make configurable the pool size instead of hard code to the cpu number.
Fixes #47
2019-09-13 08:14:07 +02:00
Nicola Murino
6b33d3d9f1
add version to the default server banner
2019-09-12 09:21:58 +02:00
Nicola Murino
784f75f45b
use net.Conn instead of ssh.Conn to close connections
2019-09-11 09:41:46 +02:00
Nicola Murino
c1effdf701
atomic upload mode: remove temporary file on error
...
If a transfer error is detected, then the partial temporary file will
be removed and not renamed to requested path
2019-09-10 18:47:21 +02:00
Nicola Murino
7010f513e3
macOS: add launchd sample plist
2019-09-10 08:03:53 +02:00
Nicola Murino
e8db7d8539
improve configuration docs
...
viper will not use env vars if no configuration file is found
https://github.com/spf13/viper/issues/584
As workaround we could manually bind/set a default for each configuration
option using viper.SetDefault("key") and then generate a default config
using viper.Get("key").
This manual solution is error prone and it will become increasingly
difficult to maintain since the configuration options will grow, so
we avoid it for now.
Let's see if viper will solve this issue
Fixes #35
2019-09-04 09:09:17 +02:00
Nicola Murino
3441b75a58
allow empty log file, use the standard output in this case
...
Fixes #34
2019-09-03 23:13:33 +02:00
Nicola Murino
49a40f7a0b
sftpd: add configuration options for allowed ciphers, MACs and KEX algorithms
...
add support for login banner too
Fixes #32
2019-09-03 12:08:09 +02:00
Nicola Murino
dc5eeb54fd
scp: fix quota update after file overwrite
...
added a test case too
2019-09-02 23:12:41 +02:00
Nicola Murino
4a1b67454e
set version to 0.9.1
2019-09-02 09:36:02 +02:00
Nicola Murino
838426b3a4
improve docs
2019-08-29 13:52:22 +02:00
Nicola Murino
0e6113eef6
remove uneeded deps using go mod tidy
2019-08-27 15:59:00 +02:00
Nicola Murino
d4969cde03
scp: add test case and document remote to remote transfers
2019-08-25 13:51:54 +02:00