beenull/sftpgo
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
1098 commits 8 branches 48 tags 35 MiB
Commit graph

142 commits

Author SHA1 Message Date
Nicola Murino
5eabaf98e0
gcs: remove a superfluous debug log 2020-09-29 09:17:08 +02:00
Nicola Murino
d1f0e9ae9f
CGS: implement MimeTyper interface 2020-09-28 22:12:46 +02:00
Thomas Blommaert
cd56039ab7
GCS mime-type detection (#179) 
Fixes #178
 2020-09-28 21:52:18 +02:00
Nicola Murino
55515fee95
update deps, GCS can no finally use attribute selection 
See https://github.com/googleapis/google-cloud-go/pull/2661
 2020-09-28 12:51:19 +02:00
Nicola Murino
bf708cb8bc osfs: improve isSubDir check 2020-09-21 19:32:33 +02:00
Nicola Murino
6c1a7449fe ssh commands: return better error messages 
This improve the fix for #171 and return better error message for
SSH commands other than SCP too
 2020-09-19 10:14:30 +02:00
Nicola Murino
209badf10c scp: return better error messages 
Fixes #171
 2020-09-18 19:13:09 +02:00
Nicola Murino
02e35ee002 sftpd: add Readlink support 2020-08-22 14:52:17 +02:00
Nicola Murino
f41ce6619f sftpd: add SSH_FXP_FSETSTAT support 
This change will fix file editing from sshfs, we need this patch

https://github.com/pkg/sftp/pull/373

for pkg/sftp to support this feature
 2020-08-20 13:54:36 +02:00
Nicola Murino
a9e21c282a add WebDAV support 
Fixes #147
 2020-08-11 23:56:10 +02:00
Nicola Murino
fa41bfd06a Cloud backends: add support for FTP REST command 
So partial downloads are now supported as for local fs
 2020-08-03 18:03:09 +02:00
Nicola Murino
2a9f7db1e2 Cloud FS: don't propagate the error if removing a folder returns not found 
for Cloud FS the folders are virtual and they, generally, disappear when the
last file is removed.

This fix doesn't work for FTP protocol for now.

Fixes #149
 2020-07-31 19:24:57 +02:00
Nicola Murino
93ce96d011 add support for the venerable FTP protocol 
Fixes #46
 2020-07-29 21:56:56 +02:00
Nicola Murino
4e41a5583d refactoring: add common package 
The common package defines the interfaces that a protocol must implement
and contain code that can be shared among supported protocols.

This way should be easier to support new protocols
 2020-07-24 23:39:38 +02:00
Nicola Murino
cf541d62ea recursive permissions check before renaming/copying directories 2020-06-26 23:38:29 +02:00
Nicola Murino
23a80b01b6 add build tag to disable metrics 2020-06-19 17:08:51 +02:00
Nicola Murino
e86089a9f3 quota: improve size check 
get the remaining allowed size when an upload starts and check it against the
uploaded bytes

Fixes #128
 2020-06-18 22:38:03 +02:00
Nicola Murino
3ceba7a147 sftpgo-copy: add quota limits check 2020-06-16 22:49:18 +02:00
Nicola Murino
7807fa7cc2 use os.ModePerm for files and directory creation 2020-06-08 19:40:17 +02:00
Nicola Murino
8306b6bde6 refactor virtual folders 
The same virtual folder can now be shared among users and different
folder quota limits for each user are supported.

Fixes #120
 2020-06-07 23:30:18 +02:00
Nicola Murino
ad53429cf1 add support for build tag to allow to disable some features 
The following build tags are available:

- "nogcs", disable Google Cloud Storage backend
- "nos3", disable S3 Compabible Object Storage backends
- "nobolt", disable Bolt data provider
- "nomysql", disable MySQL data provider
- "nopgsql", disable PostgreSQL data provider
- "nosqlite", disable SQLite data provider
- "noportable", disable portable mode
 2020-05-23 11:58:05 +02:00
Nicola Murino
cfa710037c cloud backends: fix SFTP error message for some write failures 
Fixes #119

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
 2020-05-19 19:17:43 +02:00
Nicola Murino
f02e24437a add more linters 
test cases migration to testify is now complete.
Linters are enabled for test cases too
 2020-05-06 19:36:34 +02:00
Nicola Murino
f369fdf6f2 httpclient: add a configuration parameter to skip TLS certificate validation 
In this mode, TLS is susceptible to man-in-the-middle attacks.
This should be used only for testing.
 2020-05-03 11:37:50 +02:00
Nicola Murino
b006c5f914 NewOsFs: return an interface and not a pointer 2020-05-02 15:01:56 +02:00
Nicola Murino
3f75d46a16 sftpd: add support for excluding virtual folders from user quota limit 
Fixes #110
 2020-05-01 15:27:53 +02:00
Nicola Murino
d70959c34c fix some lint issues 2020-04-30 14:23:55 +02:00
Nicola Murino
075bbe2aef added test case that checks quota for files inside virtual folders 2020-03-29 11:10:03 +02:00
Nicola Murino
1770da545d s3: upload concurrency is now configurable 
Please note that if the upload bandwidth between the SFTP client and
SFTPGo is greater than the upload bandwidth between SFTPGo and S3 then
the SFTP client have to wait for the upload of the last parts to S3
after it ends the file upload to SFTPGo, and it may time out.
Keep this in mind if you customize parts size and upload concurrency
 2020-03-13 19:13:58 +01:00
Nicola Murino
de3e69f846 s3: add documentation and test cases for upload part size 2020-03-13 17:28:55 +01:00
Michael Bonfils
cdf1233065 s3: export PartSize parameter 
By default AWS SDK use part_size of 5 MB. For big files,
it is not ideal case. For Hadoop, it is not uncommon to
use 512 MB.
 2020-03-13 17:28:04 +01:00
Nicola Murino
45b9366dd0 add support for virtual folders 
directories outside the user home directory can be exposed as virtual folders
 2020-02-23 11:30:26 +01:00
Nicola Murino
ae8ed75ae5 gcs: add support for automatic credentials 
We can now also support implicit credentials using the Application
Default Credentials strategy
 2020-02-19 09:41:15 +01:00
Nicola Murino
58253968fc s3: improve credentials validation 
access secret can now be empty, so check if not empty before encrypting
the secret
 2020-02-16 10:14:44 +01:00
Enes Çakır
da01848855 s3: enable shared config state 2020-02-16 09:06:25 +01:00
Nicola Murino
9359669cd4 s3: set upload concurrency to 2 
we upload a file while receiving it via SFTP not a file stored on a local
disk. We use concurrent uploads only to be able to send files of arbitrary
size, so concurrency is not really useful here. Setting the concurrency to
2 we have a max difference of 10 MB between the writer (sftp client) and
the reader (aws sdk), with the default concurrency value this difference
is 25MB.
 2020-02-04 23:14:55 +01:00
Nicola Murino
8b039e0447 httpd: add support for basic auth and HTTPS 2020-02-04 00:08:00 +01:00
Nicola Murino
3491717c26 add support for serving Google Cloud Storage over SFTP/SCP 
Each user can be mapped with a Google Cloud Storage bucket or a bucket
virtual folder
 2020-01-31 19:04:00 +01:00
Nicola Murino
5f4efc9148 S3: add metrics 2020-01-23 23:17:00 +01:00
Nicola Murino
4463421028 S3: add support for serving virtual folders 
inside the same bucket each user can be assigned to a virtual folder.
This is similar to a chroot directory for local filesystem
 2020-01-19 23:23:09 +01:00
Nicola Murino
d75f56b914 vfs: store root dir 
so we don't need to pass it over and over
 2020-01-19 13:58:55 +01:00
Nicola Murino
a4834f4a83 add basic S3-Compatible Object Storage support 
we have now an interface for filesystem backeds, this make easy to add
new filesystem backends
 2020-01-19 07:41:05 +01:00