Nicola Murino
1ac66d27b6
Use IEC units for byte counting everywhere
2021-02-12 22:16:35 +01:00
Nicola Murino
51f110bc7b
sftpd: add statvfs@openssh.com support
2021-02-11 19:45:52 +01:00
Nicola Murino
70bb3c34ce
sftpfs: improve endpoint validation
...
Validation will fail if the endpoint is not specified as host:port
2021-02-03 11:29:04 +01:00
Nicola Murino
78bf808322
virtual folders: change dataprovider structure
...
This way we no longer depend on the local file system path and so we can
add support for cloud backends in future updates
2021-02-01 19:04:15 +01:00
Nicola Murino
ea096db8e4
sftpfs: set the correct file mode
2021-01-23 10:32:15 +01:00
Nicola Murino
0caeb68680
sftpfs: fix stat info
2021-01-23 09:42:49 +01:00
Giorgio Pellero
d42fcc3786
s3: don't paginate to find zero-byte-keyed dirs ( #277 )
...
Fixes #275
2021-01-14 12:01:25 +01:00
Nicola Murino
5d4f758c47
GCS: don't paginate to find compat "dirs"
2021-01-12 19:22:12 +01:00
Nicola Murino
2a95d031ea
FTP: add support for AVBL command
2020-12-25 11:14:08 +01:00
Jochen Munz
5b1d8666b3
S3fs: Handle non-ascii filename in rename operations ( #257 )
...
SFTP is based on UTF-8 filenames, so non-ASCII filenames get transported with utf-8 escaped character sequences.
At least for the S3fs provider, if such a file is stored in a nested path it cannot be used as the source for a rename operations.
This adds the necessary escaping of the path fragments.
The patch is not required for MinIO but it doesn't hurt
2020-12-24 11:13:42 +01:00
Nicola Murino
7ab7941ddd
sftpfs: fix race condition
2020-12-23 17:15:55 +01:00
Nicola Murino
f876c728ad
add support for the latest ftpserverlib and azblob versions
2020-12-17 13:40:36 +01:00
Nicola Murino
a6985075b9
add sftpfs storage backend
...
Fixes #224
2020-12-12 10:31:09 +01:00
Nicola Murino
034d89876d
webdav: fix proppatch handling
...
also respect login delay for cached webdav users and check the home dir as
soon as the user authenticates
Fixes #239
2020-12-06 08:19:41 +01:00
Nicola Murino
4a88ea5c03
add Data At Rest Encryption support
2020-12-05 13:48:13 +01:00
Nicola Murino
634b723b5d
add KMS support
...
Fixes #226
2020-11-30 21:46:34 +01:00
Nicola Murino
99cd1ccfe5
S3: fix empty directory detection
...
when listing empty directory MinIO returns no contents while S3 returns
1 object with the key equal to the prefix. Make detection work in both
cases
Fixes #227
2020-11-23 15:36:42 +01:00
Nicola Murino
dccc583b5d
add a dedicated struct to store encrypted credentials
...
also gcs credentials are now encrypted, both on disk and inside the
provider.
Data provider is automatically migrated and load data will accept
old format too but you should upgrade to the new format to avoid future
issues
2020-11-22 21:53:04 +01:00
Nicola Murino
ca0ff0d630
add a File interface so we can avoid to use os.File directly
2020-11-17 19:36:39 +01:00
Nicola Murino
1a6863f4b1
GCS uploads: check Close() error
...
some code simplification too
2020-11-13 18:40:18 +01:00
Nicola Murino
5720d40fee
add setstat_mode 2
...
in this mode chmod/chtimes/chown can be silently ignored only for cloud
based file systems
Fixes #223
2020-11-12 10:39:46 +01:00
Nicola Murino
c6ab6f94e7
azblob: container level SAS cannot access container properties
...
so return the root directory without checking if the bucket exists
2020-11-05 15:03:35 +01:00
Nicola Murino
0a14297b48
webdav: performance improvements and bug fixes
...
we need my custom golang/x/net/webdav fork for now
https://github.com/drakkan/net/tree/sftpgo
2020-11-04 19:11:40 +01:00
Nicola Murino
6ad4cc317c
cloud backends: stat and other performance improvements
2020-11-02 19:16:12 +01:00
Nicola Murino
fcfdd633f6
Azure Blob: update SDK and add access tier support
2020-10-30 22:17:17 +01:00
Nicola Murino
e54828a7b8
add metrics for Azure Blob storage
2020-10-26 19:01:17 +01:00
Nicola Murino
f2acde789d
portable mode: add Azure Blob support
2020-10-25 21:42:43 +01:00
Nicola Murino
9b49f63a97
azure: implement multipart uploads using low level API
...
The high level wrapper seems to hang if there are network issues
2020-10-25 17:41:04 +01:00
Nicola Murino
14bcc6f2fc
s3, azblob: check upper limit for part size
2020-10-25 12:10:11 +01:00
Nicola Murino
5ff8f75917
add Azure Blob support
2020-10-25 08:18:48 +01:00
Sean Hildebrand
db7e81e9d0
add prefer_database_credentials configuration parameter
...
When true, users' Google Cloud Storage credentials will be written to
the data provider instead of disk.
Pre-existing credentials on disk will be used as a fallback
Fixes #201
2020-10-22 10:42:40 +02:00
Nicola Murino
5eabaf98e0
gcs: remove a superfluous debug log
2020-09-29 09:17:08 +02:00
Nicola Murino
d1f0e9ae9f
CGS: implement MimeTyper interface
2020-09-28 22:12:46 +02:00
Thomas Blommaert
cd56039ab7
GCS mime-type detection ( #179 )
...
Fixes #178
2020-09-28 21:52:18 +02:00
Nicola Murino
55515fee95
update deps, GCS can no finally use attribute selection
...
See https://github.com/googleapis/google-cloud-go/pull/2661
2020-09-28 12:51:19 +02:00
Nicola Murino
bf708cb8bc
osfs: improve isSubDir check
2020-09-21 19:32:33 +02:00
Nicola Murino
6c1a7449fe
ssh commands: return better error messages
...
This improve the fix for #171 and return better error message for
SSH commands other than SCP too
2020-09-19 10:14:30 +02:00
Nicola Murino
209badf10c
scp: return better error messages
...
Fixes #171
2020-09-18 19:13:09 +02:00
Nicola Murino
02e35ee002
sftpd: add Readlink support
2020-08-22 14:52:17 +02:00
Nicola Murino
f41ce6619f
sftpd: add SSH_FXP_FSETSTAT support
...
This change will fix file editing from sshfs, we need this patch
https://github.com/pkg/sftp/pull/373
for pkg/sftp to support this feature
2020-08-20 13:54:36 +02:00
Nicola Murino
a9e21c282a
add WebDAV support
...
Fixes #147
2020-08-11 23:56:10 +02:00
Nicola Murino
fa41bfd06a
Cloud backends: add support for FTP REST command
...
So partial downloads are now supported as for local fs
2020-08-03 18:03:09 +02:00
Nicola Murino
2a9f7db1e2
Cloud FS: don't propagate the error if removing a folder returns not found
...
for Cloud FS the folders are virtual and they, generally, disappear when the
last file is removed.
This fix doesn't work for FTP protocol for now.
Fixes #149
2020-07-31 19:24:57 +02:00
Nicola Murino
93ce96d011
add support for the venerable FTP protocol
...
Fixes #46
2020-07-29 21:56:56 +02:00
Nicola Murino
4e41a5583d
refactoring: add common package
...
The common package defines the interfaces that a protocol must implement
and contain code that can be shared among supported protocols.
This way should be easier to support new protocols
2020-07-24 23:39:38 +02:00
Nicola Murino
cf541d62ea
recursive permissions check before renaming/copying directories
2020-06-26 23:38:29 +02:00
Nicola Murino
23a80b01b6
add build tag to disable metrics
2020-06-19 17:08:51 +02:00
Nicola Murino
e86089a9f3
quota: improve size check
...
get the remaining allowed size when an upload starts and check it against the
uploaded bytes
Fixes #128
2020-06-18 22:38:03 +02:00
Nicola Murino
3ceba7a147
sftpgo-copy: add quota limits check
2020-06-16 22:49:18 +02:00
Nicola Murino
7807fa7cc2
use os.ModePerm for files and directory creation
2020-06-08 19:40:17 +02:00