beenull/sftpgo-mirror
1
0
Fork 0
mirror of https://github.com/drakkan/sftpgo.git synced 2024-11-26 01:20:29 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 6
sftpgo-mirror/docs/s3.md
Nicola Murino 5ff8f75917
add Azure Blob support
2020-10-25 08:18:48 +01:00

2.9 KiB
Raw Blame History

S3 Compatible Object Storage backends

To connect SFTPGo to AWS, you need to specify credentials, a bucket and a region. Here is the list of available AWS regions. For example, if your bucket is at Frankfurt, you have to set the region to eu-central-1. You can specify an AWS storage class too. Leave it blank to use the default AWS storage class. An endpoint is required if you are connecting to a Compatible AWS Storage such as MinIO.

AWS SDK has different options for credentials. More Detail. We support:

  1. Providing Access Keys.
  2. Use IAM roles for Amazon EC2
  3. Use IAM roles for tasks if your application uses an ECS task definition

So, you need to provide access keys to activate option 1, or leave them blank to use the other ways to specify credentials.

Specifying a different key_prefix, you can assign different "folders" of the same bucket to different users. This is similar to a chroot directory for local filesystem. Each SFTP/SCP user can only access the assigned folder and its contents. The folder identified by key_prefix does not need to be pre-created.

SFTPGo uses multipart uploads and parallel downloads for storing and retrieving files from S3.

For multipart uploads you can customize the parts size and the upload concurrency. Please note that if the upload bandwidth between the client and SFTPGo is greater than the upload bandwidth between SFTPGo and S3 then the client should wait for the last parts to be uploaded to S3 after finishing uploading the file to SFTPGo, and it may time out. Keep this in mind if you customize these parameters.

The configured bucket must exist.

Some SFTP commands don't work over S3:

  • symlink and chtimes will fail
  • chown and chmod are silently ignored
  • truncate is not supported
  • opening a file for both reading and writing at the same time is not supported
  • upload resume is not supported
  • upload mode atomic is ignored since S3 uploads are already atomic

Other notes:

  • rename is a two step operation: server-side copy and then deletion. So, it is not atomic as for local filesystem.
  • We don't support renaming non empty directories since we should rename all the contents too and this could take a long time: think about directories with thousands of files: for each file we should do an AWS API call.
  • For server side encryption, you have to configure the mapped bucket to automatically encrypt objects.
  • A local home directory is still required to store temporary files.
  • Clients that require advanced filesystem-like features such as sshfs are not supported.