sftpgo-mirror/README.md

200 lines
12 KiB
Markdown
Raw Normal View History

2019-07-20 10:26:52 +00:00
# SFTPGo
2020-03-04 22:10:58 +00:00
[![Build Status](https://travis-ci.org/drakkan/sftpgo.svg?branch=master)](https://travis-ci.org/drakkan/sftpgo) [![Code Coverage](https://codecov.io/gh/drakkan/sftpgo/branch/master/graph/badge.svg)](https://codecov.io/gh/drakkan/sftpgo/branch/master) [![Go Report Card](https://goreportcard.com/badge/github.com/drakkan/sftpgo)](https://goreportcard.com/report/github.com/drakkan/sftpgo) [![License: GPL v3](https://img.shields.io/badge/License-GPLv3-blue.svg)](https://www.gnu.org/licenses/gpl-3.0) [![Mentioned in Awesome Go](https://awesome.re/mentioned-badge.svg)](https://github.com/avelino/awesome-go)
2019-07-20 10:26:52 +00:00
2020-03-04 22:10:58 +00:00
Fully featured and highly configurable SFTP server, written in Go
2019-07-20 10:26:52 +00:00
## Features
2020-03-04 22:10:58 +00:00
- Each account is chrooted to its home directory.
2019-08-29 11:52:22 +00:00
- SFTP accounts are virtual accounts stored in a "data provider".
2020-03-04 05:28:59 +00:00
- SQLite, MySQL, PostgreSQL, bbolt (key/value store in pure Go) and in-memory data providers are supported.
2019-08-29 11:52:22 +00:00
- Public key and password authentication. Multiple public keys per user are supported.
- SSH user [certificate authentication](https://cvsweb.openbsd.org/src/usr.bin/ssh/PROTOCOL.certkeys?rev=1.8).
2020-02-25 11:18:45 +00:00
- Keyboard interactive authentication. You can easily setup a customizable multi-factor authentication.
- Partial authentication. You can configure multi-step authentication requiring, for example, the user password after successful public key authentication.
2020-05-23 10:47:44 +00:00
- Per user authentication methods. You can configure the allowed authentication methods for each user.
2020-04-30 12:23:55 +00:00
- Custom authentication via external programs/HTTP API is supported.
- Dynamic user modification before login via external programs/HTTP API is supported.
2019-09-02 07:36:02 +00:00
- Quota support: accounts can have individual quota expressed as max total size and/or max number of files.
2019-08-29 11:52:22 +00:00
- Bandwidth throttling is supported, with distinct settings for upload and download.
- Per user maximum concurrent sessions.
2020-02-25 11:18:45 +00:00
- Per user and per directory permission management: list directory contents, upload, overwrite, download, delete, rename, create directories, create symlinks, change owner/group and mode, change access and modification times.
2020-03-04 22:10:58 +00:00
- Per user files/folders ownership mapping: you can map all the users to the system account that runs SFTPGo (all platforms are supported) or you can run SFTPGo as root user and map each user or group of users to a different system account (\*NIX only).
- Per user IP filters are supported: login can be restricted to specific ranges of IP addresses or to a specific IP address.
- Per user and per directory file extensions filters are supported: files can be allowed or denied based on their extensions.
- Virtual folders are supported: directories outside the user home directory can be exposed as virtual folders.
- Configurable custom commands and/or HTTP notifications on file upload, download, pre-delete, delete, rename, on SSH commands and on user add, update and delete.
2019-08-29 11:52:22 +00:00
- Automatically terminating idle connections.
- Atomic uploads are configurable.
- Support for Git repositories over SSH.
2019-12-31 10:11:07 +00:00
- SCP and rsync are supported.
- Support for serving local filesystem, S3 Compatible Object Storage and Google Cloud Storage over SFTP/SCP.
2020-03-04 22:10:58 +00:00
- [Prometheus metrics](./docs/metrics.md) are exposed.
- Support for HAProxy PROXY protocol: you can proxy and/or load balance the SFTP/SCP service without losing the information about the client's address.
- [REST API](./docs/rest-api.md) for users and folders management, backup, restore and real time reports of the active connections with possibility of forcibly closing a connection.
- [Web based administration interface](./docs/web-admin.md) to easily manage users, folders and connections.
- Easy [migration](./examples/rest-api-cli#convert-users-from-other-stores) from Linux system user accounts.
2020-03-04 22:10:58 +00:00
- [Portable mode](./docs/portable-mode.md): a convenient way to share a single directory on demand.
2020-03-15 22:33:12 +00:00
- Performance analysis using built-in [profiler](./docs/profiling.md).
2020-02-25 11:18:45 +00:00
- Configuration format is at your choice: JSON, TOML, YAML, HCL, envfile are supported.
2020-03-04 22:10:58 +00:00
- Log files are accurate and they are saved in the easily parsable JSON format ([more information](./docs/logs.md)).
2019-07-20 10:26:52 +00:00
## Platforms
2020-03-04 05:28:59 +00:00
SFTPGo is developed and tested on Linux. After each commit, the code is automatically built and tested on Linux and macOS using Travis CI.
2020-03-04 22:10:58 +00:00
The test cases are regularly manually executed and passed on Windows. Other UNIX variants such as \*BSD should work too.
2019-07-20 10:26:52 +00:00
## Requirements
- Go 1.13 or higher as build only dependency.
2020-03-04 05:28:59 +00:00
- A suitable SQL server or key/value store to use as data provider: PostgreSQL 9.4+ or MySQL 5.6+ or SQLite 3.x or bbolt 1.3.x
2019-07-20 10:26:52 +00:00
## Installation
2020-03-04 22:10:58 +00:00
Binary releases for Linux, macOS, and Windows are available. Please visit the [releases](https://github.com/drakkan/sftpgo/releases "releases") page.
2019-12-27 22:12:44 +00:00
2020-05-23 10:47:44 +00:00
Sample Dockerfiles for [Debian](https://www.debian.org) and [Alpine](https://alpinelinux.org) are available inside the source tree [docker](./docker) directory.
2019-12-27 22:12:44 +00:00
2020-01-01 22:44:33 +00:00
Some Linux distro packages are available:
- For Arch Linux via AUR:
2020-03-05 08:32:29 +00:00
- [sftpgo](https://aur.archlinux.org/packages/sftpgo/). This package follows stable releases. It requires `git`, `gcc` and `go` to build.
- [sftpgo-bin](https://aur.archlinux.org/packages/sftpgo-bin/). This package follows stable releases downloading the prebuilt linux binary from GitHub. It does not require `git`, `gcc` and `go` to build.
- [sftpgo-git](https://aur.archlinux.org/packages/sftpgo-git/). This package builds and installs the latest git master. It requires `git`, `gcc` and `go` to build.
2020-01-01 22:44:33 +00:00
2020-03-04 22:10:58 +00:00
Alternately, you can [build from source](./docs/build-from-source.md).
2019-09-16 06:52:58 +00:00
2019-07-20 10:26:52 +00:00
## Configuration
2020-03-04 22:10:58 +00:00
A full explanation of all configuration methods can be found [here](./docs/full-configuration.md).
2019-09-16 07:22:27 +00:00
2020-03-04 22:10:58 +00:00
Please make sure to [initialize the data provider](#data-provider-initialization) before running the daemon!
2020-03-04 22:10:58 +00:00
To start the SFTP server with default settings, simply run:
2019-07-20 10:26:52 +00:00
2020-03-04 22:10:58 +00:00
```bash
sftpgo serve
2019-08-01 13:39:30 +00:00
```
2020-03-04 22:10:58 +00:00
Check out [this documentation](./docs/service.md) if you want to run SFTPGo as a service.
### Data provider initialization
2020-03-04 22:10:58 +00:00
Before starting the SFTPGo server, please ensure that the configured data provider is properly initialized.
2020-03-04 05:28:59 +00:00
SQL based data providers (SQLite, MySQL, PostgreSQL) require the creation of a database containing the required tables. Memory and bolt data providers do not require an initialization.
2020-03-04 05:28:59 +00:00
After configuring the data provider using the configuration file, you can create the required database structure using the `initprovider` command.
2020-03-03 22:25:23 +00:00
For SQLite provider, the `initprovider` command will auto create the database file, if missing, and the required tables.
2020-03-04 05:28:59 +00:00
For PostgreSQL and MySQL providers, you need to create the configured database, and the `initprovider` command will create the required tables.
2020-03-04 05:28:59 +00:00
For example, you can simply execute the following command from the configuration directory:
2020-03-04 22:10:58 +00:00
```bash
sftpgo initprovider
```
Take a look at the CLI usage to learn how to specify a different configuration file:
2020-03-04 22:10:58 +00:00
```bash
sftpgo initprovider --help
```
The `initprovider` command is enough for new installations. From now on, the database structure will be automatically checked and updated, if required, at startup.
2020-03-04 22:10:58 +00:00
#### Upgrading
2020-03-04 05:28:59 +00:00
If you are upgrading from version 0.9.5 or before, you have to manually execute the SQL scripts to create the required database structure. These scripts can be found inside the source tree [sql](./sql "sql") directory. The SQL scripts filename is, by convention, the date as `YYYYMMDD` and the suffix `.sql`. You need to apply all the SQL scripts for your database ordered by name. For example, `20190828.sql` must be applied before `20191112.sql`, and so on.
Example for SQLite: `find sql/sqlite/ -type f -iname '*.sql' -print | sort -n | xargs cat | sqlite3 sftpgo.db`.
2020-03-04 05:28:59 +00:00
After applying these scripts, your database structure is the same as the one obtained using `initprovider` for new installations, so from now on, you don't have to manually upgrade your database anymore.
2020-03-04 22:10:58 +00:00
## Authentication options
2020-03-04 22:10:58 +00:00
### External Authentication
2020-03-04 22:10:58 +00:00
Custom authentication methods can easily be added. SFTPGo supports external authentication modules, and writing a new backend can be as simple as a few lines of shell script. More information can be found [here](./docs/external-auth.md).
2020-01-07 08:39:20 +00:00
2020-03-04 22:10:58 +00:00
### Keyboard Interactive Authentication
2020-03-04 05:28:59 +00:00
Keyboard interactive authentication is, in general, a series of questions asked by the server with responses provided by the client.
This authentication method is typically used for multi-factor authentication.
2020-03-04 22:10:58 +00:00
More information can be found [here](./docs/keyboard-interactive.md).
## Dynamic user creation or modification
A user can be created or modified by an external program just before the login. More information about this can be found [here](./docs/dynamic-user-mod.md).
## Custom Actions
2020-05-23 10:47:44 +00:00
SFTPGo allows to configure custom commands and/or HTTP notifications on file upload, download, delete, rename, on SSH commands and on user add, update and delete.
2020-03-04 22:10:58 +00:00
More information about custom actions can be found [here](./docs/custom-actions.md).
## Virtual folders
Directories outside the user home directory can be exposed as virtual folders, more information [here](./docs/virtual-folders.md).
2020-03-04 22:10:58 +00:00
## Storage backends
2020-03-04 22:10:58 +00:00
### S3 Compabible Object Storage backends
2020-03-04 22:10:58 +00:00
Each user can be mapped to whole bucket or to a bucket virtual folder. This way, the mapped bucket/virtual folder is exposed over SFTP/SCP. More information about S3 integration can be found [here](./docs/s3.md).
2020-03-04 22:10:58 +00:00
### Google Cloud Storage backend
2020-03-04 22:51:16 +00:00
Each user can be mapped with a Google Cloud Storage bucket or a bucket virtual folder. This way, the mapped bucket/virtual folder is exposed over SFTP/SCP. More information about Google Cloud Storage integration can be found [here](./docs/google-cloud-storage.md).
2020-03-04 22:10:58 +00:00
### Other Storage backends
2020-03-04 05:28:59 +00:00
Adding new storage backends is quite easy:
- implement the [Fs interface](./vfs/vfs.go#L18 "interface for filesystem backends").
- update the user method `GetFilesystem` to return the new backend
- update the web interface and the REST API CLI
- add the flags for the new storage backed to the `portable` mode
2020-03-04 05:28:59 +00:00
Anyway, some backends require a pay per use account (or they offer free account for a limited time period only). To be able to add support for such backends or to review pull requests, please provide a test account. The test account must be available for enough time to be able to maintain the backend and do basic tests before each new release.
2020-03-04 22:10:58 +00:00
## Brute force protection
2020-03-04 23:01:40 +00:00
The [connection failed logs](./docs/logs.md) can be used for integration in tools such as [Fail2ban](http://www.fail2ban.org/). Example of [jails](./fail2ban/jails) and [filters](./fail2ban/filters) working with `systemd`/`journald` are available in fail2ban directory.
2019-07-20 10:26:52 +00:00
## Account's configuration properties
2020-03-04 22:10:58 +00:00
Details information about account configuration properties can be found [here](./docs/account.md).
## Performance
SFTPGo can easily saturate a Gigabit connection on low end hardware with no special configuration, this is generally enough for most use cases.
2020-03-04 22:10:58 +00:00
More in-depth analysis of performance can be found [here](./docs/performance.md).
2019-07-20 10:26:52 +00:00
## Acknowledgements
- [pkg/sftp](https://github.com/pkg/sftp)
- [go-chi](https://github.com/go-chi/chi)
- [zerolog](https://github.com/rs/zerolog)
- [lumberjack](https://gopkg.in/natefinch/lumberjack.v2)
- [argon2id](https://github.com/alexedwards/argon2id)
- [go-sqlite3](https://github.com/mattn/go-sqlite3)
- [go-sql-driver/mysql](https://github.com/go-sql-driver/mysql)
- [bbolt](https://github.com/etcd-io/bbolt)
2019-07-20 10:26:52 +00:00
- [lib/pq](https://github.com/lib/pq)
- [viper](https://github.com/spf13/viper)
- [cobra](https://github.com/spf13/cobra)
- [xid](https://github.com/rs/xid)
- [nathanaelle/password](https://github.com/nathanaelle/password)
- [PipeAt](https://github.com/eikenb/pipeat)
- [ZeroConf](https://github.com/grandcat/zeroconf)
- [SB Admin 2](https://github.com/BlackrockDigital/startbootstrap-sb-admin-2)
- [shlex](https://github.com/google/shlex)
- [go-proxyproto](https://github.com/pires/go-proxyproto)
2019-07-20 10:26:52 +00:00
Some code was initially taken from [Pterodactyl sftp server](https://github.com/pterodactyl/sftp-server)
## License
2019-09-02 07:36:02 +00:00
GNU GPLv3