Add API endpoint to set current quota

Fixes #130
This commit is contained in:
Nicola Murino 2020-06-20 12:38:04 +02:00
parent 23a80b01b6
commit 8cb47817f6
16 changed files with 746 additions and 143 deletions

View file

@ -1,4 +1,4 @@
## Dockerfile examples # Dockerfile examples
Sample Dockerfiles for `sftpgo` daemon and the REST API CLI. Sample Dockerfiles for `sftpgo` daemon and the REST API CLI.

View file

@ -2,8 +2,10 @@
This DockerFile is made to build image to host multiple instances of SFTPGo started with different users. This DockerFile is made to build image to host multiple instances of SFTPGo started with different users.
### Example ## Example
> 1003 is a custom uid:gid for this instance of SFTPGo > 1003 is a custom uid:gid for this instance of SFTPGo
```bash ```bash
# Prereq on docker host # Prereq on docker host
sudo groupadd -g 1003 sftpgrp && \ sudo groupadd -g 1003 sftpgrp && \
@ -48,7 +50,8 @@ The script `entrypoint.sh` makes sure to correct the permissions of directories
Several images can be run with different parameters. Several images can be run with different parameters.
### Custom systemd script ## Custom systemd script
An example of systemd script is present [here](sftpgo.service), with `Environment` parameter to set `PUID` and `GUID` An example of systemd script is present [here](sftpgo.service), with `Environment` parameter to set `PUID` and `GUID`
`WorkingDirectory` parameter must be exist with one file in this directory like `sftpgo-${PUID}.env` corresponding to the variable file for SFTPGo instance. `WorkingDirectory` parameter must be exist with one file in this directory like `sftpgo-${PUID}.env` corresponding to the variable file for SFTPGo instance.

View file

@ -1,4 +1,4 @@
## Dockerfile based on Debian stable # Dockerfile based on Debian stable
Please read the comments inside the `Dockerfile` to learn how to customize things for your setup. Please read the comments inside the `Dockerfile` to learn how to customize things for your setup.

View file

@ -1,4 +1,4 @@
## LDAPAuth # LDAPAuth
This is an example for an external authentication program. It performs authentication against an LDAP server. This is an example for an external authentication program. It performs authentication against an LDAP server.
It is tested against [389ds](https://directory.fedoraproject.org/) and can be used as starting point to authenticate using any LDAP server including Active Directory. It is tested against [389ds](https://directory.fedoraproject.org/) and can be used as starting point to authenticate using any LDAP server including Active Directory.
@ -6,13 +6,13 @@ It is tested against [389ds](https://directory.fedoraproject.org/) and can be us
You need to change the LDAP connection parameters and the user search query to match your environment. You need to change the LDAP connection parameters and the user search query to match your environment.
You can build this example using the following command: You can build this example using the following command:
``` ```console
go build -i -ldflags "-s -w" -o ldapauth go build -i -ldflags "-s -w" -o ldapauth
``` ```
This program assumes that the 389ds schema was extended to add support for public keys using the following ldif file placed in `/etc/dirsrv/schema/98openssh-ldap.ldif`: This program assumes that the 389ds schema was extended to add support for public keys using the following ldif file placed in `/etc/dirsrv/schema/98openssh-ldap.ldif`:
``` ```console
dn: cn=schema dn: cn=schema
changetype: modify changetype: modify
add: attributetypes add: attributetypes

View file

@ -1,4 +1,4 @@
## LDAPAuthServer # LDAPAuthServer
This is an example for an HTTP server to use as external authentication HTTP hook. It performs authentication against an LDAP server. This is an example for an HTTP server to use as external authentication HTTP hook. It performs authentication against an LDAP server.
It is tested against [389ds](https://directory.fedoraproject.org/) and can be used as starting point to authenticate using any LDAP server including Active Directory. It is tested against [389ds](https://directory.fedoraproject.org/) and can be used as starting point to authenticate using any LDAP server including Active Directory.
@ -6,6 +6,6 @@ It is tested against [389ds](https://directory.fedoraproject.org/) and can be us
You can configure the server using the [ldapauth.toml](./ldapauth.toml) configuration file. You can configure the server using the [ldapauth.toml](./ldapauth.toml) configuration file.
You can build this example using the following command: You can build this example using the following command:
``` ```console
go build -i -ldflags "-s -w" -o ldapauthserver go build -i -ldflags "-s -w" -o ldapauthserver
``` ```

View file

@ -1,4 +1,4 @@
## REST API CLI client # REST API CLI client
`sftpgo_api_cli.py` is a very simple command line client for `SFTPGo` REST API written in python. `sftpgo_api_cli.py` is a very simple command line client for `SFTPGo` REST API written in python.
@ -10,13 +10,13 @@ It has the following requirements:
You can see the usage with the following command: You can see the usage with the following command:
``` ```console
python sftpgo_api_cli.py --help python sftpgo_api_cli.py --help
``` ```
and and
``` ```console
python sftpgo_api_cli.py [sub-command] --help python sftpgo_api_cli.py [sub-command] --help
``` ```
@ -39,11 +39,11 @@ Additionally it can convert users to the SFTPGo format from some supported users
Let's see a sample usage for each REST API. Let's see a sample usage for each REST API.
### Add user ## Add user
Command: Command:
``` ```console
python sftpgo_api_cli.py add-user test_username --password "test_pwd" --home-dir="/tmp/test_home_dir" --uid 33 --gid 1000 --max-sessions 2 --quota-size 0 --quota-files 3 --permissions "list" "download" "upload" "delete" "rename" "create_dirs" "overwrite" --subdirs-permissions "/dir1::list,download" "/dir2::*" --upload-bandwidth 100 --download-bandwidth 60 --status 0 --expiration-date 2019-01-01 --allowed-ip "192.168.1.1/32" --fs S3 --s3-bucket test --s3-region eu-west-1 --s3-access-key accesskey --s3-access-secret secret --s3-endpoint "http://127.0.0.1:9000" --s3-storage-class Standard --s3-key-prefix "vfolder/" --s3-upload-part-size 10 --s3-upload-concurrency 4 --denied-login-methods "password" "keyboard-interactive" --allowed-extensions "/dir1::.jpg,.png" "/dir2::.rar,.png" --denied-extensions "/dir3::.zip,.rar" python sftpgo_api_cli.py add-user test_username --password "test_pwd" --home-dir="/tmp/test_home_dir" --uid 33 --gid 1000 --max-sessions 2 --quota-size 0 --quota-files 3 --permissions "list" "download" "upload" "delete" "rename" "create_dirs" "overwrite" --subdirs-permissions "/dir1::list,download" "/dir2::*" --upload-bandwidth 100 --download-bandwidth 60 --status 0 --expiration-date 2019-01-01 --allowed-ip "192.168.1.1/32" --fs S3 --s3-bucket test --s3-region eu-west-1 --s3-access-key accesskey --s3-access-secret secret --s3-endpoint "http://127.0.0.1:9000" --s3-storage-class Standard --s3-key-prefix "vfolder/" --s3-upload-part-size 10 --s3-upload-concurrency 4 --denied-login-methods "password" "keyboard-interactive" --allowed-extensions "/dir1::.jpg,.png" "/dir2::.rar,.png" --denied-extensions "/dir3::.zip,.rar"
``` ```
@ -135,11 +135,11 @@ Output:
} }
``` ```
### Update user ## Update user
Command: Command:
``` ```console
python sftpgo_api_cli.py update-user 9576 test_username --password "test_pwd" --home-dir="/tmp/test_home_dir" --uid 0 --gid 33 --max-sessions 3 --quota-size 0 --quota-files 4 --permissions "*" --subdirs-permissions "/dir1::list,download,create_symlinks" --upload-bandwidth 90 --download-bandwidth 80 --status 1 --expiration-date "" --allowed-ip "" --denied-ip "192.168.1.0/24" --denied-login-methods "" --fs local --virtual-folders "/vdir1::/tmp/mapped1::-1::-1" "/vdir2::/tmp/mapped2::100::104857600" --allowed-extensions "" --denied-extensions "" python sftpgo_api_cli.py update-user 9576 test_username --password "test_pwd" --home-dir="/tmp/test_home_dir" --uid 0 --gid 33 --max-sessions 3 --quota-size 0 --quota-files 4 --permissions "*" --subdirs-permissions "/dir1::list,download,create_symlinks" --upload-bandwidth 90 --download-bandwidth 80 --status 1 --expiration-date "" --allowed-ip "" --denied-ip "192.168.1.0/24" --denied-login-methods "" --fs local --virtual-folders "/vdir1::/tmp/mapped1::-1::-1" "/vdir2::/tmp/mapped2::100::104857600" --allowed-extensions "" --denied-extensions ""
``` ```
@ -153,11 +153,11 @@ Output:
} }
``` ```
### Get user by id ## Get user by id
Command: Command:
``` ```console
python sftpgo_api_cli.py get-user-by-id 9576 python sftpgo_api_cli.py get-user-by-id 9576
``` ```
@ -226,11 +226,11 @@ Output:
} }
``` ```
### Get users ## Get users
Command: Command:
``` ```console
python sftpgo_api_cli.py get-users --limit 1 --offset 0 --username test_username --order DESC python sftpgo_api_cli.py get-users --limit 1 --offset 0 --username test_username --order DESC
``` ```
@ -291,11 +291,11 @@ Output:
] ]
``` ```
### Get active connections ## Get active connections
Command: Command:
``` ```console
python sftpgo_api_cli.py get-connections python sftpgo_api_cli.py get-connections
``` ```
@ -325,11 +325,11 @@ Output:
] ]
``` ```
### Get folders ## Get folders
Command: Command:
``` ```console
python sftpgo_api_cli.py get-folders --limit 1 --offset 0 --folder-path /tmp/mapped1 --order DESC python sftpgo_api_cli.py get-folders --limit 1 --offset 0 --folder-path /tmp/mapped1 --order DESC
``` ```
@ -350,9 +350,9 @@ Output:
] ]
``` ```
### Add folder ## Add folder
``` ```console
python sftpgo_api_cli.py add-folder /tmp/mapped_folder python sftpgo_api_cli.py add-folder /tmp/mapped_folder
``` ```
@ -368,11 +368,11 @@ Output:
} }
``` ```
### Close connection ## Close connection
Command: Command:
``` ```console
python sftpgo_api_cli.py close-connection f82cfec6a391ad673edd4ae9a144f32ccb59456139f8e1185b070134fffbab7c python sftpgo_api_cli.py close-connection f82cfec6a391ad673edd4ae9a144f32ccb59456139f8e1185b070134fffbab7c
``` ```
@ -386,19 +386,19 @@ Output:
} }
``` ```
### Get quota scans ## Get quota scans
Command: Command:
``` ```console
python sftpgo_api_cli.py get-quota-scans python sftpgo_api_cli.py get-quota-scans
``` ```
### Start quota scan ## Start quota scan
Command: Command:
``` ```console
python sftpgo_api_cli.py start-quota-scan test_username python sftpgo_api_cli.py start-quota-scan test_username
``` ```
@ -412,19 +412,19 @@ Output:
} }
``` ```
### Get folder quota scans ## Get folder quota scans
Command: Command:
``` ```console
python sftpgo_api_cli.py get-folders-quota-scans python sftpgo_api_cli.py get-folders-quota-scans
``` ```
### Start folder quota scan ## Start folder quota scan
Command: Command:
``` ```console
python sftpgo_api_cli.py start-folder-quota-scan /tmp/mapped_folder python sftpgo_api_cli.py start-folder-quota-scan /tmp/mapped_folder
``` ```
@ -438,11 +438,47 @@ Output:
} }
``` ```
### Delete user ## Update quota usage
Command: Command:
```console
python sftpgo_api_cli.py -d update-quota-usage a -S 123 -F 1 -M reset
``` ```
Output:
```json
{
"error": "",
"message": "Quota updated",
"status": 200
}
```
## Update folder quota usage
Command:
```console
python sftpgo_api_cli.py -d update-quota-usage /tmp/mapped_folder -S 123 -F 1 -M add
```
Output:
```json
{
"error": "",
"message": "Quota updated",
"status": 200
}
```
## Delete user
Command:
```console
python sftpgo_api_cli.py delete-user 9576 python sftpgo_api_cli.py delete-user 9576
``` ```
@ -456,9 +492,9 @@ Output:
} }
``` ```
### Delete folder ## Delete folder
``` ```console
python sftpgo_api_cli.py delete-folder /tmp/mapped_folder python sftpgo_api_cli.py delete-folder /tmp/mapped_folder
``` ```
@ -472,11 +508,11 @@ Output:
} }
``` ```
### Get version ## Get version
Command: Command:
``` ```console
python sftpgo_api_cli.py get-version python sftpgo_api_cli.py get-version
``` ```
@ -490,11 +526,11 @@ Output:
} }
``` ```
### Get provider status ## Get provider status
Command: Command:
``` ```console
python sftpgo_api_cli.py get-provider-status python sftpgo_api_cli.py get-provider-status
``` ```
@ -508,11 +544,11 @@ Output:
} }
``` ```
### Backup data ## Backup data
Command: Command:
``` ```console
python sftpgo_api_cli.py dumpdata backup.json --indent 1 python sftpgo_api_cli.py dumpdata backup.json --indent 1
``` ```
@ -526,11 +562,11 @@ Output:
} }
``` ```
### Restore data ## Restore data
Command: Command:
``` ```console
python sftpgo_api_cli.py loaddata /app/data/backups/backup.json --scan-quota 2 --mode 0 python sftpgo_api_cli.py loaddata /app/data/backups/backup.json --scan-quota 2 --mode 0
``` ```
@ -544,7 +580,7 @@ Output:
} }
``` ```
### Convert users from other stores ## Convert users from other stores
You can convert users to the SFTPGo format from the following users stores: You can convert users to the SFTPGo format from the following users stores:
@ -554,21 +590,21 @@ You can convert users to the SFTPGo format from the following users stores:
For details give a look at the `convert-users` subcommand usage: For details give a look at the `convert-users` subcommand usage:
``` ```console
python sftpgo_api_cli.py convert-users --help python sftpgo_api_cli.py convert-users --help
``` ```
Let's see some examples: Let's see some examples:
``` ```console
python sftpgo_api_cli.py convert-users "" unix-passwd unix_users.json --min-uid 500 --force-uid 1000 --force-gid 1000 python sftpgo_api_cli.py convert-users "" unix-passwd unix_users.json --min-uid 500 --force-uid 1000 --force-gid 1000
``` ```
``` ```console
python sftpgo_api_cli.py convert-users pureftpd.passwd pure-ftpd pure_users.json --usernames "user1" "user2" python sftpgo_api_cli.py convert-users pureftpd.passwd pure-ftpd pure_users.json --usernames "user1" "user2"
``` ```
``` ```console
python sftpgo_api_cli.py convert-users proftpd.passwd proftpd pro_users.json python sftpgo_api_cli.py convert-users proftpd.passwd proftpd pro_users.json
``` ```
@ -576,7 +612,7 @@ The json file generated using the `convert-users` subcommand can be used as inpu
Please note that when importing Linux/Unix users the input file is not required: `/etc/passwd` and `/etc/shadow` are automatically parsed. `/etc/shadow` read permission is is typically granted to the `root` user, so you need to execute the `convert-users` subcommand as `root`. Please note that when importing Linux/Unix users the input file is not required: `/etc/passwd` and `/etc/shadow` are automatically parsed. `/etc/shadow` read permission is is typically granted to the `root` user, so you need to execute the `convert-users` subcommand as `root`.
### Colors highlight for Windows command prompt ## Colors highlight for Windows command prompt
If your Windows command prompt does not recognize ANSI/VT100 escape sequences you can download [ANSICON](https://github.com/adoxa/ansicon "ANSICON") extract proper files depending on your Windows OS, and install them using `ansicon -i`. If your Windows command prompt does not recognize ANSI/VT100 escape sequences you can download [ANSICON](https://github.com/adoxa/ansicon "ANSICON") extract proper files depending on your Windows OS, and install them using `ansicon -i`.
Thats all. From now on, your Windows command prompt will be aware of ANSI colors. Thats all. From now on, your Windows command prompt will be aware of ANSI colors.

View file

@ -40,6 +40,8 @@ class SFTPGoApiRequests:
self.providerStatusPath = urlparse.urljoin(baseUrl, '/api/v1/providerstatus') self.providerStatusPath = urlparse.urljoin(baseUrl, '/api/v1/providerstatus')
self.dumpDataPath = urlparse.urljoin(baseUrl, '/api/v1/dumpdata') self.dumpDataPath = urlparse.urljoin(baseUrl, '/api/v1/dumpdata')
self.loadDataPath = urlparse.urljoin(baseUrl, '/api/v1/loaddata') self.loadDataPath = urlparse.urljoin(baseUrl, '/api/v1/loaddata')
self.updateUsedQuotaPath = urlparse.urljoin(baseUrl, "/api/v1/quota_update")
self.updateFolderUsedQuotaPath = urlparse.urljoin(baseUrl, "/api/v1/folder_quota_update")
self.debug = debug self.debug = debug
if authType == 'basic': if authType == 'basic':
self.auth = requests.auth.HTTPBasicAuth(authUser, authPassword) self.auth = requests.auth.HTTPBasicAuth(authUser, authPassword)
@ -284,6 +286,16 @@ class SFTPGoApiRequests:
r = requests.delete(urlparse.urljoin(self.userPath, 'user/' + str(user_id)), auth=self.auth, verify=self.verify) r = requests.delete(urlparse.urljoin(self.userPath, 'user/' + str(user_id)), auth=self.auth, verify=self.verify)
self.printResponse(r) self.printResponse(r)
def updateQuotaUsage(self, username, used_quota_size, used_quota_files, mode):
req = {"username":username, "used_quota_files":used_quota_files, "used_quota_size":used_quota_size}
r = requests.put(self.updateUsedQuotaPath, params={'mode':mode}, json=req, auth=self.auth, verify=self.verify)
self.printResponse(r)
def updateFolderQuotaUsage(self, mapped_path, used_quota_size, used_quota_files, mode):
req = {"mapped_path":mapped_path, "used_quota_files":used_quota_files, "used_quota_size":used_quota_size}
r = requests.put(self.updateFolderUsedQuotaPath, params={'mode':mode}, json=req, auth=self.auth, verify=self.verify)
self.printResponse(r)
def getConnections(self): def getConnections(self):
r = requests.get(self.activeConnectionsPath, auth=self.auth, verify=self.verify) r = requests.get(self.activeConnectionsPath, auth=self.auth, verify=self.verify)
self.printResponse(r) self.printResponse(r)
@ -688,6 +700,22 @@ if __name__ == '__main__':
help='0 means new users are added, existing users are updated. 1 means new users are added,' + help='0 means new users are added, existing users are updated. 1 means new users are added,' +
' existing users are not modified. Default: %(default)s') ' existing users are not modified. Default: %(default)s')
parserUpdateQuotaUsage = subparsers.add_parser('update-quota-usage', help='Update the user used quota limits')
parserUpdateQuotaUsage.add_argument('username', type=str)
parserUpdateQuotaUsage.add_argument('-M', '--mode', type=str, choices=["add", "reset"], default="reset",
help='the update mode specifies if the given quota usage values should be added or ' +
'replace the current ones. Default: %(default)s')
parserUpdateQuotaUsage.add_argument('-S', '--used_quota_size', type=int, default=0, help='Default: %(default)s')
parserUpdateQuotaUsage.add_argument('-F', '--used_quota_files', type=int, default=0, help='Default: %(default)s')
parserUpdateFolderQuotaUsage = subparsers.add_parser('update-folder-quota-usage', help='Update the folder used quota limits')
parserUpdateFolderQuotaUsage.add_argument('folder_path', type=str)
parserUpdateFolderQuotaUsage.add_argument('-M', '--mode', type=str, choices=["add", "reset"], default="reset",
help='the update mode specifies if the given quota usage values should be added or ' +
'replace the current ones. Default: %(default)s')
parserUpdateFolderQuotaUsage.add_argument('-S', '--used_quota_size', type=int, default=0, help='Default: %(default)s')
parserUpdateFolderQuotaUsage.add_argument('-F', '--used_quota_files', type=int, default=0, help='Default: %(default)s')
parserConvertUsers = subparsers.add_parser('convert-users', help='Convert users to a JSON format suitable to use ' + parserConvertUsers = subparsers.add_parser('convert-users', help='Convert users to a JSON format suitable to use ' +
'with loadddata') 'with loadddata')
supportedUsersFormats = [] supportedUsersFormats = []
@ -765,6 +793,10 @@ if __name__ == '__main__':
api.dumpData(args.output_file, args.indent) api.dumpData(args.output_file, args.indent)
elif args.command == 'loaddata': elif args.command == 'loaddata':
api.loadData(args.input_file, args.scan_quota, args.mode) api.loadData(args.input_file, args.scan_quota, args.mode)
elif args.command == 'update-quota-usage':
api.updateQuotaUsage(args.username, args.used_quota_size, args.used_quota_files, args.mode)
elif args.command == 'update-folder-quota-usage':
api.updateFolderQuotaUsage(args.folder_path, args.used_quota_size, args.used_quota_files, args.mode)
elif args.command == 'convert-users': elif args.command == 'convert-users':
convertUsers = ConvertUsers(args.input_file, args.users_format, args.output_file, args.min_uid, args.max_uid, convertUsers = ConvertUsers(args.input_file, args.users_format, args.output_file, args.min_uid, args.max_uid,
args.usernames, args.force_uid, args.force_gid) args.usernames, args.force_uid, args.force_gid)

View file

@ -69,7 +69,7 @@ func addFolder(w http.ResponseWriter, r *http.Request) {
if err == nil { if err == nil {
render.JSON(w, r, folder) render.JSON(w, r, folder)
} else { } else {
sendAPIResponse(w, r, err, "", http.StatusInternalServerError) sendAPIResponse(w, r, err, "", getRespStatus(err))
} }
} else { } else {
sendAPIResponse(w, r, err, "", getRespStatus(err)) sendAPIResponse(w, r, err, "", getRespStatus(err))
@ -88,11 +88,8 @@ func deleteFolderByPath(w http.ResponseWriter, r *http.Request) {
} }
folder, err := dataprovider.GetFolderByPath(dataProvider, folderPath) folder, err := dataprovider.GetFolderByPath(dataProvider, folderPath)
if _, ok := err.(*dataprovider.RecordNotFoundError); ok { if err != nil {
sendAPIResponse(w, r, err, "", http.StatusNotFound) sendAPIResponse(w, r, err, "", getRespStatus(err))
return
} else if err != nil {
sendAPIResponse(w, r, err, "", http.StatusInternalServerError)
return return
} }
err = dataprovider.DeleteFolder(dataProvider, folder) err = dataprovider.DeleteFolder(dataProvider, folder)

View file

@ -1,6 +1,7 @@
package httpd package httpd
import ( import (
"errors"
"net/http" "net/http"
"github.com/go-chi/render" "github.com/go-chi/render"
@ -11,6 +12,11 @@ import (
"github.com/drakkan/sftpgo/vfs" "github.com/drakkan/sftpgo/vfs"
) )
const (
quotaUpdateModeAdd = "add"
quotaUpdateModeReset = "reset"
)
func getQuotaScans(w http.ResponseWriter, r *http.Request) { func getQuotaScans(w http.ResponseWriter, r *http.Request) {
render.JSON(w, r, sftpd.GetQuotaScans()) render.JSON(w, r, sftpd.GetQuotaScans())
} }
@ -19,8 +25,89 @@ func getVFolderQuotaScans(w http.ResponseWriter, r *http.Request) {
render.JSON(w, r, sftpd.GetVFoldersQuotaScans()) render.JSON(w, r, sftpd.GetVFoldersQuotaScans())
} }
func updateUserQuotaUsage(w http.ResponseWriter, r *http.Request) {
r.Body = http.MaxBytesReader(w, r.Body, maxRequestSize)
var u dataprovider.User
err := render.DecodeJSON(r.Body, &u)
if err != nil {
sendAPIResponse(w, r, err, "", http.StatusBadRequest)
return
}
if u.UsedQuotaFiles < 0 || u.UsedQuotaSize < 0 {
sendAPIResponse(w, r, errors.New("Invalid used quota parameters, negative values are not allowed"),
"", http.StatusBadRequest)
return
}
mode, err := getQuotaUpdateMode(r)
if err != nil {
sendAPIResponse(w, r, err, "", http.StatusBadRequest)
return
}
user, err := dataprovider.UserExists(dataProvider, u.Username)
if err != nil {
sendAPIResponse(w, r, err, "", getRespStatus(err))
return
}
if mode == quotaUpdateModeAdd && !user.HasQuotaRestrictions() && dataprovider.GetQuotaTracking() == 2 {
sendAPIResponse(w, r, errors.New("this user has no quota restrictions, only reset mode is supported"),
"", http.StatusBadRequest)
return
}
if !sftpd.AddQuotaScan(user.Username) {
sendAPIResponse(w, r, err, "A quota scan is in progress for this user", http.StatusConflict)
return
}
defer sftpd.RemoveQuotaScan(user.Username) //nolint:errcheck
err = dataprovider.UpdateUserQuota(dataProvider, user, u.UsedQuotaFiles, u.UsedQuotaSize, mode == quotaUpdateModeReset)
if err != nil {
sendAPIResponse(w, r, err, "", getRespStatus(err))
} else {
sendAPIResponse(w, r, err, "Quota updated", http.StatusOK)
}
}
func updateVFolderQuotaUsage(w http.ResponseWriter, r *http.Request) {
r.Body = http.MaxBytesReader(w, r.Body, maxRequestSize)
var f vfs.BaseVirtualFolder
err := render.DecodeJSON(r.Body, &f)
if err != nil {
sendAPIResponse(w, r, err, "", http.StatusBadRequest)
return
}
if f.UsedQuotaFiles < 0 || f.UsedQuotaSize < 0 {
sendAPIResponse(w, r, errors.New("Invalid used quota parameters, negative values are not allowed"),
"", http.StatusBadRequest)
return
}
mode, err := getQuotaUpdateMode(r)
if err != nil {
sendAPIResponse(w, r, err, "", http.StatusBadRequest)
return
}
folder, err := dataprovider.GetFolderByPath(dataProvider, f.MappedPath)
if err != nil {
sendAPIResponse(w, r, err, "", getRespStatus(err))
return
}
if !sftpd.AddVFolderQuotaScan(folder.MappedPath) {
sendAPIResponse(w, r, err, "A quota scan is in progress for this folder", http.StatusConflict)
return
}
defer sftpd.RemoveVFolderQuotaScan(folder.MappedPath) //nolint:errcheck
err = dataprovider.UpdateVirtualFolderQuota(dataProvider, folder, f.UsedQuotaFiles, f.UsedQuotaSize, mode == quotaUpdateModeReset)
if err != nil {
sendAPIResponse(w, r, err, "", getRespStatus(err))
} else {
sendAPIResponse(w, r, err, "Quota updated", http.StatusOK)
}
}
func startQuotaScan(w http.ResponseWriter, r *http.Request) { func startQuotaScan(w http.ResponseWriter, r *http.Request) {
r.Body = http.MaxBytesReader(w, r.Body, maxRequestSize) r.Body = http.MaxBytesReader(w, r.Body, maxRequestSize)
if dataprovider.GetQuotaTracking() == 0 {
sendAPIResponse(w, r, nil, "Quota tracking is disabled!", http.StatusForbidden)
return
}
var u dataprovider.User var u dataprovider.User
err := render.DecodeJSON(r.Body, &u) err := render.DecodeJSON(r.Body, &u)
if err != nil { if err != nil {
@ -29,11 +116,7 @@ func startQuotaScan(w http.ResponseWriter, r *http.Request) {
} }
user, err := dataprovider.UserExists(dataProvider, u.Username) user, err := dataprovider.UserExists(dataProvider, u.Username)
if err != nil { if err != nil {
sendAPIResponse(w, r, err, "", http.StatusNotFound) sendAPIResponse(w, r, err, "", getRespStatus(err))
return
}
if dataprovider.GetQuotaTracking() == 0 {
sendAPIResponse(w, r, nil, "Quota tracking is disabled!", http.StatusForbidden)
return return
} }
if sftpd.AddQuotaScan(user.Username) { if sftpd.AddQuotaScan(user.Username) {
@ -46,6 +129,10 @@ func startQuotaScan(w http.ResponseWriter, r *http.Request) {
func startVFolderQuotaScan(w http.ResponseWriter, r *http.Request) { func startVFolderQuotaScan(w http.ResponseWriter, r *http.Request) {
r.Body = http.MaxBytesReader(w, r.Body, maxRequestSize) r.Body = http.MaxBytesReader(w, r.Body, maxRequestSize)
if dataprovider.GetQuotaTracking() == 0 {
sendAPIResponse(w, r, nil, "Quota tracking is disabled!", http.StatusForbidden)
return
}
var f vfs.BaseVirtualFolder var f vfs.BaseVirtualFolder
err := render.DecodeJSON(r.Body, &f) err := render.DecodeJSON(r.Body, &f)
if err != nil { if err != nil {
@ -54,11 +141,7 @@ func startVFolderQuotaScan(w http.ResponseWriter, r *http.Request) {
} }
folder, err := dataprovider.GetFolderByPath(dataProvider, f.MappedPath) folder, err := dataprovider.GetFolderByPath(dataProvider, f.MappedPath)
if err != nil { if err != nil {
sendAPIResponse(w, r, err, "", http.StatusNotFound) sendAPIResponse(w, r, err, "", getRespStatus(err))
return
}
if dataprovider.GetQuotaTracking() == 0 {
sendAPIResponse(w, r, nil, "Quota tracking is disabled!", http.StatusForbidden)
return return
} }
if sftpd.AddVFolderQuotaScan(folder.MappedPath) { if sftpd.AddVFolderQuotaScan(folder.MappedPath) {
@ -98,3 +181,14 @@ func doFolderQuotaScan(folder vfs.BaseVirtualFolder) error {
logger.Debug(logSender, "", "virtual folder %#v scanned, error: %v", folder.MappedPath, err) logger.Debug(logSender, "", "virtual folder %#v scanned, error: %v", folder.MappedPath, err)
return err return err
} }
func getQuotaUpdateMode(r *http.Request) (string, error) {
mode := quotaUpdateModeReset
if _, ok := r.URL.Query()["mode"]; ok {
mode = r.URL.Query().Get("mode")
if mode != quotaUpdateModeReset && mode != quotaUpdateModeAdd {
return "", errors.New("Invalid mode")
}
}
return mode, nil
}

View file

@ -66,10 +66,8 @@ func getUserByID(w http.ResponseWriter, r *http.Request) {
user, err := dataprovider.GetUserByID(dataProvider, userID) user, err := dataprovider.GetUserByID(dataProvider, userID)
if err == nil { if err == nil {
render.JSON(w, r, dataprovider.HideUserSensitiveData(&user)) render.JSON(w, r, dataprovider.HideUserSensitiveData(&user))
} else if _, ok := err.(*dataprovider.RecordNotFoundError); ok {
sendAPIResponse(w, r, err, "", http.StatusNotFound)
} else { } else {
sendAPIResponse(w, r, err, "", http.StatusInternalServerError) sendAPIResponse(w, r, err, "", getRespStatus(err))
} }
} }
@ -87,7 +85,7 @@ func addUser(w http.ResponseWriter, r *http.Request) {
if err == nil { if err == nil {
render.JSON(w, r, dataprovider.HideUserSensitiveData(&user)) render.JSON(w, r, dataprovider.HideUserSensitiveData(&user))
} else { } else {
sendAPIResponse(w, r, err, "", http.StatusInternalServerError) sendAPIResponse(w, r, err, "", getRespStatus(err))
} }
} else { } else {
sendAPIResponse(w, r, err, "", getRespStatus(err)) sendAPIResponse(w, r, err, "", getRespStatus(err))
@ -103,6 +101,10 @@ func updateUser(w http.ResponseWriter, r *http.Request) {
return return
} }
user, err := dataprovider.GetUserByID(dataProvider, userID) user, err := dataprovider.GetUserByID(dataProvider, userID)
if err != nil {
sendAPIResponse(w, r, err, "", getRespStatus(err))
return
}
currentPermissions := user.Permissions currentPermissions := user.Permissions
currentFileExtensions := user.Filters.FileExtensions currentFileExtensions := user.Filters.FileExtensions
currentS3AccessSecret := "" currentS3AccessSecret := ""
@ -111,13 +113,6 @@ func updateUser(w http.ResponseWriter, r *http.Request) {
} }
user.Permissions = make(map[string][]string) user.Permissions = make(map[string][]string)
user.Filters.FileExtensions = []dataprovider.ExtensionsFilter{} user.Filters.FileExtensions = []dataprovider.ExtensionsFilter{}
if _, ok := err.(*dataprovider.RecordNotFoundError); ok {
sendAPIResponse(w, r, err, "", http.StatusNotFound)
return
} else if err != nil {
sendAPIResponse(w, r, err, "", http.StatusInternalServerError)
return
}
err = render.DecodeJSON(r.Body, &user) err = render.DecodeJSON(r.Body, &user)
if err != nil { if err != nil {
sendAPIResponse(w, r, err, "", http.StatusBadRequest) sendAPIResponse(w, r, err, "", http.StatusBadRequest)
@ -158,11 +153,8 @@ func deleteUser(w http.ResponseWriter, r *http.Request) {
return return
} }
user, err := dataprovider.GetUserByID(dataProvider, userID) user, err := dataprovider.GetUserByID(dataProvider, userID)
if _, ok := err.(*dataprovider.RecordNotFoundError); ok { if err != nil {
sendAPIResponse(w, r, err, "", http.StatusNotFound) sendAPIResponse(w, r, err, "", getRespStatus(err))
return
} else if err != nil {
sendAPIResponse(w, r, err, "", http.StatusInternalServerError)
return return
} }
err = dataprovider.DeleteUser(dataProvider, user) err = dataprovider.DeleteUser(dataProvider, user)

View file

@ -82,6 +82,9 @@ func getRespStatus(err error) int {
if _, ok := err.(*dataprovider.MethodDisabledError); ok { if _, ok := err.(*dataprovider.MethodDisabledError); ok {
return http.StatusForbidden return http.StatusForbidden
} }
if _, ok := err.(*dataprovider.RecordNotFoundError); ok {
return http.StatusNotFound
}
if os.IsNotExist(err) { if os.IsNotExist(err) {
return http.StatusBadRequest return http.StatusBadRequest
} }
@ -218,7 +221,7 @@ func GetQuotaScans(expectedStatusCode int) ([]sftpd.ActiveQuotaScan, []byte, err
return quotaScans, body, err return quotaScans, body, err
} }
// StartQuotaScan start a new quota scan for the given user and checks the received HTTP Status code against expectedStatusCode. // StartQuotaScan starts a new quota scan for the given user and checks the received HTTP Status code against expectedStatusCode.
func StartQuotaScan(user dataprovider.User, expectedStatusCode int) ([]byte, error) { func StartQuotaScan(user dataprovider.User, expectedStatusCode int) ([]byte, error) {
var body []byte var body []byte
userAsJSON, _ := json.Marshal(user) userAsJSON, _ := json.Marshal(user)
@ -231,6 +234,23 @@ func StartQuotaScan(user dataprovider.User, expectedStatusCode int) ([]byte, err
return body, checkResponse(resp.StatusCode, expectedStatusCode) return body, checkResponse(resp.StatusCode, expectedStatusCode)
} }
// UpdateQuotaUsage updates the user used quota limits and checks the received HTTP Status code against expectedStatusCode.
func UpdateQuotaUsage(user dataprovider.User, mode string, expectedStatusCode int) ([]byte, error) {
var body []byte
userAsJSON, _ := json.Marshal(user)
url, err := addModeQueryParam(buildURLRelativeToBase(updateUsedQuotaPath), mode)
if err != nil {
return body, err
}
resp, err := sendHTTPRequest(http.MethodPut, url.String(), bytes.NewBuffer(userAsJSON), "")
if err != nil {
return body, err
}
defer resp.Body.Close()
body, _ = getResponseBody(resp)
return body, checkResponse(resp.StatusCode, expectedStatusCode)
}
// GetConnections returns status and stats for active SFTP/SCP connections // GetConnections returns status and stats for active SFTP/SCP connections
func GetConnections(expectedStatusCode int) ([]sftpd.ConnectionStatus, []byte, error) { func GetConnections(expectedStatusCode int) ([]sftpd.ConnectionStatus, []byte, error) {
var connections []sftpd.ConnectionStatus var connections []sftpd.ConnectionStatus
@ -370,6 +390,23 @@ func StartFolderQuotaScan(folder vfs.BaseVirtualFolder, expectedStatusCode int)
return body, checkResponse(resp.StatusCode, expectedStatusCode) return body, checkResponse(resp.StatusCode, expectedStatusCode)
} }
// UpdateFolderQuotaUsage updates the folder used quota limits and checks the received HTTP Status code against expectedStatusCode.
func UpdateFolderQuotaUsage(folder vfs.BaseVirtualFolder, mode string, expectedStatusCode int) ([]byte, error) {
var body []byte
folderAsJSON, _ := json.Marshal(folder)
url, err := addModeQueryParam(buildURLRelativeToBase(updateFolderUsedQuotaPath), mode)
if err != nil {
return body, err
}
resp, err := sendHTTPRequest(http.MethodPut, url.String(), bytes.NewBuffer(folderAsJSON), "")
if err != nil {
return body, err
}
defer resp.Body.Close()
body, _ = getResponseBody(resp)
return body, checkResponse(resp.StatusCode, expectedStatusCode)
}
// GetVersion returns version details // GetVersion returns version details
func GetVersion(expectedStatusCode int) (version.Info, []byte, error) { func GetVersion(expectedStatusCode int) (version.Info, []byte, error) {
var appVersion version.Info var appVersion version.Info
@ -778,3 +815,16 @@ func addLimitAndOffsetQueryParams(rawurl string, limit, offset int64) (*url.URL,
url.RawQuery = q.Encode() url.RawQuery = q.Encode()
return url, err return url, err
} }
func addModeQueryParam(rawurl, mode string) (*url.URL, error) {
url, err := url.Parse(rawurl)
if err != nil {
return nil, err
}
q := url.Query()
if len(mode) > 0 {
q.Add("mode", mode)
}
url.RawQuery = q.Encode()
return url, err
}

View file

@ -32,6 +32,8 @@ const (
providerStatusPath = "/api/v1/providerstatus" providerStatusPath = "/api/v1/providerstatus"
dumpDataPath = "/api/v1/dumpdata" dumpDataPath = "/api/v1/dumpdata"
loadDataPath = "/api/v1/loaddata" loadDataPath = "/api/v1/loaddata"
updateUsedQuotaPath = "/api/v1/quota_update"
updateFolderUsedQuotaPath = "/api/v1/folder_quota_update"
metricsPath = "/metrics" metricsPath = "/metrics"
pprofBasePath = "/debug" pprofBasePath = "/debug"
webBasePath = "/web" webBasePath = "/web"

View file

@ -47,6 +47,8 @@ const (
activeConnectionsPath = "/api/v1/connection" activeConnectionsPath = "/api/v1/connection"
quotaScanPath = "/api/v1/quota_scan" quotaScanPath = "/api/v1/quota_scan"
quotaScanVFolderPath = "/api/v1/folder_quota_scan" quotaScanVFolderPath = "/api/v1/folder_quota_scan"
updateUsedQuotaPath = "/api/v1/quota_update"
updateFolderUsedQuotaPath = "/api/v1/folder_quota_update"
versionPath = "/api/v1/version" versionPath = "/api/v1/version"
metricsPath = "/metrics" metricsPath = "/metrics"
pprofPath = "/debug/pprof/" pprofPath = "/debug/pprof/"
@ -603,8 +605,13 @@ func TestUserPublicKey(t *testing.T) {
} }
func TestUpdateUser(t *testing.T) { func TestUpdateUser(t *testing.T) {
user, _, err := httpd.AddUser(getTestUser(), http.StatusOK) u := getTestUser()
u.UsedQuotaFiles = 1
u.UsedQuotaSize = 2
user, _, err := httpd.AddUser(u, http.StatusOK)
assert.NoError(t, err) assert.NoError(t, err)
assert.Equal(t, 0, user.UsedQuotaFiles)
assert.Equal(t, int64(0), user.UsedQuotaSize)
user.HomeDir = filepath.Join(homeBasePath, "testmod") user.HomeDir = filepath.Join(homeBasePath, "testmod")
user.UID = 33 user.UID = 33
user.GID = 101 user.GID = 101
@ -683,6 +690,48 @@ func TestUpdateUser(t *testing.T) {
} }
} }
func TestUpdateUserQuotaUsage(t *testing.T) {
u := getTestUser()
usedQuotaFiles := 1
usedQuotaSize := int64(65535)
u.UsedQuotaFiles = usedQuotaFiles
u.UsedQuotaSize = usedQuotaSize
user, _, err := httpd.AddUser(u, http.StatusOK)
assert.NoError(t, err)
_, err = httpd.UpdateQuotaUsage(u, "invalid_mode", http.StatusBadRequest)
assert.NoError(t, err)
_, err = httpd.UpdateQuotaUsage(u, "", http.StatusOK)
assert.NoError(t, err)
user, _, err = httpd.GetUserByID(user.ID, http.StatusOK)
assert.NoError(t, err)
assert.Equal(t, usedQuotaFiles, user.UsedQuotaFiles)
assert.Equal(t, usedQuotaSize, user.UsedQuotaSize)
_, err = httpd.UpdateQuotaUsage(u, "add", http.StatusBadRequest)
assert.NoError(t, err, "user has no quota restrictions add mode should fail")
user, _, err = httpd.GetUserByID(user.ID, http.StatusOK)
assert.NoError(t, err)
assert.Equal(t, usedQuotaFiles, user.UsedQuotaFiles)
assert.Equal(t, usedQuotaSize, user.UsedQuotaSize)
user.QuotaFiles = 100
user, _, err = httpd.UpdateUser(user, http.StatusOK)
assert.NoError(t, err)
_, err = httpd.UpdateQuotaUsage(u, "add", http.StatusOK)
assert.NoError(t, err)
user, _, err = httpd.GetUserByID(user.ID, http.StatusOK)
assert.NoError(t, err)
assert.Equal(t, 2*usedQuotaFiles, user.UsedQuotaFiles)
assert.Equal(t, 2*usedQuotaSize, user.UsedQuotaSize)
u.UsedQuotaFiles = -1
_, err = httpd.UpdateQuotaUsage(u, "", http.StatusBadRequest)
assert.NoError(t, err)
u.UsedQuotaFiles = usedQuotaFiles
u.Username = u.Username + "1"
_, err = httpd.UpdateQuotaUsage(u, "", http.StatusNotFound)
assert.NoError(t, err)
_, err = httpd.RemoveUser(user, http.StatusOK)
assert.NoError(t, err)
}
func TestUserFolderMapping(t *testing.T) { func TestUserFolderMapping(t *testing.T) {
mappedPath1 := filepath.Join(os.TempDir(), "mapped_dir1") mappedPath1 := filepath.Join(os.TempDir(), "mapped_dir1")
mappedPath2 := filepath.Join(os.TempDir(), "mapped_dir2") mappedPath2 := filepath.Join(os.TempDir(), "mapped_dir2")
@ -1038,6 +1087,47 @@ func TestStartQuotaScan(t *testing.T) {
assert.NoError(t, err) assert.NoError(t, err)
} }
func TestUpdateFolderQuotaUsage(t *testing.T) {
f := vfs.BaseVirtualFolder{
MappedPath: filepath.Join(os.TempDir(), "folder"),
}
usedQuotaFiles := 1
usedQuotaSize := int64(65535)
f.UsedQuotaFiles = usedQuotaFiles
f.UsedQuotaSize = usedQuotaSize
folder, _, err := httpd.AddFolder(f, http.StatusOK)
assert.NoError(t, err)
_, err = httpd.UpdateFolderQuotaUsage(folder, "invalid mode", http.StatusBadRequest)
assert.NoError(t, err)
_, err = httpd.UpdateFolderQuotaUsage(f, "reset", http.StatusOK)
assert.NoError(t, err)
folders, _, err := httpd.GetFolders(0, 0, f.MappedPath, http.StatusOK)
assert.NoError(t, err)
if assert.Len(t, folders, 1) {
folder = folders[0]
assert.Equal(t, usedQuotaFiles, folder.UsedQuotaFiles)
assert.Equal(t, usedQuotaSize, folder.UsedQuotaSize)
}
_, err = httpd.UpdateFolderQuotaUsage(f, "add", http.StatusOK)
assert.NoError(t, err)
folders, _, err = httpd.GetFolders(0, 0, f.MappedPath, http.StatusOK)
assert.NoError(t, err)
if assert.Len(t, folders, 1) {
folder = folders[0]
assert.Equal(t, 2*usedQuotaFiles, folder.UsedQuotaFiles)
assert.Equal(t, 2*usedQuotaSize, folder.UsedQuotaSize)
}
f.UsedQuotaSize = -1
_, err = httpd.UpdateFolderQuotaUsage(f, "", http.StatusBadRequest)
assert.NoError(t, err)
f.UsedQuotaSize = usedQuotaSize
f.MappedPath = f.MappedPath + "1"
_, err = httpd.UpdateFolderQuotaUsage(f, "", http.StatusNotFound)
assert.NoError(t, err)
_, err = httpd.RemoveFolder(folder, http.StatusOK)
assert.NoError(t, err)
}
func TestGetVersion(t *testing.T) { func TestGetVersion(t *testing.T) {
_, _, err := httpd.GetVersion(http.StatusOK) _, _, err := httpd.GetVersion(http.StatusOK)
assert.NoError(t, err) assert.NoError(t, err)
@ -1115,6 +1205,8 @@ func TestQuotaTrackingDisabled(t *testing.T) {
assert.NoError(t, err) assert.NoError(t, err)
_, err = httpd.StartQuotaScan(user, http.StatusForbidden) _, err = httpd.StartQuotaScan(user, http.StatusForbidden)
assert.NoError(t, err) assert.NoError(t, err)
_, err = httpd.UpdateQuotaUsage(user, "", http.StatusForbidden)
assert.NoError(t, err)
_, err = httpd.RemoveUser(user, http.StatusOK) _, err = httpd.RemoveUser(user, http.StatusOK)
assert.NoError(t, err) assert.NoError(t, err)
// folder quota scan must fail // folder quota scan must fail
@ -1125,6 +1217,8 @@ func TestQuotaTrackingDisabled(t *testing.T) {
assert.NoError(t, err) assert.NoError(t, err)
_, err = httpd.StartFolderQuotaScan(folder, http.StatusForbidden) _, err = httpd.StartFolderQuotaScan(folder, http.StatusForbidden)
assert.NoError(t, err) assert.NoError(t, err)
_, err = httpd.UpdateFolderQuotaUsage(folder, "", http.StatusForbidden)
assert.NoError(t, err)
_, err = httpd.RemoveFolder(folder, http.StatusOK) _, err = httpd.RemoveFolder(folder, http.StatusOK)
assert.NoError(t, err) assert.NoError(t, err)
@ -1554,6 +1648,42 @@ func TestUpdateUserMock(t *testing.T) {
checkResponseCode(t, http.StatusOK, rr.Code) checkResponseCode(t, http.StatusOK, rr.Code)
} }
func TestUpdateUserQuotaUsageMock(t *testing.T) {
var user dataprovider.User
u := getTestUser()
usedQuotaFiles := 1
usedQuotaSize := int64(65535)
u.UsedQuotaFiles = usedQuotaFiles
u.UsedQuotaSize = usedQuotaSize
userAsJSON := getUserAsJSON(t, u)
req, _ := http.NewRequest(http.MethodPost, userPath, bytes.NewBuffer(userAsJSON))
rr := executeRequest(req)
checkResponseCode(t, http.StatusOK, rr.Code)
err := render.DecodeJSON(rr.Body, &user)
assert.NoError(t, err)
req, _ = http.NewRequest(http.MethodPut, updateUsedQuotaPath, bytes.NewBuffer(userAsJSON))
rr = executeRequest(req)
checkResponseCode(t, http.StatusOK, rr.Code)
req, _ = http.NewRequest(http.MethodGet, userPath+"/"+strconv.FormatInt(user.ID, 10), nil)
rr = executeRequest(req)
checkResponseCode(t, http.StatusOK, rr.Code)
err = render.DecodeJSON(rr.Body, &user)
assert.NoError(t, err)
assert.Equal(t, usedQuotaFiles, user.UsedQuotaFiles)
assert.Equal(t, usedQuotaSize, user.UsedQuotaSize)
req, _ = http.NewRequest(http.MethodPut, updateUsedQuotaPath, bytes.NewBuffer([]byte("string")))
rr = executeRequest(req)
checkResponseCode(t, http.StatusBadRequest, rr.Code)
assert.True(t, sftpd.AddQuotaScan(user.Username))
req, _ = http.NewRequest(http.MethodPut, updateUsedQuotaPath, bytes.NewBuffer(userAsJSON))
rr = executeRequest(req)
checkResponseCode(t, http.StatusConflict, rr.Code)
assert.NoError(t, sftpd.RemoveQuotaScan(user.Username))
req, _ = http.NewRequest(http.MethodDelete, userPath+"/"+strconv.FormatInt(user.ID, 10), nil)
rr = executeRequest(req)
checkResponseCode(t, http.StatusOK, rr.Code)
}
func TestUserPermissionsMock(t *testing.T) { func TestUserPermissionsMock(t *testing.T) {
user := getTestUser() user := getTestUser()
user.Permissions = make(map[string][]string) user.Permissions = make(map[string][]string)
@ -1784,6 +1914,64 @@ func TestStartQuotaScanMock(t *testing.T) {
assert.NoError(t, err) assert.NoError(t, err)
} }
func TestUpdateFolderQuotaUsageMock(t *testing.T) {
mappedPath := filepath.Join(os.TempDir(), "vfolder")
f := vfs.BaseVirtualFolder{
MappedPath: mappedPath,
}
usedQuotaFiles := 1
usedQuotaSize := int64(65535)
f.UsedQuotaFiles = usedQuotaFiles
f.UsedQuotaSize = usedQuotaSize
var folder vfs.BaseVirtualFolder
folderAsJSON, err := json.Marshal(f)
assert.NoError(t, err)
req, _ := http.NewRequest(http.MethodPost, folderPath, bytes.NewBuffer(folderAsJSON))
rr := executeRequest(req)
checkResponseCode(t, http.StatusOK, rr.Code)
err = render.DecodeJSON(rr.Body, &folder)
assert.NoError(t, err)
req, _ = http.NewRequest(http.MethodPut, updateFolderUsedQuotaPath, bytes.NewBuffer(folderAsJSON))
rr = executeRequest(req)
checkResponseCode(t, http.StatusOK, rr.Code)
var folders []vfs.BaseVirtualFolder
url, err := url.Parse(folderPath)
assert.NoError(t, err)
q := url.Query()
q.Add("folder_path", mappedPath)
url.RawQuery = q.Encode()
req, _ = http.NewRequest(http.MethodGet, url.String(), nil)
rr = executeRequest(req)
checkResponseCode(t, http.StatusOK, rr.Code)
err = render.DecodeJSON(rr.Body, &folders)
assert.NoError(t, err)
if assert.Len(t, folders, 1) {
folder = folders[0]
assert.Equal(t, usedQuotaFiles, folder.UsedQuotaFiles)
assert.Equal(t, usedQuotaSize, folder.UsedQuotaSize)
}
req, _ = http.NewRequest(http.MethodPut, updateFolderUsedQuotaPath, bytes.NewBuffer([]byte("string")))
rr = executeRequest(req)
checkResponseCode(t, http.StatusBadRequest, rr.Code)
assert.True(t, sftpd.AddVFolderQuotaScan(mappedPath))
req, _ = http.NewRequest(http.MethodPut, updateFolderUsedQuotaPath, bytes.NewBuffer(folderAsJSON))
rr = executeRequest(req)
checkResponseCode(t, http.StatusConflict, rr.Code)
assert.NoError(t, sftpd.RemoveVFolderQuotaScan(mappedPath))
url, err = url.Parse(folderPath)
assert.NoError(t, err)
q = url.Query()
q.Add("folder_path", mappedPath)
url.RawQuery = q.Encode()
req, _ = http.NewRequest(http.MethodDelete, url.String(), nil)
rr = executeRequest(req)
checkResponseCode(t, http.StatusOK, rr.Code)
}
func TestStartFolderQuotaScanMock(t *testing.T) { func TestStartFolderQuotaScanMock(t *testing.T) {
mappedPath := filepath.Join(os.TempDir(), "vfolder") mappedPath := filepath.Join(os.TempDir(), "vfolder")
folder := vfs.BaseVirtualFolder{ folder := vfs.BaseVirtualFolder{

View file

@ -357,17 +357,24 @@ func TestApiCallsWithBadURL(t *testing.T) {
oldAuthUsername := authUsername oldAuthUsername := authUsername
oldAuthPassword := authPassword oldAuthPassword := authPassword
SetBaseURLAndCredentials(invalidURL, oldAuthUsername, oldAuthPassword) SetBaseURLAndCredentials(invalidURL, oldAuthUsername, oldAuthPassword)
folder := vfs.BaseVirtualFolder{
MappedPath: os.TempDir(),
}
u := dataprovider.User{} u := dataprovider.User{}
_, _, err := UpdateUser(u, http.StatusBadRequest) _, _, err := UpdateUser(u, http.StatusBadRequest)
assert.Error(t, err) assert.Error(t, err)
_, err = RemoveUser(u, http.StatusNotFound) _, err = RemoveUser(u, http.StatusNotFound)
assert.Error(t, err) assert.Error(t, err)
_, err = RemoveFolder(vfs.BaseVirtualFolder{}, http.StatusNotFound) _, err = RemoveFolder(folder, http.StatusNotFound)
assert.Error(t, err) assert.Error(t, err)
_, _, err = GetUsers(1, 0, "", http.StatusBadRequest) _, _, err = GetUsers(1, 0, "", http.StatusBadRequest)
assert.Error(t, err) assert.Error(t, err)
_, _, err = GetFolders(1, 0, "", http.StatusBadRequest) _, _, err = GetFolders(1, 0, "", http.StatusBadRequest)
assert.Error(t, err) assert.Error(t, err)
_, err = UpdateQuotaUsage(u, "", http.StatusNotFound)
assert.Error(t, err)
_, err = UpdateFolderQuotaUsage(folder, "", http.StatusNotFound)
assert.Error(t, err)
_, err = CloseConnection("non_existent_id", http.StatusNotFound) _, err = CloseConnection("non_existent_id", http.StatusNotFound)
assert.Error(t, err) assert.Error(t, err)
_, _, err = Dumpdata("backup.json", "", http.StatusBadRequest) _, _, err = Dumpdata("backup.json", "", http.StatusBadRequest)
@ -393,6 +400,8 @@ func TestApiCallToNotListeningServer(t *testing.T) {
assert.Error(t, err) assert.Error(t, err)
_, _, err = GetUsers(100, 0, "", http.StatusOK) _, _, err = GetUsers(100, 0, "", http.StatusOK)
assert.Error(t, err) assert.Error(t, err)
_, err = UpdateQuotaUsage(u, "", http.StatusNotFound)
assert.Error(t, err)
_, _, err = GetQuotaScans(http.StatusOK) _, _, err = GetQuotaScans(http.StatusOK)
assert.Error(t, err) assert.Error(t, err)
_, err = StartQuotaScan(u, http.StatusNotFound) _, err = StartQuotaScan(u, http.StatusNotFound)
@ -408,6 +417,8 @@ func TestApiCallToNotListeningServer(t *testing.T) {
assert.Error(t, err) assert.Error(t, err)
_, _, err = GetFolders(0, 0, "", http.StatusOK) _, _, err = GetFolders(0, 0, "", http.StatusOK)
assert.Error(t, err) assert.Error(t, err)
_, err = UpdateFolderQuotaUsage(folder, "", http.StatusNotFound)
assert.Error(t, err)
_, _, err = GetFoldersQuotaScans(http.StatusOK) _, _, err = GetFoldersQuotaScans(http.StatusOK)
assert.Error(t, err) assert.Error(t, err)
_, _, err = GetConnections(http.StatusOK) _, _, err = GetConnections(http.StatusOK)

View file

@ -86,6 +86,8 @@ func initializeRouter(staticFilesPath string, enableProfiler, enableWebAdmin boo
router.Delete(folderPath, deleteFolderByPath) router.Delete(folderPath, deleteFolderByPath)
router.Get(dumpDataPath, dumpData) router.Get(dumpDataPath, dumpData)
router.Get(loadDataPath, loadData) router.Get(loadDataPath, loadData)
router.Put(updateUsedQuotaPath, updateUserQuotaUsage)
router.Put(updateFolderUsedQuotaPath, updateVFolderQuotaUsage)
if enableWebAdmin { if enableWebAdmin {
router.Get(webUsersPath, handleGetWebUsers) router.Get(webUsersPath, handleGetWebUsers)
router.Get(webUserPath, handleWebAddUserGet) router.Get(webUserPath, handleWebAddUserGet)

View file

@ -2,7 +2,7 @@ openapi: 3.0.1
info: info:
title: SFTPGo title: SFTPGo
description: 'SFTPGo REST API' description: 'SFTPGo REST API'
version: 1.9.0 version: 1.9.1
servers: servers:
- url: /api/v1 - url: /api/v1
@ -348,6 +348,202 @@ paths:
status: 500 status: 500
message: "" message: ""
error: "Error description if any" error: "Error description if any"
/quota_update:
put:
tags:
- quota
summary: update the user used quota limits
description: Set the current used quota limits for the given user
operationId: quota_update
parameters:
- in: query
name: mode
required: false
description: the update mode specifies if the given quota usage values should be added or replace the current ones
schema:
type: string
enum: [add, reset]
description: >
Update type:
* `add` - add the specified quota limits to the current used ones
* `reset` - reset the values to the specified ones. This is the default
example: reset
requestBody:
required: true
description: The only user mandatory fields are username,used_quota_size and used_quota_files. Please note that if the quota fields are missing they will default to 0
content:
application/json:
schema:
$ref : '#/components/schemas/User'
responses:
200:
description: successful operation
content:
application/json:
schema:
$ref: '#/components/schemas/ApiResponse'
example:
status: 200
message: "Quota updated"
error: ""
400:
description: Bad request
content:
application/json:
schema:
$ref: '#/components/schemas/ApiResponse'
example:
status: 400
message: ""
error: "Error description if any"
401:
description: Unauthorized
content:
application/json:
schema:
$ref: '#/components/schemas/ApiResponse'
example:
status: 401
message: ""
error: "Error description if any"
403:
description: Forbidden
content:
application/json:
schema:
$ref: '#/components/schemas/ApiResponse'
example:
status: 403
message: ""
error: "Error description if any"
404:
description: Not Found
content:
application/json:
schema:
$ref: '#/components/schemas/ApiResponse'
example:
status: 404
message: ""
error: "Error description if any"
409:
description: A quota scan is in progress for this user
content:
application/json:
schema:
$ref: '#/components/schemas/ApiResponse'
example:
status: 409
message: "A quota scan is in progress"
error: "Error description if any"
500:
description: Internal Server Error
content:
application/json:
schema:
$ref: '#/components/schemas/ApiResponse'
example:
status: 500
message: ""
error: "Error description if any"
/folder_quota_update:
put:
tags:
- quota
summary: update the folder used quota limits
description: Set the current used quota limits for the given folder
operationId: folder_quota_update
parameters:
- in: query
name: mode
required: false
description: the update mode specifies if the given quota usage values should be added or replace the current ones
schema:
type: string
enum: [add, reset]
description: >
Update type:
* `add` - add the specified quota limits to the current used ones
* `reset` - reset the values to the specified ones. This is the default
example: reset
requestBody:
required: true
description: The only folder mandatory fields are mapped_path,used_quota_size and used_quota_files. Please note that if the used quota fields are missing they will default to 0
content:
application/json:
schema:
$ref : '#/components/schemas/BaseVirtualFolder'
responses:
200:
description: successful operation
content:
application/json:
schema:
$ref: '#/components/schemas/ApiResponse'
example:
status: 200
message: "Quota updated"
error: ""
400:
description: Bad request
content:
application/json:
schema:
$ref: '#/components/schemas/ApiResponse'
example:
status: 400
message: ""
error: "Error description if any"
401:
description: Unauthorized
content:
application/json:
schema:
$ref: '#/components/schemas/ApiResponse'
example:
status: 401
message: ""
error: "Error description if any"
403:
description: Forbidden
content:
application/json:
schema:
$ref: '#/components/schemas/ApiResponse'
example:
status: 403
message: ""
error: "Error description if any"
404:
description: Not Found
content:
application/json:
schema:
$ref: '#/components/schemas/ApiResponse'
example:
status: 404
message: ""
error: "Error description if any"
409:
description: A quota scan is in progress for this folder
content:
application/json:
schema:
$ref: '#/components/schemas/ApiResponse'
example:
status: 409
message: "A quota scan is in progress"
error: "Error description if any"
500:
description: Internal Server Error
content:
application/json:
schema:
$ref: '#/components/schemas/ApiResponse'
example:
status: 500
message: ""
error: "Error description if any"
/folder_quota_scan: /folder_quota_scan:
get: get:
tags: tags: