beenull/sftpgo-mirror
1
0
Fork 0
mirror of https://github.com/drakkan/sftpgo.git synced 2024-11-21 23:20:24 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 6

eventmanager: add support for filesystem actions

Browse source 
Fixes #931

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
This commit is contained in:
Nicola Murino 2022-08-10 18:41:59 +02:00
parent 890dde0e00
commit 4cd340e07f
No known key found for this signature in database
GPG key ID: 2F1FB59433D5A8CB
22 changed files with 1300 additions and 144 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

16
README.md
View file

@ -96,8 +96,10 @@ SFTPGo is developed and tested on Linux. After each commit, the code is automati
## Requirements
- Go as build only dependency. We support the Go version(s) used in [continuous integration workflows](./.github/workflows).
- A suitable SQL server to use as data provider: PostgreSQL 9.4+, MySQL 5.6+, SQLite 3.x, CockroachDB stable.
- The SQL server is optional: you can choose to use an embedded bolt database as key/value store or an in memory data provider.
- A suitable SQL server to use as data provider:
- upstream supported versions of PostgreSQL, MySQL and MariaDB.
- CockroachDB stable.
- The SQL server is optional: you can choose to use an embedded SQLite, bolt or in memory data provider.
## Installation
@ -238,16 +240,18 @@ The `revertprovider` command is not supported for the memory provider.
Please note that we only support the current release branch and the current main branch, if you find a bug it is better to report it rather than downgrading to an older unsupported version.
## Users and folders management
## Users, groups and folders management
After starting SFTPGo you can manage users and folders using:
After starting SFTPGo you can manage users, groups, folders and other resources using:
- the [web based administration interface](./docs/web-admin.md)
- the [REST API](./docs/rest-api.md)
To support embedded data providers like `bolt` and `SQLite` we can't have a CLI that directly write users and folders to the data provider, we always have to use the REST API.
To support embedded data providers like `bolt` and `SQLite`, which do not support concurrent connections, we can't have a CLI that directly write users and other resources to the data provider, we always have to use the REST API.
Full details for users, folders, admins and other resources are documented in the [OpenAPI](./openapi/openapi.yaml) schema. If you want to render the schema without importing it manually, you can explore it on [Stoplight](https://sftpgo.stoplight.io/docs/sftpgo/openapi.yaml).
Full details for users, groups, folders, admins and other resources are documented in the [OpenAPI](./openapi/openapi.yaml) schema. If you want to render the schema without importing it manually, you can explore it on [Stoplight](https://sftpgo.stoplight.io/docs/sftpgo/openapi.yaml).
:warning: SFTPGo users, groups and folders are virtual and therefore unrelated to the system ones. There is no need to create system-wide users and groups.
## Tutorials

4
docs/custom-actions.md
View file

@ -43,7 +43,7 @@ If the `hook` defines a path to an external program, then this program can read
- `SFTPGO_ACTION_BUCKET`, non-empty for S3, GCS and Azure backends
- `SFTPGO_ACTION_ENDPOINT`, non-empty for S3, SFTP and Azure backend if configured
- `SFTPGO_ACTION_STATUS`, integer. Status for `upload`, `download` and `ssh_cmd` actions. 1 means no error, 2 means a generic error occurred, 3 means quota exceeded error
- `SFTPGO_ACTION_PROTOCOL`, string. Possible values are `SSH`, `SFTP`, `SCP`, `FTP`, `DAV`, `HTTP`, `HTTPShare`, `OIDC`, `DataRetention`
- `SFTPGO_ACTION_PROTOCOL`, string. Possible values are `SSH`, `SFTP`, `SCP`, `FTP`, `DAV`, `HTTP`, `HTTPShare`, `OIDC`, `DataRetention`, `EventAction`
- `SFTPGO_ACTION_IP`, the action was executed from this IP address
- `SFTPGO_ACTION_SESSION_ID`, string. Unique protocol session identifier. For stateless protocols such as HTTP the session id will change for each request
- `SFTPGO_ACTION_OPEN_FLAGS`, integer. File open flags, can be non-zero for `pre-upload` action. If `SFTPGO_ACTION_FILE_SIZE` is greater than zero and `SFTPGO_ACTION_OPEN_FLAGS&512 == 0` the target file will not be truncated
@ -66,7 +66,7 @@ If the `hook` defines an HTTP URL then this URL will be invoked as HTTP POST. Th
- `bucket`, string, included for S3, GCS and Azure backends
- `endpoint`, string, included for S3, SFTP and Azure backend if configured
- `status`, integer. Status for `upload`, `download` and `ssh_cmd` actions. 1 means no error, 2 means a generic error occurred, 3 means quota exceeded error
- `protocol`, string. Possible values are `SSH`, `SFTP`, `SCP`, `FTP`, `DAV`, `HTTP`, `HTTPShare`, `OIDC`, `DataRetention`
- `protocol`, string. Possible values are `SSH`, `SFTP`, `SCP`, `FTP`, `DAV`, `HTTP`, `HTTPShare`, `OIDC`, `DataRetention`, `EventAction`
- `ip`, string. The action was executed from this IP address
- `session_id`, string. Unique protocol session identifier. For stateless protocols such as HTTP the session id will change for each request
- `open_flags`, integer. File open flags, can be non-zero for `pre-upload` action. If `file_size` is greater than zero and `file_size&512 == 0` the target file will not be truncated

16
docs/eventmanager.md
View file

@ -12,6 +12,10 @@ The following actions are supported:
- `Folder quota reset`. The quota used by virtual folders will be updated based on current usage.
- `Transfer quota reset`. The transfer quota values will be reset to `0`.
- `Data retention check`. You can define per-folder retention policies.
- `Filesystem`. For these actions, the required permissions are automatically granted. This is the same as executing the actions from an SFTP client and the same restrictions applies. Supported actions:
- `Rename`. You can rename one or more files or directories.
- `Delete`. You can delete one or more files and directories.
- `Create directories`. You can create one or more directories including sub-directories.
The following placeholders are supported:
@ -34,15 +38,23 @@ Event rules are based on the premise that an event occours. To each rule you can
The following trigger events are supported:
- `Filesystem events`, for example `upload`, `download` etc.
- `Provider events`, for example add/update/delete user.
- `Provider events`, for example `add`, `update`, `delete` user or other resources.
- `Schedules`.
You can further restrict a rule by specifying additional conditions that must be met before the rule’s actions are taken. For example you can react to uploads only if they are performed by a particular user or using a specified protocol.
Actions are executed in a sequential order. For each action associated to a rule you can define the following settings:
Actions such as user quota reset, transfer quota reset, data retention check and folder quota reset are executed for all matching users if the trigger is a schedule or for the affected user if the trigger is a provider event or a filesystem action.
Actions are executed in a sequential order except for sync actions that are executed before the others. For each action associated to a rule you can define the following settings:
- `Stop on failure`, the next action will not be executed if the current one fails.
- `Failure action`, this action will be executed only if at least another one fails.
- `Execute sync`, for upload events, you can execute the action synchronously. Executing an action synchronously means that SFTPGo will not return a result code to the client (which is waiting for it) until your action have completed its execution. If your acion takes a long time to complete this could cause a timeout on the client side, which wouldn't receive the server response in a timely manner and eventually drop the connection.
If you are running multiple SFTPGo instances connected to the same data provider, you can choose whether to allow simultaneous execution for scheduled actions.
Some actions are not supported for some triggers, rules containing incompatible actions are skipped at runtime:
- `Filesystem events`, folder quota reset cannot be executed, we don't have a direct way to get the affected folder.
- `Provider events`, user quota reset, transfer quota reset, data retention check and filesystem actions can be executed only if we modify a user. They will be executed for the affected user. Folder quota reset can be executed only for folders. Filesystem actions are not executed for `delete` user events because the actions is executed after the user deletion.
- `Schedules`, filesystem actions cannot be executed, they require a user.

56
go.mod
View file

@ -4,19 +4,19 @@ go 1.19
require (
cloud.google.com/go/storage v1.24.0
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.1.1
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.1.2
github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v0.4.1
github.com/GehirnInc/crypt v0.0.0-20200316065508-bb7000b8a962
github.com/alexedwards/argon2id v0.0.0-20211130144151-3585854a6387
github.com/aws/aws-sdk-go-v2 v1.16.8
github.com/aws/aws-sdk-go-v2/config v1.15.15
github.com/aws/aws-sdk-go-v2/credentials v1.12.10
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.12.9
github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.11.21
github.com/aws/aws-sdk-go-v2/service/marketplacemetering v1.13.9
github.com/aws/aws-sdk-go-v2/service/s3 v1.27.2
github.com/aws/aws-sdk-go-v2/service/secretsmanager v1.15.14
github.com/aws/aws-sdk-go-v2/service/sts v1.16.10
github.com/aws/aws-sdk-go-v2 v1.16.10
github.com/aws/aws-sdk-go-v2/config v1.15.17
github.com/aws/aws-sdk-go-v2/credentials v1.12.12
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.12.11
github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.11.23
github.com/aws/aws-sdk-go-v2/service/marketplacemetering v1.13.11
github.com/aws/aws-sdk-go-v2/service/s3 v1.27.4
github.com/aws/aws-sdk-go-v2/service/secretsmanager v1.15.16
github.com/aws/aws-sdk-go-v2/service/sts v1.16.12
github.com/cockroachdb/cockroach-go/v2 v2.2.15
github.com/coreos/go-oidc/v3 v3.2.0
github.com/eikenb/pipeat v0.0.0-20210730190139-06b3e6902001
@ -46,7 +46,7 @@ require (
github.com/pires/go-proxyproto v0.6.2
github.com/pkg/sftp v1.13.5
github.com/pquerna/otp v1.3.0
github.com/prometheus/client_golang v1.12.2
github.com/prometheus/client_golang v1.13.0
github.com/robfig/cron/v3 v3.0.1
github.com/rs/cors v1.8.3-0.20220619195839-da52b0701de5
github.com/rs/xid v1.4.0
@ -67,8 +67,8 @@ require (
gocloud.dev v0.26.0
golang.org/x/crypto v0.0.0-20220722155217-630584e8d5aa
golang.org/x/net v0.0.0-20220805013720-a33c5aa5df48
golang.org/x/oauth2 v0.0.0-20220722155238-128564f6959c
golang.org/x/sys v0.0.0-20220804214406-8e32c043e418
golang.org/x/oauth2 v0.0.0-20220808172628-8227340efae7
golang.org/x/sys v0.0.0-20220808155132-1c4a2a72c664
golang.org/x/time v0.0.0-20220722155302-e5dcc9cfc0b9
google.golang.org/api v0.91.0
gopkg.in/natefinch/lumberjack.v2 v2.0.0
@ -76,21 +76,21 @@ require (
require (
cloud.google.com/go v0.103.0 // indirect
cloud.google.com/go/compute v1.7.0 // indirect
cloud.google.com/go/compute v1.8.0 // indirect
cloud.google.com/go/iam v0.3.0 // indirect
github.com/Azure/azure-sdk-for-go/sdk/internal v1.0.0 // indirect
github.com/ajg/form v1.5.1 // indirect
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.4.3 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.15 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.9 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.3.16 // indirect
github.com/aws/aws-sdk-go-v2/internal/v4a v1.0.6 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.9.3 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.1.10 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.9 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.13.9 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.11.13 // indirect
github.com/aws/smithy-go v1.12.0 // indirect
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.4.4 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.17 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.11 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.3.18 // indirect
github.com/aws/aws-sdk-go-v2/internal/v4a v1.0.8 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.9.4 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.1.12 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.11 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.13.11 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.11.15 // indirect
github.com/aws/smithy-go v1.12.1 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/boombuler/barcode v1.0.1 // indirect
github.com/cenkalti/backoff v2.2.1+incompatible // indirect
@ -113,7 +113,7 @@ require (
github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
github.com/hashicorp/hcl v1.0.0 // indirect
github.com/hashicorp/yamux v0.1.1 // indirect
github.com/inconshreveable/mousetrap v1.0.0 // indirect
github.com/inconshreveable/mousetrap v1.0.1 // indirect
github.com/jmespath/go-jmespath v0.4.0 // indirect
github.com/klauspost/cpuid/v2 v2.1.0 // indirect
github.com/kr/fs v0.1.0 // indirect
@ -155,10 +155,10 @@ require (
golang.org/x/tools v0.1.12 // indirect
golang.org/x/xerrors v0.0.0-20220609144429-65e65417b02f // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/genproto v0.0.0-20220805133916-01dd62135a58 // indirect
google.golang.org/genproto v0.0.0-20220810155839-1856144b1d9c // indirect
google.golang.org/grpc v1.48.0 // indirect
google.golang.org/protobuf v1.28.1 // indirect
gopkg.in/ini.v1 v1.66.6 // indirect
gopkg.in/ini.v1 v1.67.0 // indirect
gopkg.in/square/go-jose.v2 v2.6.0 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect

110
go.sum
View file

@ -47,8 +47,9 @@ cloud.google.com/go/compute v1.3.0/go.mod h1:cCZiE1NHEtai4wiufUhW8I8S1JKkAnhnQJW
cloud.google.com/go/compute v1.5.0/go.mod h1:9SMHyhJlzhlkJqrPAc839t2BZFTSk6Jdj6mkzQJeu0M=
cloud.google.com/go/compute v1.6.0/go.mod h1:T29tfhtVbq1wvAPo0E3+7vhgmkOYeXjhFvz/FMzPu0s=
cloud.google.com/go/compute v1.6.1/go.mod h1:g85FgpzFvNULZ+S8AYq87axRKuf2Kh7deLqV/jJ3thU=
cloud.google.com/go/compute v1.7.0 h1:v/k9Eueb8aAJ0vZuxKMrgm6kPhCLZU9HxFU+AFDs9Uk=
cloud.google.com/go/compute v1.7.0/go.mod h1:435lt8av5oL9P3fv1OEzSbSUe+ybHXGMPQHHZWZxy9U=
cloud.google.com/go/compute v1.8.0 h1:NLtR56/eKx9K1s2Tw/4hec2vsU1S3WeKRMj8HXbBo6E=
cloud.google.com/go/compute v1.8.0/go.mod h1:boQ44qJsMqZjKzzsEkoJWQGj4h8ygmyk17UArClWzmg=
cloud.google.com/go/datastore v1.0.0/go.mod h1:LXYbyblFSglQ5pkeyhO+Qmw7ukd3C+pD7TKLgZqpHYE=
cloud.google.com/go/datastore v1.1.0/go.mod h1:umbIZjpQpHh4hmRpGhH4tLFup+FVzqBi1b3c64qFpCk=
cloud.google.com/go/firestore v1.6.1/go.mod h1:asNXNOzBdyVQmEU+ggO8UPodTkEVFW5Qx+rwHnAz+EY=
@ -91,8 +92,8 @@ github.com/Azure/azure-sdk-for-go v51.1.0+incompatible/go.mod h1:9XXNKU+eRnpl9mo
github.com/Azure/azure-sdk-for-go v59.3.0+incompatible h1:dPIm0BO4jsMXFcCI/sLTPkBtE7mk8WMuRHA0JeWhlcQ=
github.com/Azure/azure-sdk-for-go v59.3.0+incompatible/go.mod h1:9XXNKU+eRnpl9moKnB4QOLf1HestfXbmab5FXxiDBjc=
github.com/Azure/azure-sdk-for-go/sdk/azcore v0.19.0/go.mod h1:h6H6c8enJmmocHUbLiiGY6sx7f9i+X3m1CHdd5c6Rdw=
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.1.1 h1:tz19qLF65vuu2ibfTqGVJxG/zZAI27NEIIbvAOQwYbw=
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.1.1/go.mod h1:uGG2W01BaETf0Ozp+QxxKJdMBNRWPdstHG0Fmdwn1/U=
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.1.2 h1:lneMk5qtUMulXa/eVxjVd+/bDYMEDIqYpLzLa2/EsNI=
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.1.2/go.mod h1:uGG2W01BaETf0Ozp+QxxKJdMBNRWPdstHG0Fmdwn1/U=
github.com/Azure/azure-sdk-for-go/sdk/azidentity v0.11.0/go.mod h1:HcM1YX14R7CJcghJGOYCgdezslRSVzqwLf/q+4Y2r/0=
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.0.0 h1:Yoicul8bnVdQrhDMTHxdEckRGX01XvwXDHUT9zYZ3k0=
github.com/Azure/azure-sdk-for-go/sdk/internal v0.7.0/go.mod h1:yqy467j36fJxcRV2TzfVZ1pCb5vxm4BtZPUdYWe/Xo8=
@ -143,67 +144,67 @@ github.com/aws/aws-sdk-go v1.15.27/go.mod h1:mFuSZ37Z9YOHbQEwBWztmVzqXrEkub65tZo
github.com/aws/aws-sdk-go v1.37.0/go.mod h1:hcU610XS61/+aQV88ixoOzUoG7v3b31pl2zKMmprdro=
github.com/aws/aws-sdk-go v1.43.31/go.mod h1:y4AeaBuwd2Lk+GepC1E9v0qOiTws0MIWAX4oIKwKHZo=
github.com/aws/aws-sdk-go-v2 v1.16.2/go.mod h1:ytwTPBG6fXTZLxxeeCCWj2/EMYp/xDUgX+OET6TLNNU=
github.com/aws/aws-sdk-go-v2 v1.16.8 h1:gOe9UPR98XSf7oEJCcojYg+N2/jCRm4DdeIsP85pIyQ=
github.com/aws/aws-sdk-go-v2 v1.16.8/go.mod h1:6CpKuLXg2w7If3ABZCl/qZ6rEgwtjZTn4eAf4RcEyuw=
github.com/aws/aws-sdk-go-v2 v1.16.10 h1:+yDD0tcuHRQZgqONkpDwzepqmElQaSlFPymHRHR9mrc=
github.com/aws/aws-sdk-go-v2 v1.16.10/go.mod h1:WTACcleLz6VZTp7fak4EO5b9Q4foxbn+8PIz3PmyKlo=
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.4.1/go.mod h1:n8Bs1ElDD2wJ9kCRTczA83gYbBmjSwZp3umc6zF4EeM=
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.4.3 h1:S/ZBwevQkr7gv5YxONYpGQxlMFFYSRfz3RMcjsC9Qhk=
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.4.3/go.mod h1:gNsR5CaXKmQSSzrmGxmwmct/r+ZBfbxorAuXYsj/M5Y=
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.4.4 h1:zfT11pa7ifu/VlLDpmc5OY2W4nYmnKkFDGeMVnmqAI0=
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.4.4/go.mod h1:ES0I1GBs+YYgcDS1ek47Erbn4TOL811JKqBXtgzqyZ8=
github.com/aws/aws-sdk-go-v2/config v1.15.3/go.mod h1:9YL3v07Xc/ohTsxFXzan9ZpFpdTOFl4X65BAKYaz8jg=
github.com/aws/aws-sdk-go-v2/config v1.15.15 h1:yBV+J7Au5KZwOIrIYhYkTGJbifZPCkAnCFSvGsF3ui8=
github.com/aws/aws-sdk-go-v2/config v1.15.15/go.mod h1:A1Lzyy/o21I5/s2FbyX5AevQfSVXpvvIDCoVFD0BC4E=
github.com/aws/aws-sdk-go-v2/config v1.15.17 h1:cM/4dqEPc5SjBOeYVdUI7iL/B6jDupCesXzg3AuUzRE=
github.com/aws/aws-sdk-go-v2/config v1.15.17/go.mod h1:eatrtwIm5WdvASoYCy5oPkinfiwiYFg2jLG9tJoKzkE=
github.com/aws/aws-sdk-go-v2/credentials v1.11.2/go.mod h1:j8YsY9TXTm31k4eFhspiQicfXPLZ0gYXA50i4gxPE8g=
github.com/aws/aws-sdk-go-v2/credentials v1.12.10 h1:7gGcMQePejwiKoDWjB9cWnpfVdnz/e5JwJFuT6OrroI=
github.com/aws/aws-sdk-go-v2/credentials v1.12.10/go.mod h1:g5eIM5XRs/OzIIK81QMBl+dAuDyoLN0VYaLP+tBqEOk=
github.com/aws/aws-sdk-go-v2/credentials v1.12.12 h1:iShu6VaWZZZfUZvlGtRjl+g1lWk44g1QmiCTD4KS0jI=
github.com/aws/aws-sdk-go-v2/credentials v1.12.12/go.mod h1:vFHC2HifIWHebmoVsfpqliKuqbAY2LaVlvy03JzF4c4=
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.12.3/go.mod h1:uk1vhHHERfSVCUnqSqz8O48LBYDSC+k6brng09jcMOk=
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.12.9 h1:hz8tc+OW17YqxyFFPSkvfSikbqWcyyHRyPVSTzC0+aI=
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.12.9/go.mod h1:KDCCm4ONIdHtUloDcFvK2+vshZvx4Zmj7UMDfusuz5s=
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.12.11 h1:zZHPdM2x09/0F8D7XyVvQnP2/jaW7bEMmtcSCPYq/iI=
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.12.11/go.mod h1:38Asv/UyQbDNpSXCurZRlDMjzIl6J+wUe8vY3TtUuzA=
github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.11.3/go.mod h1:0dHuD2HZZSiwfJSy1FO5bX1hQ1TxVV1QXXjpn3XUE44=
github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.11.21 h1:bpiKFJ9aC0xTVpygSRRRL/YHC1JZ+pHQHENATHuoiwo=
github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.11.21/go.mod h1:iIYPrQ2rYfZiB/iADYlhj9HHZ9TTi6PqKQPAqygohbE=
github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.11.23 h1:lzS1GSHBzvBMlCA030/ecL5tF2ip8RLr/LBq5fBpv/4=
github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.11.23/go.mod h1:yGuKwoNVv2eGUHlp7ciCQLHmFNeESebnHucZfRL9EkA=
github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.9/go.mod h1:AnVH5pvai0pAF4lXRq0bmhbes1u9R8wTE+g+183bZNM=
github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.15 h1:bx5F2mr6H6FC7zNIQoDoUr8wEKnvmwRncujT3FYRtic=
github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.15/go.mod h1:pWrr2OoHlT7M/Pd2y4HV3gJyPb3qj5qMmnPkKSNPYK4=
github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.17 h1:U8DZvyFFesBmK62dYC6BRXm4Cd/wPP3aPcecu3xv/F4=
github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.17/go.mod h1:6qtGip7sJEyvgsLjphRZWF9qPe3xJf1mL/MM01E35Wc=
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.3/go.mod h1:ssOhaLpRlh88H3UmEcsBoVKq309quMvm3Ds8e9d4eJM=
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.9 h1:5sbyznZC2TeFpa4fvtpvpcGbzeXEEs1l1Jo51ynUNsQ=
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.9/go.mod h1:08tUpeSGN33QKSO7fwxXczNfiwCpbj+GxK6XKwqWVv0=
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.11 h1:GMp98usVW5tzQhxd26KWhoNQPlR2noIlfbzqjVGBhLU=
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.11/go.mod h1:cYAfnB+9ZkmZWpQWmPDsuIGm4EA+6k2ZVtxKjw/XJBY=
github.com/aws/aws-sdk-go-v2/internal/ini v1.3.10/go.mod h1:8DcYQcz0+ZJaSxANlHIsbbi6S+zMwjwdDqwW3r9AzaE=
github.com/aws/aws-sdk-go-v2/internal/ini v1.3.16 h1:f0ySVcmQhwmzn7zQozd8wBM3yuGBfzdpsOaKQ0/Epzw=
github.com/aws/aws-sdk-go-v2/internal/ini v1.3.16/go.mod h1:CYmI+7x03jjJih8kBEEFKRQc40UjUokT0k7GbvrhhTc=
github.com/aws/aws-sdk-go-v2/internal/v4a v1.0.6 h1:3L8pcjvgaSOs0zzZcMKzxDSkYKEpwJ2dNVDdxm68jAY=
github.com/aws/aws-sdk-go-v2/internal/v4a v1.0.6/go.mod h1:O7Oc4peGZDEKlddivslfYFvAbgzvl/GH3J8j3JIGBXc=
github.com/aws/aws-sdk-go-v2/internal/ini v1.3.18 h1:/spg6h3tG4pefphbvhpgdMtFMegSajPPSEJd1t8lnpc=
github.com/aws/aws-sdk-go-v2/internal/ini v1.3.18/go.mod h1:hTHq8hL4bAxJyng364s9d4IUGXZOs7Y5LSqAhIiIQ2A=
github.com/aws/aws-sdk-go-v2/internal/v4a v1.0.8 h1:9PY5a+kHQzC6d9eR+KLNSJP3DHDLYmPFA5/+eSDBo9o=
github.com/aws/aws-sdk-go-v2/internal/v4a v1.0.8/go.mod h1:pcQfUOFVK4lMnSzgX3dCA81UsA9YCilRUSYgkjSU2i8=
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.9.1/go.mod h1:GeUru+8VzrTXV/83XyMJ80KpH8xO89VPoUileyNQ+tc=
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.9.3 h1:4n4KCtv5SUoT5Er5XV41huuzrCqepxlW3SDI9qHQebc=
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.9.3/go.mod h1:gkb2qADY+OHaGLKNTYxMaQNacfeyQpZ4csDTQMeFmcw=
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.9.4 h1:akfcyqM9SvrBKWZOkBcXAGDrHfKaEP4Aca8H/bCiLW8=
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.9.4/go.mod h1:oehQLbMQkppKLXvpx/1Eo0X47Fe+0971DXC9UjGnKcI=
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.1.3/go.mod h1:Seb8KNmD6kVTjwRjVEgOT5hPin6sq+v4C2ycJQDwuH8=
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.1.10 h1:7LJcuRalaLw+GYQTMGmVUl4opg2HrDZkvn/L3KvIQfw=
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.1.10/go.mod h1:Qks+dxK3O+Z2deAhNo6cJ8ls1bam3tUGUAcgxQP1c70=
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.1.12 h1:eNQYkKjDSLDjIbBQ85rIkjpBGgnavrl/U3YKDdxAz14=
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.1.12/go.mod h1:k2HaF2yfT082M+kKo3Xdf4rd5HGKvDmrPC5Kwzc2KUw=
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.3/go.mod h1:wlY6SVjuwvh3TVRpTqdy4I1JpBFLX4UGeKZdWntaocw=
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.9 h1:sHfDuhbOuuWSIAEDd3pma6p0JgUcR2iePxtCE8gfCxQ=
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.9/go.mod h1:yQowTpvdZkFVuHrLBXmczat4W+WJKg/PafBZnGBLga0=
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.11 h1:GkYtp4gi4wdWUV+pPetjk5y2aDxbr0t8n5OjVBwZdII=
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.11/go.mod h1:OEofCUKF7Hri4ShOCokF6k6hGq9PCB2sywt/9rLSXjY=
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.13.3/go.mod h1:Bm/v2IaN6rZ+Op7zX+bOUMdL4fsrYZiD0dsjLhNKwZc=
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.13.9 h1:sJdKvydGYDML9LTFcp6qq6Z5fIjN0Rdq2Gvw1hUg8tc=
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.13.9/go.mod h1:Rc5+wn2k8gFSi3V1Ch4mhxOzjMh+bYSXVFfVaqowQOY=
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.13.11 h1:ZBLEKweAzBBtJa8H+MTFfVyvo+eHdM8xec5oTm9IlqI=
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.13.11/go.mod h1:mNS1VHxYXPNqxIdCTxf87j9ROfTMa4fNpIkA+iAfz0g=
github.com/aws/aws-sdk-go-v2/service/kms v1.16.3/go.mod h1:QuiHPBqlOFCi4LqdSskYYAWpQlx3PKmohy+rE2F+o5g=
github.com/aws/aws-sdk-go-v2/service/marketplacemetering v1.13.9 h1:mp3uhL8qDbpWj6s2O2c7Dva7L+HXNDEPHqd8ByCMuzI=
github.com/aws/aws-sdk-go-v2/service/marketplacemetering v1.13.9/go.mod h1:NYicI4GfDyWaGlosi6PONumh2b//1azVE0rEeCbPUUk=
github.com/aws/aws-sdk-go-v2/service/marketplacemetering v1.13.11 h1:GFxBWTb0DLD+PkhVPvNWtPsGBFusifSwHb2uDrIV0E0=
github.com/aws/aws-sdk-go-v2/service/marketplacemetering v1.13.11/go.mod h1:jETcaK7szguipGK6ibOHjRemfxelIygcSUZe+xv9Vp8=
github.com/aws/aws-sdk-go-v2/service/s3 v1.26.3/go.mod h1:g1qvDuRsJY+XghsV6zg00Z4KJ7DtFFCx8fJD2a491Ak=
github.com/aws/aws-sdk-go-v2/service/s3 v1.27.2 h1:NvzGue25jKnuAsh6yQ+TZ4ResMcnp49AWgWGm2L4b5o=
github.com/aws/aws-sdk-go-v2/service/s3 v1.27.2/go.mod h1:u+566cosFI+d+motIz3USXEh6sN8Nq4GrNXSg2RXVMo=
github.com/aws/aws-sdk-go-v2/service/s3 v1.27.4 h1:0RPAahwT63znFepvhfS+/WYtT+gEuAwaeNcCrzTQMH0=
github.com/aws/aws-sdk-go-v2/service/s3 v1.27.4/go.mod h1:wcpDmROpK5W7oWI6JcJIYGrVpHbF/Pu+FHxyBXyoa1E=
github.com/aws/aws-sdk-go-v2/service/secretsmanager v1.15.4/go.mod h1:PJc8s+lxyU8rrre0/4a0pn2wgwiDvOEzoOjcJUBr67o=
github.com/aws/aws-sdk-go-v2/service/secretsmanager v1.15.14 h1:dvvIB9OYsOH10RUNAY7yiCq5fQwGebXx1auBOkBTUlg=
github.com/aws/aws-sdk-go-v2/service/secretsmanager v1.15.14/go.mod h1:xakbH8KMsQQKqzX87uyyzTHshc/0/Df8bsTneTS5pFU=
github.com/aws/aws-sdk-go-v2/service/secretsmanager v1.15.16 h1:+8J3OA/fUAAKpSyI6lAPyPhZVleLxDmuT2dv4lVHK20=
github.com/aws/aws-sdk-go-v2/service/secretsmanager v1.15.16/go.mod h1:vveF0vVbSg0WNZNsi27F0Tbyx9JB8NyExl5Iv0RKLcY=
github.com/aws/aws-sdk-go-v2/service/sns v1.17.4/go.mod h1:kElt+uCcXxcqFyc+bQqZPFD9DME/eC6oHBXvFzQ9Bcw=
github.com/aws/aws-sdk-go-v2/service/sqs v1.18.3/go.mod h1:skmQo0UPvsjsuYYSYMVmrPc1HWCbHUJyrCEp+ZaLzqM=
github.com/aws/aws-sdk-go-v2/service/ssm v1.24.1/go.mod h1:NR/xoKjdbRJ+qx0pMR4mI+N/H1I1ynHwXnO6FowXJc0=
github.com/aws/aws-sdk-go-v2/service/sso v1.11.3/go.mod h1:7UQ/e69kU7LDPtY40OyoHYgRmgfGM4mgsLYtcObdveU=
github.com/aws/aws-sdk-go-v2/service/sso v1.11.13 h1:DQpf+al+aWozOEmVEdml67qkVZ6vdtGUi71BZZWw40k=
github.com/aws/aws-sdk-go-v2/service/sso v1.11.13/go.mod h1:d7ptRksDDgvXaUvxyHZ9SYh+iMDymm94JbVcgvSYSzU=
github.com/aws/aws-sdk-go-v2/service/sso v1.11.15 h1:HaIE5/TtKr66qZTJpvMifDxH4lRt2JZawbkLYOo1F+Y=
github.com/aws/aws-sdk-go-v2/service/sso v1.11.15/go.mod h1:dDVD4ElJRTQXx7dOQ59EkqGyNU9tnwy1RKln+oLIOTU=
github.com/aws/aws-sdk-go-v2/service/sts v1.16.3/go.mod h1:bfBj0iVmsUyUg4weDB4NxktD9rDGeKSVWnjTnwbx9b8=
github.com/aws/aws-sdk-go-v2/service/sts v1.16.10 h1:7tquJrhjYz2EsCBvA9VTl+sBAAh1bv7h/sGASdZOGGo=
github.com/aws/aws-sdk-go-v2/service/sts v1.16.10/go.mod h1:cftkHYN6tCDNfkSasAmclSfl4l7cySoay8vz7p/ce0E=
github.com/aws/aws-sdk-go-v2/service/sts v1.16.12 h1:YU9UHPukkCCnETHEExOptF/BxPvGJKXO/NBx+RMQ/2A=
github.com/aws/aws-sdk-go-v2/service/sts v1.16.12/go.mod h1:b53qpmhHk7mTL2J/tfG6f38neZiyBQSiNXGCuNKq4+4=
github.com/aws/smithy-go v1.11.2/go.mod h1:3xHYmszWVx2c0kIwQeEVf9uSm4fYZt67FBJnwub1bgM=
github.com/aws/smithy-go v1.12.0 h1:gXpeZel/jPoWQ7OEmLIgCUnhkFftqNfwWUwAHSlp1v0=
github.com/aws/smithy-go v1.12.0/go.mod h1:Tg+OJXh4MB2R/uN61Ko2f6hTZwB/ZYGOtib8J3gBHzA=
github.com/aws/smithy-go v1.12.1 h1:yQRC55aXN/y1W10HgwHle01DRuV9Dpf31iGkotjt3Ag=
github.com/aws/smithy-go v1.12.1/go.mod h1:Tg+OJXh4MB2R/uN61Ko2f6hTZwB/ZYGOtib8J3gBHzA=
github.com/benbjohnson/clock v1.1.0/go.mod h1:J11/hYXuz8f4ySSvYwY0FKfm+ezbsZBKZxNJlLklBHA=
github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q=
github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8=
@ -476,8 +477,9 @@ github.com/hashicorp/yamux v0.1.1 h1:yrQxtgseBDrq9Y652vSRDvsKCJKOUD+GzTS4Y0Y8pvE
github.com/hashicorp/yamux v0.1.1/go.mod h1:CtWFDAQgb7dxtzFs4tWbplKIe2jSi3+5vKbgIO0SLnQ=
github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
github.com/ianlancetaylor/demangle v0.0.0-20200824232613-28f6c0f3b639/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
github.com/inconshreveable/mousetrap v1.0.0 h1:Z8tu5sraLXCXIcARxBp/8cbvlwVa7Z1NHg9XEKhtSvM=
github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8=
github.com/inconshreveable/mousetrap v1.0.1 h1:U3uMjPSQEBMNp1lFxmllqCPM6P5u/Xq7Pgzkat/bFNc=
github.com/inconshreveable/mousetrap v1.0.1/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
github.com/jackc/chunkreader v1.0.0/go.mod h1:RT6O25fNZIuasFJRyZ4R/Y2BbhasbmZXF9QQ7T3kePo=
github.com/jackc/chunkreader/v2 v2.0.0/go.mod h1:odVSm741yZoC3dpHEUXIqA9tQRhFrgOHwnPIn9lDKlk=
github.com/jackc/chunkreader/v2 v2.0.1/go.mod h1:odVSm741yZoC3dpHEUXIqA9tQRhFrgOHwnPIn9lDKlk=
@ -671,8 +673,8 @@ github.com/prometheus/client_golang v1.0.0/go.mod h1:db9x61etRT2tGnBNRi70OPL5Fsn
github.com/prometheus/client_golang v1.7.1/go.mod h1:PY5Wy2awLA44sXw4AOSfFBetzPP4j5+D6mVACh+pe2M=
github.com/prometheus/client_golang v1.11.0/go.mod h1:Z6t4BnS23TR94PD6BsDNk8yVqroYurpAkEiz0P2BEV0=
github.com/prometheus/client_golang v1.12.1/go.mod h1:3Z9XVyYiZYEO+YQWt3RD2R3jrbd179Rt297l4aS6nDY=
github.com/prometheus/client_golang v1.12.2 h1:51L9cDoUHVrXx4zWYlcLQIZ+d+VXHgqnYKkIuq4g/34=
github.com/prometheus/client_golang v1.12.2/go.mod h1:3Z9XVyYiZYEO+YQWt3RD2R3jrbd179Rt297l4aS6nDY=
github.com/prometheus/client_golang v1.13.0 h1:b71QUfeo5M8gq2+evJdTPfZhYMAU0uKPkyPJ7TPsloU=
github.com/prometheus/client_golang v1.13.0/go.mod h1:vTeo+zgvILHsnnj/39Ou/1fPN5nJFOEMgftOUOmlvYQ=
github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo=
github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
@ -871,8 +873,8 @@ golang.org/x/oauth2 v0.0.0-20220309155454-6242fa91716a/go.mod h1:DAh4E804XQdzx2j
golang.org/x/oauth2 v0.0.0-20220411215720-9780585627b5/go.mod h1:DAh4E804XQdzx2j+YRIaUnCqCV2RuMz24cGBJ5QYIrc=
golang.org/x/oauth2 v0.0.0-20220608161450-d0670ef3b1eb/go.mod h1:jaDAt6Dkxork7LmZnYtzbRWj0W47D86a3TGe0YHBvmE=
golang.org/x/oauth2 v0.0.0-20220622183110-fd043fe589d2/go.mod h1:jaDAt6Dkxork7LmZnYtzbRWj0W47D86a3TGe0YHBvmE=
golang.org/x/oauth2 v0.0.0-20220722155238-128564f6959c h1:q3gFqPqH7NVofKo3c3yETAP//pPI+G5mvB7qqj1Y5kY=
golang.org/x/oauth2 v0.0.0-20220722155238-128564f6959c/go.mod h1:h4gKUeWbJ4rQPri7E0u6Gs4e9Ri2zaLxzw5DI5XGrYg=
golang.org/x/oauth2 v0.0.0-20220808172628-8227340efae7 h1:dtndE8FcEta75/4kHF3AbpuWzV6f1LjnLrM4pe2SZrw=
golang.org/x/oauth2 v0.0.0-20220808172628-8227340efae7/go.mod h1:h4gKUeWbJ4rQPri7E0u6Gs4e9Ri2zaLxzw5DI5XGrYg=
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
@ -972,8 +974,8 @@ golang.org/x/sys v0.0.0-20220615213510-4f61da869c0c/go.mod h1:oPkhp1MJrh7nUepCBc
golang.org/x/sys v0.0.0-20220624220833-87e55d714810/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220704084225-05e143d24a9e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220728004956-3c1f35247d10/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220804214406-8e32c043e418 h1:9vYwv7OjYaky/tlAeD7C4oC9EsPTlaFl1H2jS++V+ME=
golang.org/x/sys v0.0.0-20220804214406-8e32c043e418/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220808155132-1c4a2a72c664 h1:v1W7bwXHsnLLloWYTVEdvGvA7BHMeBYsPcF0GLDxIRs=
golang.org/x/sys v0.0.0-20220808155132-1c4a2a72c664/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211 h1:JGgROgKl9N8DuW20oFS5gxc+lE67/N3FcwmBPMe7ArY=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
@ -1224,8 +1226,8 @@ google.golang.org/genproto v0.0.0-20220616135557-88e70c0c3a90/go.mod h1:KEWEmljW
google.golang.org/genproto v0.0.0-20220617124728-180714bec0ad/go.mod h1:KEWEmljWE5zPzLBa/oHl6DaEt9LmfH6WtH1OHIvleBA=
google.golang.org/genproto v0.0.0-20220624142145-8cd45d7dbd1f/go.mod h1:KEWEmljWE5zPzLBa/oHl6DaEt9LmfH6WtH1OHIvleBA=
google.golang.org/genproto v0.0.0-20220628213854-d9e0b6570c03/go.mod h1:KEWEmljWE5zPzLBa/oHl6DaEt9LmfH6WtH1OHIvleBA=
google.golang.org/genproto v0.0.0-20220805133916-01dd62135a58 h1:sRT5xdTkj1Kbk30qbYC7VyMj73N5pZYsw6v+Nrzdhno=
google.golang.org/genproto v0.0.0-20220805133916-01dd62135a58/go.mod h1:iHe1svFLAZg9VWz891+QbRMwUv9O/1Ww+/mngYeThbc=
google.golang.org/genproto v0.0.0-20220810155839-1856144b1d9c h1:IooGDWedfLC6KLczH/uduUsKQP42ZZYhKx+zd50L1Sk=
google.golang.org/genproto v0.0.0-20220810155839-1856144b1d9c/go.mod h1:dbqgFATTzChvnt+ujMdZwITVAJHFtfyN1qUhDqEiIlk=
google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
google.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38=
google.golang.org/grpc v1.21.1/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM=
@ -1284,8 +1286,8 @@ gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntN
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI=
gopkg.in/inconshreveable/log15.v2 v2.0.0-20180818164646-67afb5ed74ec/go.mod h1:aPpfJ7XW+gOuirDoZ8gHhLh3kZ1B08FtV2bbmy7Jv3s=
gopkg.in/ini.v1 v1.66.6 h1:LATuAqN/shcYAOkv3wl2L4rkaKqkcgTBQjOyYDvcPKI=
gopkg.in/ini.v1 v1.66.6/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k=
gopkg.in/ini.v1 v1.67.0 h1:Dgnx+6+nfE+IfzjUEISNeydPJh9AXNNsWbGP9KzCsOA=
gopkg.in/ini.v1 v1.67.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k=
gopkg.in/natefinch/lumberjack.v2 v2.0.0 h1:1Lc07Kr7qY4U2YPouBjpCLxpiyxIVoxqXgkXLknAOE8=
gopkg.in/natefinch/lumberjack.v2 v2.0.0/go.mod h1:l0ndWWf7gzL7RNwBG7wST/UCcT4T24xpD6X8LsfU/+k=
gopkg.in/square/go-jose.v2 v2.5.1/go.mod h1:M9dMgbHiYLoDGQrXy7OpJDJWiKiU//h+vD76mk0e1AI=

16
internal/common/actions.go
View file

@ -43,8 +43,17 @@ var (
errUnconfiguredAction = errors.New("no hook is configured for this action")
errNoHook = errors.New("unable to execute action, no hook defined")
errUnexpectedHTTResponse = errors.New("unexpected HTTP hook response code")
hooksConcurrencyGuard = make(chan struct{}, 150)
)
func startNewHook() {
hooksConcurrencyGuard <- struct{}{}
}
func hookEnded() {
<-hooksConcurrencyGuard
}
// ProtocolActions defines the action to execute on file operations and SSH commands
type ProtocolActions struct {
// Valid values are download, upload, pre-delete, delete, rename, ssh_cmd. Empty slice to disable
@ -135,7 +144,12 @@ func ExecuteActionNotification(conn *BaseConnection, operation, filePath, virtua
errRes = errHook
}
} else {
go actionHandler.Handle(notification) //nolint:errcheck
go func() {
startNewHook()
defer hookEnded()
actionHandler.Handle(notification) //nolint:errcheck
}()
}
}
return errRes

4
internal/common/common.go
View file

@ -100,6 +100,7 @@ const (
ProtocolHTTPShare = "HTTPShare"
ProtocolDataRetention = "DataRetention"
ProtocolOIDC = "OIDC"
protocolEventAction = "EventAction"
)
// Upload modes
@ -587,6 +588,9 @@ func (c *Configuration) ExecuteStartupHook() error {
}
func (c *Configuration) executePostDisconnectHook(remoteAddr, protocol, username, connID string, connectionTime time.Time) {
startNewHook()
defer hookEnded()
ipAddr := util.GetIPFromRemoteAddress(remoteAddr)
connDuration := int64(time.Since(connectionTime) / time.Millisecond)

4
internal/common/connection.go
View file

@ -253,7 +253,7 @@ func (c *BaseConnection) getRealFsPath(fsPath string) string {
defer c.RUnlock()
for _, t := range c.activeTransfers {
if p := t.GetRealFsPath(fsPath); len(p) > 0 {
if p := t.GetRealFsPath(fsPath); p != "" {
return p
}
}
@ -858,7 +858,7 @@ func (c *BaseConnection) isRenamePermitted(fsSrc, fsDst vfs.Fs, fsSourcePath, fs
c.Log(logger.LevelWarn, "renaming to a directory mapped as virtual folder is not allowed: %#v", fsTargetPath)
return false
}
if fsSrc.GetRelativePath(fsSourcePath) == "/" {
if virtualSourcePath == "/" || virtualTargetPath == "/" || fsSrc.GetRelativePath(fsSourcePath) == "/" {
c.Log(logger.LevelWarn, "renaming root dir is not allowed")
return false
}

3
internal/common/dataretention.go
View file

@ -384,6 +384,9 @@ func (c *RetentionCheck) sendEmailNotification(elapsed time.Duration, errCheck e
}
func (c *RetentionCheck) sendHookNotification(elapsed time.Duration, errCheck error) error {
startNewHook()
defer hookEnded()
data := make(map[string]any)
totalDeletedFiles := 0
totalDeletedSize := int64(0)

2
internal/common/defender_test.go
View file

@ -366,7 +366,7 @@ func TestLoadHostListFromFile(t *testing.T) {
assert.Len(t, hostList.IPAddresses, 0)
assert.Equal(t, 0, hostList.Ranges.Len())
if runtime.GOOS != "windows" {
if runtime.GOOS != osWindows {
err = os.Chmod(hostsFilePath, 0111)
assert.NoError(t, err)

246
internal/common/eventmanager.go
View file

@ -17,6 +17,7 @@ package common
import (
"bytes"
"context"
"errors"
"fmt"
"io"
"net/http"
@ -30,6 +31,7 @@ import (
"time"
"github.com/robfig/cron/v3"
"github.com/rs/xid"
"github.com/drakkan/sftpgo/v2/internal/dataprovider"
"github.com/drakkan/sftpgo/v2/internal/logger"
@ -47,6 +49,9 @@ var (
func init() {
eventManager = eventRulesContainer{
schedulesMapping: make(map[string][]cron.EntryID),
// arbitrary maximum number of concurrent asynchronous tasks,
// each task could execute multiple actions
concurrencyGuard: make(chan struct{}, 200),
}
dataprovider.SetEventRulesCallbacks(eventManager.loadRules, eventManager.RemoveRule,
func(operation, executor, ip, objectType, objectName string, object plugin.Renderer) {
@ -71,6 +76,15 @@ type eventRulesContainer struct {
Schedules []dataprovider.EventRule
schedulesMapping map[string][]cron.EntryID
lastLoad int64
concurrencyGuard chan struct{}
}
func (r *eventRulesContainer) addAsyncTask() {
r.concurrencyGuard <- struct{}{}
}
func (r *eventRulesContainer) removeAsyncTask() {
<-r.concurrencyGuard
}
func (r *eventRulesContainer) getLastLoadTime() int64 {
@ -245,11 +259,19 @@ func (r *eventRulesContainer) hasFsRules() bool {
// handleFsEvent executes the rules actions defined for the specified event
func (r *eventRulesContainer) handleFsEvent(params EventParams) error {
if params.Protocol == protocolEventAction {
return nil
}
r.RLock()
var rulesWithSyncActions, rulesAsync []dataprovider.EventRule
for _, rule := range r.FsEvents {
if r.checkFsEventMatch(rule.Conditions, params) {
if err := rule.CheckActionsConsistency(""); err != nil {
eventManagerLog(logger.LevelWarn, "rule %q skipped: %v, event %q",
rule.Name, err, params.Event)
continue
}
hasSyncActions := false
for _, action := range rule.Actions {
if action.Options.ExecuteSync {
@ -267,6 +289,7 @@ func (r *eventRulesContainer) handleFsEvent(params EventParams) error {
r.RUnlock()
params.sender = params.Name
if len(rulesAsync) > 0 {
go executeAsyncRulesActions(rulesAsync, params)
}
@ -277,6 +300,7 @@ func (r *eventRulesContainer) handleFsEvent(params EventParams) error {
return nil
}
// username is populated for user objects
func (r *eventRulesContainer) handleProviderEvent(params EventParams) {
r.RLock()
defer r.RUnlock()
@ -284,11 +308,17 @@ func (r *eventRulesContainer) handleProviderEvent(params EventParams) {
var rules []dataprovider.EventRule
for _, rule := range r.ProviderEvents {
if r.checkProviderEventMatch(rule.Conditions, params) {
rules = append(rules, rule)
if err := rule.CheckActionsConsistency(params.ObjectType); err == nil {
rules = append(rules, rule)
} else {
eventManagerLog(logger.LevelWarn, "rule %q skipped: %v, event %q object type %q",
rule.Name, err, params.Event, params.ObjectType)
}
}
}
if len(rules) > 0 {
params.sender = params.ObjectName
go executeAsyncRulesActions(rules, params)
}
}
@ -309,6 +339,30 @@ type EventParams struct {
IP string
Timestamp int64
Object plugin.Renderer
sender string
}
// getUsers returns users with group settings not applied
func (p *EventParams) getUsers() ([]dataprovider.User, error) {
if p.sender == "" {
return dataprovider.DumpUsers()
}
user, err := dataprovider.UserExists(p.sender)
if err != nil {
return nil, fmt.Errorf("error getting user %q: %w", p.sender, err)
}
return []dataprovider.User{user}, nil
}
func (p *EventParams) getFolders() ([]vfs.BaseVirtualFolder, error) {
if p.sender == "" {
return dataprovider.DumpFolders()
}
folder, err := dataprovider.GetFolderByName(p.sender)
if err != nil {
return nil, fmt.Errorf("error getting folder %q: %w", p.sender, err)
}
return []vfs.BaseVirtualFolder{folder}, nil
}
func (p *EventParams) getStringReplacements(addObjectData bool) []string {
@ -495,6 +549,123 @@ func executeEmailRuleAction(c dataprovider.EventActionEmailConfig, params EventP
return err
}
func getUserForEventAction(username string) (dataprovider.User, error) {
user, err := dataprovider.GetUserWithGroupSettings(username)
if err != nil {
return dataprovider.User{}, err
}
user.Filters.DisableFsChecks = false
user.Filters.FilePatterns = nil
for k := range user.Permissions {
user.Permissions[k] = []string{dataprovider.PermAny}
}
return user, err
}
func executeDeleteFileFsAction(conn *BaseConnection, item string, info os.FileInfo) error {
fs, fsPath, err := conn.GetFsAndResolvedPath(item)
if err != nil {
return err
}
return conn.RemoveFile(fs, fsPath, item, info)
}
func executeDeleteFsAction(deletes []string, replacer *strings.Replacer, username string) error {
user, err := getUserForEventAction(username)
if err != nil {
return err
}
connectionID := fmt.Sprintf("%s_%s", protocolEventAction, xid.New().String())
err = user.CheckFsRoot(connectionID)
defer user.CloseFs() //nolint:errcheck
if err != nil {
return err
}
conn := NewBaseConnection(connectionID, protocolEventAction, "", "", user)
for _, item := range deletes {
item = replaceWithReplacer(item, replacer)
info, err := conn.DoStat(item, 0, false)
if err != nil {
if conn.IsNotExistError(err) {
continue
}
return err
}
if info.IsDir() {
if err = conn.RemoveDir(item); err != nil {
return err
}
} else {
if err = executeDeleteFileFsAction(conn, item, info); err != nil {
return err
}
}
}
return nil
}
func executeMkDirsFsAction(dirs []string, replacer *strings.Replacer, username string) error {
user, err := getUserForEventAction(username)
if err != nil {
return err
}
connectionID := fmt.Sprintf("%s_%s", protocolEventAction, xid.New().String())
err = user.CheckFsRoot(connectionID)
defer user.CloseFs() //nolint:errcheck
if err != nil {
return err
}
conn := NewBaseConnection(connectionID, protocolEventAction, "", "", user)
for _, item := range dirs {
item = replaceWithReplacer(item, replacer)
if err = conn.CheckParentDirs(path.Dir(item)); err != nil {
return err
}
if err = conn.CreateDir(item, false); err != nil {
return err
}
}
return nil
}
func executeRenameFsAction(renames []dataprovider.KeyValue, replacer *strings.Replacer, username string) error {
user, err := getUserForEventAction(username)
if err != nil {
return err
}
connectionID := fmt.Sprintf("%s_%s", protocolEventAction, xid.New().String())
err = user.CheckFsRoot(connectionID)
defer user.CloseFs() //nolint:errcheck
if err != nil {
return err
}
conn := NewBaseConnection(connectionID, protocolEventAction, "", "", user)
for _, item := range renames {
source := replaceWithReplacer(item.Key, replacer)
target := replaceWithReplacer(item.Value, replacer)
if err = conn.Rename(source, target); err != nil {
return err
}
}
return nil
}
func executeFsRuleAction(c dataprovider.EventActionFilesystemConfig, params EventParams) error {
addObjectData := false
replacements := params.getStringReplacements(addObjectData)
replacer := strings.NewReplacer(replacements...)
switch c.Type {
case dataprovider.FilesystemActionRename:
return executeRenameFsAction(c.Renames, replacer, params.sender)
case dataprovider.FilesystemActionDelete:
return executeDeleteFsAction(c.Deletes, replacer, params.sender)
case dataprovider.FilesystemActionMkdirs:
return executeMkDirsFsAction(c.MkDirs, replacer, params.sender)
default:
return fmt.Errorf("unsupported filesystem action %d", c.Type)
}
}
func executeQuotaResetForUser(user dataprovider.User) error {
if err := user.LoadAndApplyGroupSettings(); err != nil {
eventManagerLog(logger.LevelDebug, "skipping scheduled quota reset for user %s, cannot apply group settings: %v",
@ -520,18 +691,21 @@ func executeQuotaResetForUser(user dataprovider.User) error {
return nil
}
func executeUsersQuotaResetRuleAction(conditions dataprovider.ConditionOptions) error {
users, err := dataprovider.DumpUsers()
func executeUsersQuotaResetRuleAction(conditions dataprovider.ConditionOptions, params EventParams) error {
users, err := params.getUsers()
if err != nil {
return fmt.Errorf("unable to get users: %w", err)
}
var failedResets []string
executed := 0
for _, user := range users {
if !checkEventConditionPatterns(user.Username, conditions.Names) {
// if sender is set, the conditions have already been evaluated
if params.sender == "" && !checkEventConditionPatterns(user.Username, conditions.Names) {
eventManagerLog(logger.LevelDebug, "skipping scheduled quota reset for user %s, name conditions don't match",
user.Username)
continue
}
executed++
if err = executeQuotaResetForUser(user); err != nil {
failedResets = append(failedResets, user.Username)
continue
@ -540,17 +714,23 @@ func executeUsersQuotaResetRuleAction(conditions dataprovider.ConditionOptions)
if len(failedResets) > 0 {
return fmt.Errorf("quota reset failed for users: %+v", failedResets)
}
if executed == 0 {
eventManagerLog(logger.LevelError, "no user quota reset executed")
return errors.New("no user quota reset executed")
}
return nil
}
func executeFoldersQuotaResetRuleAction(conditions dataprovider.ConditionOptions) error {
folders, err := dataprovider.DumpFolders()
func executeFoldersQuotaResetRuleAction(conditions dataprovider.ConditionOptions, params EventParams) error {
folders, err := params.getFolders()
if err != nil {
return fmt.Errorf("unable to get folders: %w", err)
}
var failedResets []string
executed := 0
for _, folder := range folders {
if !checkEventConditionPatterns(folder.Name, conditions.Names) {
// if sender is set, the conditions have already been evaluated
if params.sender == "" && !checkEventConditionPatterns(folder.Name, conditions.Names) {
eventManagerLog(logger.LevelDebug, "skipping scheduled quota reset for folder %s, name conditions don't match",
folder.Name)
continue
@ -560,6 +740,7 @@ func executeFoldersQuotaResetRuleAction(conditions dataprovider.ConditionOptions
failedResets = append(failedResets, folder.Name)
continue
}
executed++
f := vfs.VirtualFolder{
BaseVirtualFolder: folder,
VirtualPath: "/",
@ -580,21 +761,28 @@ func executeFoldersQuotaResetRuleAction(conditions dataprovider.ConditionOptions
if len(failedResets) > 0 {
return fmt.Errorf("quota reset failed for folders: %+v", failedResets)
}
if executed == 0 {
eventManagerLog(logger.LevelError, "no folder quota reset executed")
return errors.New("no folder quota reset executed")
}
return nil
}
func executeTransferQuotaResetRuleAction(conditions dataprovider.ConditionOptions) error {
users, err := dataprovider.DumpUsers()
func executeTransferQuotaResetRuleAction(conditions dataprovider.ConditionOptions, params EventParams) error {
users, err := params.getUsers()
if err != nil {
return fmt.Errorf("unable to get users: %w", err)
}
var failedResets []string
executed := 0
for _, user := range users {
if !checkEventConditionPatterns(user.Username, conditions.Names) {
// if sender is set, the conditions have already been evaluated
if params.sender == "" && !checkEventConditionPatterns(user.Username, conditions.Names) {
eventManagerLog(logger.LevelDebug, "skipping scheduled transfer quota reset for user %s, name conditions don't match",
user.Username)
continue
}
executed++
err = dataprovider.UpdateUserTransferQuota(&user, 0, 0, true)
if err != nil {
eventManagerLog(logger.LevelError, "error updating transfer quota for user %s: %v", user.Username, err)
@ -604,6 +792,10 @@ func executeTransferQuotaResetRuleAction(conditions dataprovider.ConditionOption
if len(failedResets) > 0 {
return fmt.Errorf("transfer quota reset failed for users: %+v", failedResets)
}
if executed == 0 {
eventManagerLog(logger.LevelError, "no transfer quota reset executed")
return errors.New("no transfer quota reset executed")
}
return nil
}
@ -629,19 +821,22 @@ func executeDataRetentionCheckForUser(user dataprovider.User, folders []dataprov
}
func executeDataRetentionCheckRuleAction(config dataprovider.EventActionDataRetentionConfig,
conditions dataprovider.ConditionOptions,
conditions dataprovider.ConditionOptions, params EventParams,
) error {
users, err := dataprovider.DumpUsers()
users, err := params.getUsers()
if err != nil {
return fmt.Errorf("unable to get users: %w", err)
}
var failedChecks []string
executed := 0
for _, user := range users {
if !checkEventConditionPatterns(user.Username, conditions.Names) {
// if sender is set, the conditions have already been evaluated
if params.sender == "" && !checkEventConditionPatterns(user.Username, conditions.Names) {
eventManagerLog(logger.LevelDebug, "skipping scheduled retention check for user %s, name conditions don't match",
user.Username)
continue
}
executed++
if err = executeDataRetentionCheckForUser(user, config.Folders); err != nil {
failedChecks = append(failedChecks, user.Username)
continue
@ -650,6 +845,10 @@ func executeDataRetentionCheckRuleAction(config dataprovider.EventActionDataRete
if len(failedChecks) > 0 {
return fmt.Errorf("retention check failed for users: %+v", failedChecks)
}
if executed == 0 {
eventManagerLog(logger.LevelError, "no retention check executed")
return errors.New("no retention check executed")
}
return nil
}
@ -664,13 +863,15 @@ func executeRuleAction(action dataprovider.BaseEventAction, params EventParams,
case dataprovider.ActionTypeBackup:
return dataprovider.ExecuteBackup()
case dataprovider.ActionTypeUserQuotaReset:
return executeUsersQuotaResetRuleAction(conditions)
return executeUsersQuotaResetRuleAction(conditions, params)
case dataprovider.ActionTypeFolderQuotaReset:
return executeFoldersQuotaResetRuleAction(conditions)
return executeFoldersQuotaResetRuleAction(conditions, params)
case dataprovider.ActionTypeTransferQuotaReset:
return executeTransferQuotaResetRuleAction(conditions)
return executeTransferQuotaResetRuleAction(conditions, params)
case dataprovider.ActionTypeDataRetentionCheck:
return executeDataRetentionCheckRuleAction(action.Options.RetentionConfig, conditions)
return executeDataRetentionCheckRuleAction(action.Options.RetentionConfig, conditions, params)
case dataprovider.ActionTypeFilesystem:
return executeFsRuleAction(action.Options.FsConfig, params)
default:
return fmt.Errorf("unsupported action type: %d", action.Type)
}
@ -707,6 +908,9 @@ func executeSyncRulesActions(rules []dataprovider.EventRule, params EventParams)
}
func executeAsyncRulesActions(rules []dataprovider.EventRule, params EventParams) {
eventManager.addAsyncTask()
defer eventManager.removeAsyncTask()
for _, rule := range rules {
executeRuleAsyncActions(rule, params, nil)
}
@ -784,6 +988,10 @@ func (j *eventCronJob) Run() {
eventManagerLog(logger.LevelError, "unable to load rule with name %q", j.ruleName)
return
}
if err = rule.CheckActionsConsistency(""); err != nil {
eventManagerLog(logger.LevelWarn, "scheduled rule %q skipped: %v", rule.Name, err)
return
}
task, err := j.getTask(rule)
if err != nil {
return
@ -823,9 +1031,9 @@ func (j *eventCronJob) Run() {
}
}(task.Name)
executeRuleAsyncActions(rule, EventParams{}, nil)
executeAsyncRulesActions([]dataprovider.EventRule{rule}, EventParams{})
} else {
executeRuleAsyncActions(rule, EventParams{}, nil)
executeAsyncRulesActions([]dataprovider.EventRule{rule}, EventParams{})
}
eventManagerLog(logger.LevelDebug, "execution for scheduled rule %q finished", j.ruleName)
}

167
internal/common/eventmanager_test.go
View file

@ -20,6 +20,8 @@ import (
"os"
"path"
"path/filepath"
"runtime"
"strings"
"testing"
"time"
@ -259,11 +261,19 @@ func TestEventManagerErrors(t *testing.T) {
err := dataprovider.Close()
assert.NoError(t, err)
err = executeUsersQuotaResetRuleAction(dataprovider.ConditionOptions{})
params := EventParams{
sender: "sender",
}
_, err = params.getUsers()
assert.Error(t, err)
err = executeFoldersQuotaResetRuleAction(dataprovider.ConditionOptions{})
_, err = params.getFolders()
assert.Error(t, err)
err = executeTransferQuotaResetRuleAction(dataprovider.ConditionOptions{})
err = executeUsersQuotaResetRuleAction(dataprovider.ConditionOptions{}, EventParams{})
assert.Error(t, err)
err = executeFoldersQuotaResetRuleAction(dataprovider.ConditionOptions{}, EventParams{})
assert.Error(t, err)
err = executeTransferQuotaResetRuleAction(dataprovider.ConditionOptions{}, EventParams{})
assert.Error(t, err)
err = executeQuotaResetForUser(dataprovider.User{
Groups: []sdk.GroupMapping{
@ -488,6 +498,17 @@ func TestEventRuleActions(t *testing.T) {
})
assert.Error(t, err)
assert.True(t, QuotaScans.RemoveUserQuotaScan(username1))
// non matching pattern
err = executeRuleAction(action, EventParams{}, dataprovider.ConditionOptions{
Names: []dataprovider.ConditionPattern{
{
Pattern: "don't match",
},
},
})
if assert.Error(t, err) {
assert.Contains(t, err.Error(), "no user quota reset executed")
}
dataRetentionAction := dataprovider.BaseEventAction{
Type: dataprovider.ActionTypeDataRetentionCheck,
@ -571,6 +592,17 @@ func TestEventRuleActions(t *testing.T) {
assert.Error(t, err)
RetentionChecks.remove(user1.Username)
err = executeRuleAction(dataRetentionAction, EventParams{}, dataprovider.ConditionOptions{
Names: []dataprovider.ConditionPattern{
{
Pattern: "no match",
},
},
})
if assert.Error(t, err) {
assert.Contains(t, err.Error(), "no retention check executed")
}
err = os.RemoveAll(user1.GetHomeDir())
assert.NoError(t, err)
@ -591,6 +623,17 @@ func TestEventRuleActions(t *testing.T) {
assert.Equal(t, int64(0), userGet.UsedDownloadDataTransfer)
assert.Equal(t, int64(0), userGet.UsedUploadDataTransfer)
err = executeRuleAction(action, EventParams{}, dataprovider.ConditionOptions{
Names: []dataprovider.ConditionPattern{
{
Pattern: "no match",
},
},
})
if assert.Error(t, err) {
assert.Contains(t, err.Error(), "no transfer quota reset executed")
}
err = dataprovider.DeleteUser(username1, "", "")
assert.NoError(t, err)
err = dataprovider.DeleteUser(username2, "", "")
@ -649,6 +692,17 @@ func TestEventRuleActions(t *testing.T) {
assert.Error(t, err)
assert.True(t, QuotaScans.RemoveVFolderQuotaScan(foldername1))
err = executeRuleAction(action, EventParams{}, dataprovider.ConditionOptions{
Names: []dataprovider.ConditionPattern{
{
Pattern: "no folder match",
},
},
})
if assert.Error(t, err) {
assert.Contains(t, err.Error(), "no folder quota reset executed")
}
err = os.RemoveAll(folder1.MappedPath)
assert.NoError(t, err)
err = dataprovider.DeleteFolder(foldername1, "", "")
@ -657,6 +711,102 @@ func TestEventRuleActions(t *testing.T) {
assert.NoError(t, err)
}
func TestFilesystemActionErrors(t *testing.T) {
err := executeFsRuleAction(dataprovider.EventActionFilesystemConfig{}, EventParams{})
if assert.Error(t, err) {
assert.Contains(t, err.Error(), "unsupported filesystem action")
}
username := "test_user_for_actions"
testReplacer := strings.NewReplacer("old", "new")
err = executeDeleteFsAction(nil, testReplacer, username)
assert.Error(t, err)
err = executeMkDirsFsAction(nil, testReplacer, username)
assert.Error(t, err)
err = executeRenameFsAction(nil, testReplacer, username)
assert.Error(t, err)
user := dataprovider.User{
BaseUser: sdk.BaseUser{
Username: username,
Permissions: map[string][]string{
"/": {dataprovider.PermAny},
},
HomeDir: filepath.Join(os.TempDir(), username),
},
FsConfig: vfs.Filesystem{
Provider: sdk.SFTPFilesystemProvider,
SFTPConfig: vfs.SFTPFsConfig{
BaseSFTPFsConfig: sdk.BaseSFTPFsConfig{
Endpoint: "127.0.0.1:4022",
Username: username,
},
Password: kms.NewPlainSecret("pwd"),
},
},
}
conn := NewBaseConnection("", protocolEventAction, "", "", user)
err = executeDeleteFileFsAction(conn, "", nil)
assert.Error(t, err)
err = dataprovider.AddUser(&user, "", "")
assert.NoError(t, err)
// check root fs fails
err = executeDeleteFsAction(nil, testReplacer, username)
assert.Error(t, err)
err = executeMkDirsFsAction(nil, testReplacer, username)
assert.Error(t, err)
err = executeRenameFsAction(nil, testReplacer, username)
assert.Error(t, err)
user.FsConfig.Provider = sdk.LocalFilesystemProvider
user.Permissions["/"] = []string{dataprovider.PermUpload}
err = dataprovider.DeleteUser(username, "", "")
assert.NoError(t, err)
err = dataprovider.AddUser(&user, "", "")
assert.NoError(t, err)
err = executeRenameFsAction([]dataprovider.KeyValue{
{
Key: "/p1",
Value: "/p1",
},
}, testReplacer, username)
if assert.Error(t, err) {
assert.Contains(t, err.Error(), "the rename source and target cannot be the same")
}
if runtime.GOOS != osWindows {
dirPath := filepath.Join(user.HomeDir, "adir", "sub")
err := os.MkdirAll(dirPath, os.ModePerm)
assert.NoError(t, err)
filePath := filepath.Join(dirPath, "f.dat")
err = os.WriteFile(filePath, nil, 0666)
assert.NoError(t, err)
err = os.Chmod(dirPath, 0001)
assert.NoError(t, err)
err = executeDeleteFsAction([]string{"/adir/sub"}, testReplacer, username)
assert.Error(t, err)
err = executeDeleteFsAction([]string{"/adir/sub/f.dat"}, testReplacer, username)
assert.Error(t, err)
err = os.Chmod(dirPath, 0555)
assert.NoError(t, err)
err = executeDeleteFsAction([]string{"/adir/sub/f.dat"}, testReplacer, username)
assert.Error(t, err)
err = executeMkDirsFsAction([]string{"/adir/sub"}, testReplacer, username)
assert.Error(t, err)
err = executeMkDirsFsAction([]string{"/adir/sub/sub/sub"}, testReplacer, username)
assert.Error(t, err)
err = os.Chmod(dirPath, os.ModePerm)
assert.NoError(t, err)
}
err = dataprovider.DeleteUser(username, "", "")
assert.NoError(t, err)
err = os.RemoveAll(user.GetHomeDir())
assert.NoError(t, err)
}
func TestQuotaActionsWithQuotaTrackDisabled(t *testing.T) {
oldProviderConf := dataprovider.GetProviderConfig()
providerConf := dataprovider.GetProviderConfig()
@ -788,6 +938,17 @@ func TestScheduledActions(t *testing.T) {
job.Run()
assert.DirExists(t, backupsPath)
action.Type = dataprovider.ActionTypeFilesystem
action.Options = dataprovider.BaseEventActionOptions{
FsConfig: dataprovider.EventActionFilesystemConfig{
Type: dataprovider.FilesystemActionMkdirs,
MkDirs: []string{"/dir"},
},
}
err = dataprovider.UpdateEventAction(action, "", "")
assert.NoError(t, err)
job.Run() // action is not compatible with a scheduled rule
err = dataprovider.DeleteEventRule(rule.Name, "", "")
assert.NoError(t, err)
err = dataprovider.DeleteEventAction(action.Name, "", "")

258
internal/common/protocol_test.go
View file

@ -3272,6 +3272,264 @@ func TestEventRuleProviderEvents(t *testing.T) {
assert.NoError(t, err)
}
func TestEventRuleFsActions(t *testing.T) {
dirsToCreate := []string{
"/basedir/1",
"/basedir/sub/2",
"/basedir/3",
}
a1 := dataprovider.BaseEventAction{
Name: "a1",
Type: dataprovider.ActionTypeFilesystem,
Options: dataprovider.BaseEventActionOptions{
FsConfig: dataprovider.EventActionFilesystemConfig{
Type: dataprovider.FilesystemActionMkdirs,
MkDirs: dirsToCreate,
},
},
}
a2 := dataprovider.BaseEventAction{
Name: "a2",
Type: dataprovider.ActionTypeFilesystem,
Options: dataprovider.BaseEventActionOptions{
FsConfig: dataprovider.EventActionFilesystemConfig{
Type: dataprovider.FilesystemActionRename,
Renames: []dataprovider.KeyValue{
{
Key: "/{{VirtualPath}}",
Value: "/{{ObjectName}}_renamed",
},
},
},
},
}
a3 := dataprovider.BaseEventAction{
Name: "a3",
Type: dataprovider.ActionTypeFilesystem,
Options: dataprovider.BaseEventActionOptions{
FsConfig: dataprovider.EventActionFilesystemConfig{
Type: dataprovider.FilesystemActionDelete,
Deletes: []string{"/{{ObjectName}}_renamed"},
},
},
}
a4 := dataprovider.BaseEventAction{
Name: "a4",
Type: dataprovider.ActionTypeFolderQuotaReset,
}
a5 := dataprovider.BaseEventAction{
Name: "a5",
Type: dataprovider.ActionTypeUserQuotaReset,
}
action1, resp, err := httpdtest.AddEventAction(a1, http.StatusCreated)
assert.NoError(t, err, string(resp))
action2, resp, err := httpdtest.AddEventAction(a2, http.StatusCreated)
assert.NoError(t, err, string(resp))
action3, resp, err := httpdtest.AddEventAction(a3, http.StatusCreated)
assert.NoError(t, err, string(resp))
action4, resp, err := httpdtest.AddEventAction(a4, http.StatusCreated)
assert.NoError(t, err, string(resp))
action5, resp, err := httpdtest.AddEventAction(a5, http.StatusCreated)
assert.NoError(t, err, string(resp))
r1 := dataprovider.EventRule{
Name: "r1",
Trigger: dataprovider.EventTriggerProviderEvent,
Conditions: dataprovider.EventConditions{
ProviderEvents: []string{"add"},
},
Actions: []dataprovider.EventAction{
{
BaseEventAction: dataprovider.BaseEventAction{
Name: action1.Name,
},
Order: 1,
},
},
}
r2 := dataprovider.EventRule{
Name: "r2",
Trigger: dataprovider.EventTriggerFsEvent,
Conditions: dataprovider.EventConditions{
FsEvents: []string{"upload"},
},
Actions: []dataprovider.EventAction{
{
BaseEventAction: dataprovider.BaseEventAction{
Name: action2.Name,
},
Order: 1,
Options: dataprovider.EventActionOptions{
ExecuteSync: true,
},
},
{
BaseEventAction: dataprovider.BaseEventAction{
Name: action5.Name,
},
Order: 2,
},
},
}
r3 := dataprovider.EventRule{
Name: "r3",
Trigger: dataprovider.EventTriggerFsEvent,
Conditions: dataprovider.EventConditions{
FsEvents: []string{"mkdir"},
},
Actions: []dataprovider.EventAction{
{
BaseEventAction: dataprovider.BaseEventAction{
Name: action3.Name,
},
Order: 1,
},
},
}
r4 := dataprovider.EventRule{
Name: "r4",
Trigger: dataprovider.EventTriggerFsEvent,
Conditions: dataprovider.EventConditions{
FsEvents: []string{"rmdir"},
},
Actions: []dataprovider.EventAction{
{
BaseEventAction: dataprovider.BaseEventAction{
Name: action4.Name,
},
Order: 1,
},
},
}
r5 := dataprovider.EventRule{
Name: "r5",
Trigger: dataprovider.EventTriggerProviderEvent,
Conditions: dataprovider.EventConditions{
ProviderEvents: []string{"add"},
},
Actions: []dataprovider.EventAction{
{
BaseEventAction: dataprovider.BaseEventAction{
Name: action4.Name,
},
Order: 1,
},
},
}
rule1, _, err := httpdtest.AddEventRule(r1, http.StatusCreated)
assert.NoError(t, err)
rule2, _, err := httpdtest.AddEventRule(r2, http.StatusCreated)
assert.NoError(t, err)
rule3, _, err := httpdtest.AddEventRule(r3, http.StatusCreated)
assert.NoError(t, err)
rule4, _, err := httpdtest.AddEventRule(r4, http.StatusCreated)
assert.NoError(t, err)
rule5, _, err := httpdtest.AddEventRule(r5, http.StatusCreated)
assert.NoError(t, err)
folderMappedPath := filepath.Join(os.TempDir(), "folder")
err = os.MkdirAll(folderMappedPath, os.ModePerm)
assert.NoError(t, err)
err = os.WriteFile(filepath.Join(folderMappedPath, "file.txt"), []byte("1"), 0666)
assert.NoError(t, err)
folder, _, err := httpdtest.AddFolder(vfs.BaseVirtualFolder{
Name: "test folder",
MappedPath: folderMappedPath,
}, http.StatusCreated)
assert.NoError(t, err)
assert.Eventually(t, func() bool {
folderGet, _, err := httpdtest.GetFolderByName(folder.Name, http.StatusOK)
if err != nil {
return false
}
return folderGet.UsedQuotaFiles == 1 && folderGet.UsedQuotaSize == 1
}, 2*time.Second, 100*time.Millisecond)
u := getTestUser()
u.Filters.DisableFsChecks = true
user, _, err := httpdtest.AddUser(u, http.StatusCreated)
assert.NoError(t, err)
conn, client, err := getSftpClient(user)
if assert.NoError(t, err) {
defer conn.Close()
defer client.Close()
// check initial directories creation
for _, dir := range dirsToCreate {
assert.Eventually(t, func() bool {
_, err := client.Stat(dir)
return err == nil
}, 2*time.Second, 100*time.Millisecond)
}
// upload a file and check the sync rename
size := int64(32768)
err = writeSFTPFileNoCheck(path.Join("basedir", testFileName), size, client)
assert.NoError(t, err)
_, err = client.Stat(path.Join("basedir", testFileName))
assert.Error(t, err)
info, err := client.Stat(testFileName + "_renamed")
if assert.NoError(t, err) {
assert.Equal(t, size, info.Size())
}
assert.NoError(t, err)
assert.Eventually(t, func() bool {
userGet, _, err := httpdtest.GetUserByUsername(user.Username, http.StatusOK)
if err != nil {
return false
}
return userGet.UsedQuotaFiles == 1 && userGet.UsedQuotaSize == size
}, 2*time.Second, 100*time.Millisecond)
for i := 0; i < 2; i++ {
err = client.Mkdir(testFileName)
assert.NoError(t, err)
assert.Eventually(t, func() bool {
_, err = client.Stat(testFileName + "_renamed")
return err != nil
}, 2*time.Second, 100*time.Millisecond)
err = client.RemoveDirectory(testFileName)
assert.NoError(t, err)
}
err = client.Mkdir(testFileName + "_renamed")
assert.NoError(t, err)
err = client.Mkdir(testFileName)
assert.NoError(t, err)
assert.Eventually(t, func() bool {
_, err = client.Stat(testFileName + "_renamed")
return err != nil
}, 2*time.Second, 100*time.Millisecond)
}
_, err = httpdtest.RemoveUser(user, http.StatusOK)
assert.NoError(t, err)
err = os.RemoveAll(user.GetHomeDir())
assert.NoError(t, err)
_, err = httpdtest.RemoveFolder(folder, http.StatusOK)
assert.NoError(t, err)
err = os.RemoveAll(folderMappedPath)
assert.NoError(t, err)
_, err = httpdtest.RemoveEventRule(rule1, http.StatusOK)
assert.NoError(t, err)
_, err = httpdtest.RemoveEventRule(rule2, http.StatusOK)
assert.NoError(t, err)
_, err = httpdtest.RemoveEventRule(rule3, http.StatusOK)
assert.NoError(t, err)
_, err = httpdtest.RemoveEventRule(rule4, http.StatusOK)
assert.NoError(t, err)
_, err = httpdtest.RemoveEventRule(rule5, http.StatusOK)
assert.NoError(t, err)
_, err = httpdtest.RemoveEventAction(action1, http.StatusOK)
assert.NoError(t, err)
_, err = httpdtest.RemoveEventAction(action2, http.StatusOK)
assert.NoError(t, err)
_, err = httpdtest.RemoveEventAction(action3, http.StatusOK)
assert.NoError(t, err)
_, err = httpdtest.RemoveEventAction(action4, http.StatusOK)
assert.NoError(t, err)
_, err = httpdtest.RemoveEventAction(action5, http.StatusOK)
assert.NoError(t, err)
}
func TestSyncUploadAction(t *testing.T) {
if runtime.GOOS == osWindows {
t.Skip("this test is not available on Windows")

9
internal/dataprovider/actions.go
View file

@ -52,6 +52,10 @@ const (
actionObjectEventRule = "event_rule"
)
var (
actionsConcurrencyGuard = make(chan struct{}, 100)
)
func executeAction(operation, executor, ip, objectType, objectName string, object plugin.Renderer) {
if plugin.Handler.HasNotifiers() {
plugin.Handler.NotifyProviderEvent(&notifier.ProviderEvent{
@ -75,6 +79,11 @@ func executeAction(operation, executor, ip, objectType, objectName string, objec
}
go func() {
actionsConcurrencyGuard <- struct{}{}
defer func() {
<-actionsConcurrencyGuard
}()
dataAsJSON, err := object.RenderAsJSON(operation != operationDelete)
if err != nil {
providerLog(logger.LevelError, "unable to serialize user as JSON for operation %#v: %v", operation, err)

5
internal/dataprovider/dataprovider.go
View file

@ -3591,6 +3591,11 @@ func ExecutePostLoginHook(user *User, loginMethod, ip, protocol string, err erro
}
go func() {
actionsConcurrencyGuard <- struct{}{}
defer func() {
<-actionsConcurrencyGuard
}()
status := "0"
if err == nil {
status = "1"

201
internal/dataprovider/eventrule.go
View file

@ -41,12 +41,13 @@ const (
ActionTypeFolderQuotaReset
ActionTypeTransferQuotaReset
ActionTypeDataRetentionCheck
ActionTypeFilesystem
)
var (
supportedEventActions = []int{ActionTypeHTTP, ActionTypeCommand, ActionTypeEmail, ActionTypeBackup,
ActionTypeUserQuotaReset, ActionTypeFolderQuotaReset, ActionTypeTransferQuotaReset,
ActionTypeDataRetentionCheck}
ActionTypeDataRetentionCheck, ActionTypeFilesystem}
)
func isActionTypeValid(action int) bool {
@ -69,6 +70,8 @@ func getActionTypeAsString(action int) string {
return "Transfer quota reset"
case ActionTypeDataRetentionCheck:
return "Data retention check"
case ActionTypeFilesystem:
return "Filesystem"
default:
return "Command"
}
@ -102,6 +105,32 @@ func getTriggerTypeAsString(trigger int) string {
}
}
// Supported filesystem actions
const (
FilesystemActionRename = iota + 1
FilesystemActionDelete
FilesystemActionMkdirs
)
var (
supportedFsActions = []int{FilesystemActionRename, FilesystemActionDelete, FilesystemActionMkdirs}
)
func isFilesystemActionValid(value int) bool {
return util.Contains(supportedFsActions, value)
}
func getFsActionTypeAsString(value int) string {
switch value {
case FilesystemActionRename:
return "Rename"
case FilesystemActionDelete:
return "Delete"
default:
return "Create directories"
}
}
// TODO: replace the copied strings with shared constants
var (
// SupportedFsEvents defines the supported filesystem events
@ -112,8 +141,8 @@ var (
SupportedRuleConditionProtocols = []string{"SFTP", "SCP", "SSH", "FTP", "DAV", "HTTP", "HTTPShare",
"OIDC"}
// SupporteRuleConditionProviderObjects defines the supported provider objects for rule conditions
SupporteRuleConditionProviderObjects = []string{actionObjectUser, actionObjectGroup, actionObjectAdmin,
actionObjectAPIKey, actionObjectShare, actionObjectEventRule, actionObjectEventAction}
SupporteRuleConditionProviderObjects = []string{actionObjectUser, actionObjectFolder, actionObjectGroup,
actionObjectAdmin, actionObjectAPIKey, actionObjectShare, actionObjectEventRule, actionObjectEventAction}
// SupportedHTTPActionMethods defines the supported methods for HTTP actions
SupportedHTTPActionMethods = []string{http.MethodPost, http.MethodGet, http.MethodPut}
)
@ -122,6 +151,7 @@ var (
var (
EventActionTypes []EnumMapping
EventTriggerTypes []EnumMapping
FsActionTypes []EnumMapping
)
func init() {
@ -137,6 +167,12 @@ func init() {
Name: getTriggerTypeAsString(t),
})
}
for _, t := range supportedFsActions {
FsActionTypes = append(FsActionTypes, EnumMapping{
Value: t,
Name: getFsActionTypeAsString(t),
})
}
}
// EnumMapping defines a mapping between enum values and names
@ -331,12 +367,120 @@ func (c *EventActionDataRetentionConfig) validate() error {
return nil
}
// EventActionFilesystemConfig defines the configuration for filesystem actions
type EventActionFilesystemConfig struct {
// Filesystem actions, see the above enum
Type int `json:"type,omitempty"`
// files/dirs to rename, key is the source and target the value
Renames []KeyValue `json:"renames,omitempty"`
// directories to create
MkDirs []string `json:"mkdirs,omitempty"`
// files/dirs to delete
Deletes []string `json:"deletes,omitempty"`
}
// GetDeletesAsString returns the list of items to delete as comma separated string.
// Using a pointer receiver will not work in web templates
func (c EventActionFilesystemConfig) GetDeletesAsString() string {
return strings.Join(c.Deletes, ",")
}
// GetMkDirsAsString returns the list of directories to create as comma separated string.
// Using a pointer receiver will not work in web templates
func (c EventActionFilesystemConfig) GetMkDirsAsString() string {
return strings.Join(c.MkDirs, ",")
}
func (c *EventActionFilesystemConfig) validateRenames() error {
if len(c.Renames) == 0 {
return util.NewValidationError("no items to rename specified")
}
for idx, kv := range c.Renames {
key := strings.TrimSpace(kv.Key)
value := strings.TrimSpace(kv.Value)
if key == "" || value == "" {
return util.NewValidationError("invalid items to rename")
}
key = util.CleanPath(key)
value = util.CleanPath(value)
if key == value {
return util.NewValidationError("rename source and target cannot be equal")
}
if key == "/" || value == "/" {
return util.NewValidationError("renaming the root directory is not allowed")
}
c.Renames[idx] = KeyValue{
Key: key,
Value: value,
}
}
return nil
}
func (c *EventActionFilesystemConfig) validate() error {
if !isFilesystemActionValid(c.Type) {
return util.NewValidationError(fmt.Sprintf("invalid filesystem action type: %d", c.Type))
}
switch c.Type {
case FilesystemActionRename:
c.MkDirs = nil
c.Deletes = nil
if err := c.validateRenames(); err != nil {
return err
}
case FilesystemActionDelete:
c.Renames = nil
c.MkDirs = nil
if len(c.Deletes) == 0 {
return util.NewValidationError("no item to delete specified")
}
for idx, val := range c.Deletes {
val = strings.TrimSpace(val)
if val == "" {
return util.NewValidationError("invalid item to delete")
}
c.Deletes[idx] = util.CleanPath(val)
}
c.Deletes = util.RemoveDuplicates(c.Deletes, false)
case FilesystemActionMkdirs:
c.Renames = nil
c.Deletes = nil
if len(c.MkDirs) == 0 {
return util.NewValidationError("no directory to create specified")
}
for idx, val := range c.MkDirs {
val = strings.TrimSpace(val)
if val == "" {
return util.NewValidationError("invalid directory to create")
}
c.MkDirs[idx] = util.CleanPath(val)
}
c.MkDirs = util.RemoveDuplicates(c.MkDirs, false)
}
return nil
}
func (c *EventActionFilesystemConfig) getACopy() EventActionFilesystemConfig {
mkdirs := make([]string, len(c.MkDirs))
copy(mkdirs, c.MkDirs)
deletes := make([]string, len(c.Deletes))
copy(deletes, c.Deletes)
return EventActionFilesystemConfig{
Type: c.Type,
Renames: cloneKeyValues(c.Renames),
MkDirs: mkdirs,
Deletes: deletes,
}
}
// BaseEventActionOptions defines the supported configuration options for a base event actions
type BaseEventActionOptions struct {
HTTPConfig EventActionHTTPConfig `json:"http_config"`
CmdConfig EventActionCommandConfig `json:"cmd_config"`
EmailConfig EventActionEmailConfig `json:"email_config"`
RetentionConfig EventActionDataRetentionConfig `json:"retention_config"`
FsConfig EventActionFilesystemConfig `json:"fs_config"`
}
func (o *BaseEventActionOptions) getACopy() BaseEventActionOptions {
@ -378,6 +522,7 @@ func (o *BaseEventActionOptions) getACopy() BaseEventActionOptions {
RetentionConfig: EventActionDataRetentionConfig{
Folders: folders,
},
FsConfig: o.FsConfig.getACopy(),
}
}
@ -407,27 +552,38 @@ func (o *BaseEventActionOptions) validate(action int, name string) error {
o.CmdConfig = EventActionCommandConfig{}
o.EmailConfig = EventActionEmailConfig{}
o.RetentionConfig = EventActionDataRetentionConfig{}
o.FsConfig = EventActionFilesystemConfig{}
return o.HTTPConfig.validate(name)
case ActionTypeCommand:
o.HTTPConfig = EventActionHTTPConfig{}
o.EmailConfig = EventActionEmailConfig{}
o.RetentionConfig = EventActionDataRetentionConfig{}
o.FsConfig = EventActionFilesystemConfig{}
return o.CmdConfig.validate()
case ActionTypeEmail:
o.HTTPConfig = EventActionHTTPConfig{}
o.CmdConfig = EventActionCommandConfig{}
o.RetentionConfig = EventActionDataRetentionConfig{}
o.FsConfig = EventActionFilesystemConfig{}
return o.EmailConfig.validate()
case ActionTypeDataRetentionCheck:
o.HTTPConfig = EventActionHTTPConfig{}
o.CmdConfig = EventActionCommandConfig{}
o.EmailConfig = EventActionEmailConfig{}
o.FsConfig = EventActionFilesystemConfig{}
return o.RetentionConfig.validate()
case ActionTypeFilesystem:
o.HTTPConfig = EventActionHTTPConfig{}
o.CmdConfig = EventActionCommandConfig{}
o.EmailConfig = EventActionEmailConfig{}
o.RetentionConfig = EventActionDataRetentionConfig{}
return o.FsConfig.validate()
default:
o.HTTPConfig = EventActionHTTPConfig{}
o.CmdConfig = EventActionCommandConfig{}
o.EmailConfig = EventActionEmailConfig{}
o.RetentionConfig = EventActionDataRetentionConfig{}
o.FsConfig = EventActionFilesystemConfig{}
}
return nil
}
@ -844,6 +1000,45 @@ func (r *EventRule) validate() error {
return nil
}
// CheckActionsConsistency returns an error if the actions cannot be executed
func (r *EventRule) CheckActionsConsistency(providerObjectType string) error {
switch r.Trigger {
case EventTriggerProviderEvent:
// user quota reset, transfer quota reset, data retention check and filesystem actions
// can be executed only if we modify a user. They will be executed for the
// affected user. Folder quota reset can be executed only for folders.
userSpecificActions := []int{ActionTypeUserQuotaReset, ActionTypeTransferQuotaReset,
ActionTypeDataRetentionCheck, ActionTypeFilesystem}
for _, action := range r.Actions {
if util.Contains(userSpecificActions, action.Type) && providerObjectType != actionObjectUser {
return fmt.Errorf("action %q, type %q is only supported for provider user events",
action.Name, getActionTypeAsString(action.Type))
}
if action.Type == ActionTypeFolderQuotaReset && providerObjectType != actionObjectFolder {
return fmt.Errorf("action %q, type %q is only supported for provider folder events",
action.Name, getActionTypeAsString(action.Type))
}
}
case EventTriggerFsEvent:
// folder quota reset cannot be executed
for _, action := range r.Actions {
if action.Type == ActionTypeFolderQuotaReset {
return fmt.Errorf("action %q, type %q is not supported for filesystem events",
action.Name, getActionTypeAsString(action.Type))
}
}
case EventTriggerSchedule:
// to execute a filesystem action we need a user
for _, action := range r.Actions {
if action.Type == ActionTypeFilesystem {
return fmt.Errorf("action %q, type %q is not supported for scheduled events",
action.Name, getActionTypeAsString(action.Type))
}
}
}
return nil
}
// PrepareForRendering prepares an EventRule for rendering.
// It hides confidential data and set to nil the empty secrets
// so they are not serialized

90
internal/httpd/httpd_test.go
View file

@ -1592,6 +1592,56 @@ func TestEventActionValidation(t *testing.T) {
_, resp, err = httpdtest.AddEventAction(action, http.StatusBadRequest)
assert.NoError(t, err)
assert.Contains(t, string(resp), "invalid folder retention")
action.Type = dataprovider.ActionTypeFilesystem
action.Options.FsConfig = dataprovider.EventActionFilesystemConfig{
Type: dataprovider.FilesystemActionRename,
}
_, resp, err = httpdtest.AddEventAction(action, http.StatusBadRequest)
assert.NoError(t, err)
assert.Contains(t, string(resp), "no items to rename specified")
action.Options.FsConfig.Renames = []dataprovider.KeyValue{
{
Key: "",
Value: "/adir",
},
}
_, resp, err = httpdtest.AddEventAction(action, http.StatusBadRequest)
assert.NoError(t, err)
assert.Contains(t, string(resp), "invalid items to rename")
action.Options.FsConfig.Renames = []dataprovider.KeyValue{
{
Key: "adir",
Value: "/adir",
},
}
_, resp, err = httpdtest.AddEventAction(action, http.StatusBadRequest)
assert.NoError(t, err)
assert.Contains(t, string(resp), "rename source and target cannot be equal")
action.Options.FsConfig.Renames = []dataprovider.KeyValue{
{
Key: "/",
Value: "/dir",
},
}
_, resp, err = httpdtest.AddEventAction(action, http.StatusBadRequest)
assert.NoError(t, err)
assert.Contains(t, string(resp), "renaming the root directory is not allowed")
action.Options.FsConfig.Type = dataprovider.FilesystemActionMkdirs
_, resp, err = httpdtest.AddEventAction(action, http.StatusBadRequest)
assert.NoError(t, err)
assert.Contains(t, string(resp), "no directory to create specified")
action.Options.FsConfig.MkDirs = []string{"dir1", ""}
_, resp, err = httpdtest.AddEventAction(action, http.StatusBadRequest)
assert.NoError(t, err)
assert.Contains(t, string(resp), "invalid directory to create")
action.Options.FsConfig.Type = dataprovider.FilesystemActionDelete
_, resp, err = httpdtest.AddEventAction(action, http.StatusBadRequest)
assert.NoError(t, err)
assert.Contains(t, string(resp), "no item to delete specified")
action.Options.FsConfig.Deletes = []string{"item1", ""}
_, resp, err = httpdtest.AddEventAction(action, http.StatusBadRequest)
assert.NoError(t, err)
assert.Contains(t, string(resp), "invalid item to delete")
}
func TestEventRuleValidation(t *testing.T) {
@ -18550,6 +18600,7 @@ func TestWebEventAction(t *testing.T) {
form := make(url.Values)
form.Set("name", action.Name)
form.Set("description", action.Description)
form.Set("fs_action_type", "0")
form.Set("type", "a")
req, err := http.NewRequest(http.MethodPost, webAdminEventActionPath, bytes.NewBuffer([]byte(form.Encode())))
assert.NoError(t, err)
@ -18806,6 +18857,45 @@ func TestWebEventAction(t *testing.T) {
}
}
}
action.Type = dataprovider.ActionTypeFilesystem
action.Options.FsConfig = dataprovider.EventActionFilesystemConfig{
Type: dataprovider.FilesystemActionMkdirs,
MkDirs: []string{"a ", " a/b"},
}
form.Set("type", fmt.Sprintf("%d", action.Type))
form.Set("fs_mkdir_paths", strings.Join(action.Options.FsConfig.MkDirs, ","))
form.Set("fs_action_type", "invalid")
req, err = http.NewRequest(http.MethodPost, path.Join(webAdminEventActionPath, action.Name),
bytes.NewBuffer([]byte(form.Encode())))
assert.NoError(t, err)
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
setJWTCookieForReq(req, webToken)
rr = executeRequest(req)
checkResponseCode(t, http.StatusOK, rr)
assert.Contains(t, rr.Body.String(), "invalid fs action type")
form.Set("fs_action_type", fmt.Sprintf("%d", action.Options.FsConfig.Type))
req, err = http.NewRequest(http.MethodPost, path.Join(webAdminEventActionPath, action.Name),
bytes.NewBuffer([]byte(form.Encode())))
assert.NoError(t, err)
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
setJWTCookieForReq(req, webToken)
rr = executeRequest(req)
checkResponseCode(t, http.StatusSeeOther, rr)
// check the update
actionGet, _, err = httpdtest.GetEventActionByName(action.Name, http.StatusOK)
assert.NoError(t, err)
assert.Equal(t, action.Type, actionGet.Type)
if assert.Len(t, actionGet.Options.FsConfig.MkDirs, 2) {
for _, dir := range actionGet.Options.FsConfig.MkDirs {
switch dir {
case "/a":
case "/a/b":
default:
t.Errorf("unexpected dir path %v", dir)
}
}
}
req, err = http.NewRequest(http.MethodDelete, path.Join(webAdminEventActionPath, action.Name), nil)
assert.NoError(t, err)

12
internal/httpd/webadmin.go
View file

@ -304,6 +304,7 @@ type eventActionPage struct {
basePage
Action dataprovider.BaseEventAction
ActionTypes []dataprovider.EnumMapping
FsActions []dataprovider.EnumMapping
HTTPMethods []string
RedactedSecret string
Error string
@ -905,6 +906,7 @@ func (s *httpdServer) renderEventActionPage(w http.ResponseWriter, r *http.Reque
basePage: s.getBasePageData(title, currentURL, r),
Action: action,
ActionTypes: dataprovider.EventActionTypes,
FsActions: dataprovider.FsActionTypes,
HTTPMethods: dataprovider.SupportedHTTPActionMethods,
RedactedSecret: redactedSecret,
Error: error,
@ -1873,6 +1875,10 @@ func getEventActionOptionsFromPostFields(r *http.Request) (dataprovider.BaseEven
if err != nil {
return dataprovider.BaseEventActionOptions{}, err
}
fsActionType, err := strconv.Atoi(r.Form.Get("fs_action_type"))
if err != nil {
return dataprovider.BaseEventActionOptions{}, fmt.Errorf("invalid fs action type: %w", err)
}
options := dataprovider.BaseEventActionOptions{
HTTPConfig: dataprovider.EventActionHTTPConfig{
Endpoint: r.Form.Get("http_endpoint"),
@ -1898,6 +1904,12 @@ func getEventActionOptionsFromPostFields(r *http.Request) (dataprovider.BaseEven
RetentionConfig: dataprovider.EventActionDataRetentionConfig{
Folders: foldersRetention,
},
FsConfig: dataprovider.EventActionFilesystemConfig{
Type: fsActionType,
Renames: getKeyValsFromPostFields(r, "fs_rename_source", "fs_rename_target"),
Deletes: strings.Split(strings.ReplaceAll(r.Form.Get("fs_delete_paths"), " ", ""), ","),
MkDirs: strings.Split(strings.ReplaceAll(r.Form.Get("fs_mkdir_paths"), " ", ""), ","),
},
}
return options, nil
}

29
internal/httpdtest/httpdtest.go
View file

@ -1349,6 +1349,9 @@ func checkEventAction(expected, actual dataprovider.BaseEventAction) error {
if err := compareEventActionDataRetentionFields(expected.Options.RetentionConfig, actual.Options.RetentionConfig); err != nil {
return err
}
if err := compareEventActionFsConfigFields(expected.Options.FsConfig, actual.Options.FsConfig); err != nil {
return err
}
return compareEventActionHTTPConfigFields(expected.Options.HTTPConfig, actual.Options.HTTPConfig)
}
@ -2238,6 +2241,32 @@ func compareEventActionEmailConfigFields(expected, actual dataprovider.EventActi
return nil
}
func compareEventActionFsConfigFields(expected, actual dataprovider.EventActionFilesystemConfig) error {
if expected.Type != actual.Type {
return errors.New("fs type mismatch")
}
if err := compareKeyValues(expected.Renames, actual.Renames); err != nil {
return errors.New("fs renames mismatch")
}
if len(expected.Deletes) != len(actual.Deletes) {
return errors.New("fs deletes mismatch")
}
for _, v := range expected.Deletes {
if !util.Contains(actual.Deletes, v) {
return errors.New("fs deletes content mismatch")
}
}
if len(expected.MkDirs) != len(actual.MkDirs) {
return errors.New("fs mkdirs mismatch")
}
for _, v := range expected.MkDirs {
if !util.Contains(actual.MkDirs, v) {
return errors.New("fs mkdir content mismatch")
}
}
return nil
}
func compareEventActionCmdConfigFields(expected, actual dataprovider.EventActionCommandConfig) error {
if expected.Cmd != actual.Cmd {
return errors.New("command mismatch")

6
internal/plugin/notifier.go
View file

@ -200,6 +200,9 @@ func (p *notifierPlugin) notifyFsAction(event *notifier.FsEvent) {
}
go func() {
Handler.addTask()
defer Handler.removeTask()
p.sendFsEvent(event)
}()
}
@ -211,6 +214,9 @@ func (p *notifierPlugin) notifyProviderAction(event *notifier.ProviderEvent, obj
}
go func() {
Handler.addTask()
defer Handler.removeTask()
objectAsJSON, err := object.RenderAsJSON(event.Action != "delete")
if err != nil {
logger.Warn(logSender, "", "unable to render user as json for action %v: %v", event.Action, err)

56
internal/plugin/plugin.go
View file

@ -96,35 +96,37 @@ type Manager struct {
closed int32
done chan bool
// List of configured plugins
Configs []Config `json:"plugins" mapstructure:"plugins"`
notifLock sync.RWMutex
notifiers []*notifierPlugin
kmsLock sync.RWMutex
kms []*kmsPlugin
authLock sync.RWMutex
auths []*authPlugin
searcherLock sync.RWMutex
searcher *searcherPlugin
metadaterLock sync.RWMutex
metadater *metadataPlugin
ipFilterLock sync.RWMutex
filter *ipFilterPlugin
authScopes int
hasSearcher bool
hasMetadater bool
hasNotifiers bool
hasAuths bool
hasIPFilter bool
Configs []Config `json:"plugins" mapstructure:"plugins"`
notifLock sync.RWMutex
notifiers []*notifierPlugin
kmsLock sync.RWMutex
kms []*kmsPlugin
authLock sync.RWMutex
auths []*authPlugin
searcherLock sync.RWMutex
searcher *searcherPlugin
metadaterLock sync.RWMutex
metadater *metadataPlugin
ipFilterLock sync.RWMutex
filter *ipFilterPlugin
authScopes int
hasSearcher bool
hasMetadater bool
hasNotifiers bool
hasAuths bool
hasIPFilter bool
concurrencyGuard chan struct{}
}
// Initialize initializes the configured plugins
func Initialize(configs []Config, logLevel string) error {
logger.Debug(logSender, "", "initialize")
Handler = Manager{
Configs: configs,
done: make(chan bool),
closed: 0,
authScopes: -1,
Configs: configs,
done: make(chan bool),
closed: 0,
authScopes: -1,
concurrencyGuard: make(chan struct{}, 250),
}
setLogLevel(logLevel)
if len(configs) == 0 {
@ -699,6 +701,14 @@ func (m *Manager) restartIPFilterPlugin(config Config) {
m.ipFilterLock.Unlock()
}
func (m *Manager) addTask() {
m.concurrencyGuard <- struct{}{}
}
func (m *Manager) removeTask() {
<-m.concurrencyGuard
}
// Cleanup releases all the active plugins
func (m *Manager) Cleanup() {
logger.Debug(logSender, "", "cleanup")

134
templates/webadmin/eventaction.html
View file

@ -404,6 +404,89 @@ along with this program. If not, see <https://www.gnu.org/licenses/>.
</div>
</div>
<div class="form-group row action-type action-fs">
<label for="idFsActionType" class="col-sm-2 col-form-label">Fs action</label>
<div class="col-sm-10">
<select class="form-control selectpicker" id="idFsActionType" name="fs_action_type" onchange="onFsActionChanged(this.value)">
{{- range .FsActions}}
<option value="{{.Value}}" {{if eq $.Action.Options.FsConfig.Type .Value }}selected{{end}}>{{.Name}}</option>
{{- end}}
</select>
</div>
</div>
<div class="card bg-light mb-3 action-type action-fs-type action-fs-rename">
<div class="card-header">
<b>Rename</b>
</div>
<div class="card-body">
<h6 class="card-title mb-4">Paths to rename as seen by SFTPGo users. Placeholders are supported. The required permissions are granted automatically</h6>
<div class="form-group row">
<div class="col-md-12 form_field_fs_rename_outer">
{{range $idx, $val := .Action.Options.FsConfig.Renames}}
<div class="row form_field_fs_rename_outer_row">
<div class="form-group col-md-5">
<input type="text" class="form-control" id="idFsRenameSource{{$idx}}" name="fs_rename_source{{$idx}}" placeholder="Source path" value="{{$val.Key}}">
</div>
<div class="form-group col-md-5">
<input type="text" class="form-control" id="idFsRenameTarget{{$idx}}" name="fs_rename_target{{$idx}}" placeholder="Target path" value="{{$val.Value}}">
</div>
<div class="form-group col-md-1"></div>
<div class="form-group col-md-1">
<button class="btn btn-circle btn-danger remove_fs_rename_btn_frm_field">
<i class="fas fa-trash"></i>
</button>
</div>
</div>
{{else}}
<div class="row form_field_fs_rename_outer_row">
<div class="form-group col-md-5">
<input type="text" class="form-control" id="idFsRenameSource0" name="fs_rename_source0" placeholder="Source path" value="">
</div>
<div class="form-group col-md-5">
<input type="text" class="form-control" id="idFsRenameTarget0" name="fs_rename_target0" placeholder="Target path" value="">
</div>
<div class="form-group col-md-1"></div>
<div class="form-group col-md-1">
<button class="btn btn-circle btn-danger remove_fs_rename_btn_frm_field">
<i class="fas fa-trash"></i>
</button>
</div>
</div>
{{end}}
</div>
</div>
<div class="row mx-1">
<button type="button" class="btn btn-secondary add_new_fs_rename_field_btn">
<i class="fas fa-plus"></i> Add new
</button>
</div>
</div>
</div>
<div class="form-group row action-type action-fs-type action-fs-delete">
<label for="idFsDelete" class="col-sm-2 col-form-label">Paths</label>
<div class="col-sm-10">
<textarea class="form-control" id="idFsDelete" name="fs_delete_paths" rows="2"
aria-describedby="fsDeleteHelpBlock">{{.Action.Options.FsConfig.GetDeletesAsString}}</textarea>
<small id="fsDeleteHelpBlock" class="form-text text-muted">
Comma separated paths to delete as seen by SFTPGo users. Placeholders are supported. The required permissions are granted automatically
</small>
</div>
</div>
<div class="form-group row action-type action-fs-type action-fs-mkdir">
<label for="idFsMkdir" class="col-sm-2 col-form-label">Paths</label>
<div class="col-sm-10">
<textarea class="form-control" id="idFsMkdir" name="fs_mkdir_paths" rows="2"
aria-describedby="fsMkdirHelpBlock">{{.Action.Options.FsConfig.GetMkDirsAsString}}</textarea>
<small id="fsMkdirHelpBlock" class="form-text text-muted">
Comma separated directories paths to create as seen by SFTPGo users. Placeholders are supported. The required permissions are granted automatically
</small>
</div>
</div>
<input type="hidden" name="_form_token" value="{{.CSRFToken}}">
<div class="col-sm-12 text-right px-0">
<button type="submit" class="btn btn-primary mt-3 ml-3 px-5" name="form_action" value="submit">Submit</button>
@ -596,6 +679,33 @@ along with this program. If not, see <https://www.gnu.org/licenses/>.
$(this).closest(".form_field_data_retention_outer_row").remove();
});
$("body").on("click", ".add_new_fs_rename_field_btn", function () {
var index = $(".form_field_fs_rename_outer").find(".form_field_fs_rename_outer_row").length;
while (document.getElementById("idFsRenameSource"+index) != null){
index++;
}
$(".form_field_fs_rename_outer").append(`
<div class="row form_field_fs_rename_outer_row">
<div class="form-group col-md-5">
<input type="text" class="form-control" id="idFsRenameSource${index}" name="fs_rename_source${index}" placeholder="Source path" value="">
</div>
<div class="form-group col-md-5">
<input type="text" class="form-control" id="idFsRenameTarget${index}" name="fs_rename_target${index}" placeholder="Target path" value="">
</div>
<div class="form-group col-md-1"></div>
<div class="form-group col-md-1">
<button class="btn btn-circle btn-danger remove_fs_rename_btn_frm_field">
<i class="fas fa-trash"></i>
</button>
</div>
</div>
`);
});
$("body").on("click", ".remove_fs_rename_btn_frm_field", function () {
$(this).closest(".form_field_fs_rename_outer_row").remove();
});
function onTypeChanged(val){
$('.action-type').hide();
switch (val) {
@ -615,11 +725,35 @@ along with this program. If not, see <https://www.gnu.org/licenses/>.
case 8:
$('.action-dataretention').show();
break;
case '9':
case 9:
$('.action-fs').show();
onFsActionChanged($("#idFsActionType").val());
break;
}
}
function onFsActionChanged(val){
$('.action-fs-type').hide();
switch (val) {
case '1':
case 1:
$('.action-fs-rename').show();
break;
case '2':
case 2:
$('.action-fs-delete').show();
break;
case '3':
case 3:
$('.action-fs-mkdir').show();
break;
}
}
$(document).ready(function () {
onTypeChanged('{{.Action.Type}}');
onFsActionChanged('{{.Action.Options.FsConfig.Type}}');
});
</script>
{{end}}