beenull/sftpgo
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

eventmanager: add copy action

Browse source 
refactor sftpgo-copy and sftpgo-remove commands

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
This commit is contained in:
Nicola Murino 2022-12-27 18:51:53 +01:00
parent e5a8220b8a
commit ea4c4dd57f
No known key found for this signature in database
GPG key ID: 935D2952DEC4EECF
28 changed files with 1208 additions and 658 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

5
docs/custom-actions.md
View file

@ -21,6 +21,7 @@ The following `actions` are supported:
- `mkdir`
- `rmdir`
- `ssh_cmd`
- `copy`
The `upload` condition includes both uploads to new files and overwrite of existing ones. If an upload is aborted for quota limits SFTPGo tries to remove the partial file, so if the notification reports a zero size file and a quota exceeded error the file has been deleted. The `ssh_cmd` condition will be triggered after a command is successfully executed via SSH. `scp` will trigger the `download` and `upload` conditions and not `ssh_cmd`. The `first-download` and `first-upload` action are executed only if no error occour and they don't exclude the `download` and `upload` notifications, so you will get both the `first-upload` and `upload` notification after the first successful upload and the same for the first successful download.
For cloud backends directories are virtual, they are created implicitly when you upload a file and are implicitly removed when the last file within a directory is removed. The `mkdir` and `rmdir` notifications are sent only when a directory is explicitly created or removed.
@ -40,7 +41,7 @@ If the `hook` defines a path to an external program, then this program can read
- `SFTPGO_ACTION_VIRTUAL_PATH`, virtual path, seen by SFTPGo users
- `SFTPGO_ACTION_VIRTUAL_TARGET`, virtual target path, seen by SFTPGo users
- `SFTPGO_ACTION_SSH_CMD`, non-empty for `ssh_cmd` `SFTPGO_ACTION`
- `SFTPGO_ACTION_FILE_SIZE`, non-zero for `pre-upload`,`upload`, `download` and `delete` actions if the file size is greater than `0`
- `SFTPGO_ACTION_FILE_SIZE`, non-zero for `pre-upload`, `upload`, `download`, `delete`, and `copy` actions if the file size is greater than `0`
- `SFTPGO_ACTION_FS_PROVIDER`, `0` for local filesystem, `1` for S3 backend, `2` for Google Cloud Storage (GCS) backend, `3` for Azure Blob Storage backend, `4` for local encrypted backend, `5` for SFTP backend
- `SFTPGO_ACTION_BUCKET`, non-empty for S3, GCS and Azure backends
- `SFTPGO_ACTION_ENDPOINT`, non-empty for S3, SFTP and Azure backend if configured
@ -64,7 +65,7 @@ If the `hook` defines an HTTP URL then this URL will be invoked as HTTP POST. Th
- `virtual_path`, string, virtual path, seen by SFTPGo users
- `virtual_target_path`, string, virtual target path, seen by SFTPGo users
- `ssh_cmd`, string, included for `ssh_cmd` action
- `file_size`, int64, included for `pre-upload`, `upload`, `download`, `delete` actions if the file size is greater than `0`
- `file_size`, int64, included for `pre-upload`, `upload`, `download`, `delete` and `copy` actions if the file size is greater than `0`
- `fs_provider`, integer, `0` for local filesystem, `1` for S3 backend, `2` for Google Cloud Storage (GCS) backend, `3` for Azure Blob Storage backend, `4` for local encrypted backend, `5` for SFTP backend, `6` for HTTPFs backend
- `bucket`, string, included for S3, GCS and Azure backends
- `endpoint`, string, included for S3, SFTP and Azure backend if configured

1
docs/eventmanager.md
View file

@ -19,6 +19,7 @@ The following actions are supported:
- `Delete`. You can delete one or more files and directories.
- `Create directories`. You can create one or more directories including sub-directories.
- `Path exists`. Check if the specified path exists.
- `Copy`. You can copy one or more files or directories.
- `Compress paths`. You can compress (currently as zip) ore or more files and directories.
The following placeholders are supported:

2
docs/full-configuration.md
View file

@ -60,7 +60,7 @@ The configuration file contains the following sections:
- `idle_timeout`, integer. Time in minutes after which an idle client will be disconnected. 0 means disabled. Default: 15
- `upload_mode` integer. 0 means standard: the files are uploaded directly to the requested path. 1 means atomic: files are uploaded to a temporary path and renamed to the requested path when the client ends the upload. Atomic mode avoids problems such as a web server that serves partial files when the files are being uploaded. In atomic mode, if there is an upload error, the temporary file is deleted and so the requested upload path will not contain a partial file. 2 means atomic with resume support: same as atomic but if there is an upload error, the temporary file is renamed to the requested path and not deleted. This way, a client can reconnect and resume the upload. Ignored for cloud-based storage backends (uploads are always atomic and resume is not supported for these backends) and for SFTP backend if buffering is enabled. Default: 0
- `actions`, struct. It contains the command to execute and/or the HTTP URL to notify and the trigger conditions. See [Custom Actions](./custom-actions.md) for more details
- `execute_on`, list of strings. Valid values are `pre-download`, `download`, `pre-upload`, `upload`, `pre-delete`, `delete`, `rename`, `mkdir`, `rmdir`, `ssh_cmd`. Leave empty to disable actions.
- `execute_on`, list of strings. Valid values are `pre-download`, `download`, `first-download`, `pre-upload`, `upload`, `first-upload`, `pre-delete`, `delete`, `rename`, `mkdir`, `rmdir`, `ssh_cmd`, `copy`. Leave empty to disable actions.
- `execute_sync`, list of strings. Actions, defined in the `execute_on` list above, to be performed synchronously. The `pre-*` actions are always executed synchronously while the other ones are asynchronous. Executing an action synchronously means that SFTPGo will not return a result code to the client (which is waiting for it) until your hook have completed its execution. Leave empty to execute only the defined `pre-*` hook synchronously
- `hook`, string. Absolute path to the command to execute or HTTP URL to notify.
- `setstat_mode`, integer. 0 means "normal mode": requests for changing permissions, owner/group and access/modification times are executed. 1 means "ignore mode": requests for changing permissions, owner/group and access/modification times are silently ignored. 2 means "ignore mode if not supported": requests for changing permissions and owner/group are silently ignored for cloud filesystems and executed for local/SFTP filesystem. Requests for changing modification times are always executed for local/SFTP filesystems and are executed for cloud based filesystems if the target is a file and there is a metadata plugin available. A metadata plugin can be found [here](https://github.com/sftpgo/sftpgo-plugin-metadata).

6
docs/logs.md
View file

@ -20,10 +20,10 @@ The logs can be divided into the following categories:
- `username`, string
- `file_path` string
- `connection_id` string. Unique connection identifier
- `protocol` string. `SFTP`, `SCP`, `SSH`, `FTP`, `HTTP`, `DAV`, `DataRetention`
- `protocol` string. `SFTP`, `SCP`, `SSH`, `FTP`, `HTTP`, `DAV`, `DataRetention`, `EventAction`
- `ftp_mode`, string. `active` or `passive`. Included only for `FTP` protocol
- **"command logs"**, SFTP/SCP command logs:
- `sender` string. `Rename`, `Rmdir`, `Mkdir`, `Symlink`, `Remove`, `Chmod`, `Chown`, `Chtimes`, `Truncate`, `SSHCommand`
- `sender` string. `Rename`, `Rmdir`, `Mkdir`, `Symlink`, `Remove`, `Chmod`, `Chown`, `Chtimes`, `Truncate`, `Copy`, `SSHCommand`
- `level` string
- `local_addr` string. IP/port of the local address the connection arrived on. For example `127.0.0.1:1234`
- `remote_addr` string. IP and, optionally, port of the remote client. For example `127.0.0.1:1234` or `127.0.0.1`
@ -38,7 +38,7 @@ The logs can be divided into the following categories:
- `size` int64. Valid for sender `Truncate` otherwise -1
- `ssh_command`, string. Valid for sender `SSHCommand` otherwise empty
- `connection_id` string. Unique connection identifier
- `protocol` string. `SFTP`, `SCP` or `SSH`
- `protocol` string. `SFTP`, `SCP`, `SSH`, `FTP`, `HTTP`, `DAV`, `DataRetention`, `EventAction`
- **"http logs"**, REST API logs:
- `sender` string. `httpd`
- `level` string

4
docs/ssh-commands.md
View file

@ -37,8 +37,8 @@ SFTPGo supports the following built-in SSH commands:
- `scp`, SFTPGo implements the SCP protocol so we can support it for cloud filesystems too and we can avoid the other system commands limitations. SCP between two remote hosts is supported using the `-3` scp option. Wildcard expansion is not supported.
- `md5sum`, `sha1sum`, `sha256sum`, `sha384sum`, `sha512sum`. Useful to check message digests for uploaded files.
- `cd`, `pwd`. Some SFTP clients do not support the SFTP SSH_FXP_REALPATH packet type, so they use `cd` and `pwd` SSH commands to get the initial directory. Currently `cd` does nothing and `pwd` always returns the `/` path. These commands will work with any storage backend but keep in mind that to calculate the hash we need to read the whole file, for remote backends this means downloading the file, for the encrypted backend this means decrypting the file.
- `sftpgo-copy`. This is a built-in copy implementation. It allows server side copy for files and directories. The first argument is the source file/directory and the second one is the destination file/directory, for example `sftpgo-copy <src> <dst>`. The command will fail if the destination exists. Copy for directories spanning virtual folders is not supported. Only local filesystem is supported: recursive copy for Cloud Storage filesystems requires a new request for every file in any case, so a real server side copy is not possible.
- `sftpgo-remove`. This is a built-in remove implementation. It allows to remove single files and to recursively remove directories. The first argument is the file/directory to remove, for example `sftpgo-remove <dst>`. Only local and encrypted filesystems are supported: recursive remove for Cloud Storage filesystems requires a new request for every file in any case, so a server side remove is not possible.
- `sftpgo-copy`. This is a built-in copy implementation. It allows server side copy for files and directories. The first argument is the source file/directory and the second one is the destination file/directory, for example `sftpgo-copy <src> <dst>`. :warning: Copying directories that span virtual folders is supported but, for Cloud Storage filesystems, the remote copy API is not currently used.
- `sftpgo-remove`. This is a built-in remove implementation. It allows to remove single files and to recursively remove directories. The first argument is the file/directory to remove, for example `sftpgo-remove <dst>`. Removing directories spanning virtual folders is not supported.
The following SSH commands are enabled by default:

5
go.mod
View file

@ -48,7 +48,7 @@ require (
github.com/pquerna/otp v1.4.0
github.com/prometheus/client_golang v1.14.0
github.com/robfig/cron/v3 v3.0.1
github.com/rs/cors v1.8.3-0.20220619195839-da52b0701de5
github.com/rs/cors v1.8.3
github.com/rs/xid v1.4.0
github.com/rs/zerolog v1.28.0
github.com/sftpgo/sdk v0.1.3-0.20221217110036-383c1bb50fa0
@ -118,7 +118,7 @@ require (
github.com/hashicorp/yamux v0.1.1 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgservicefile v0.0.0-20200714003250-2b9c44734f2b // indirect
github.com/jackc/pgservicefile v0.0.0-20221227161230-091c0ba34f0a // indirect
github.com/jmespath/go-jmespath v0.4.0 // indirect
github.com/klauspost/cpuid/v2 v2.2.2 // indirect
github.com/kr/fs v0.1.0 // indirect
@ -171,5 +171,6 @@ require (
replace (
github.com/fclairamb/ftpserverlib => github.com/drakkan/ftpserverlib v0.0.0-20221203115213-ba73c775a9fd
github.com/jlaffaye/ftp => github.com/drakkan/ftp v0.0.0-20201114075148-9b9adce499a9
github.com/pkg/sftp => github.com/drakkan/sftp v0.0.0-20221225162142-08880975fb1e
golang.org/x/crypto => github.com/drakkan/crypto v0.0.0-20221223081523-be6917ff6f72
)

13
go.sum
View file

@ -544,6 +544,8 @@ github.com/drakkan/ftp v0.0.0-20201114075148-9b9adce499a9 h1:LPH1dEblAOO/LoG7yHP
github.com/drakkan/ftp v0.0.0-20201114075148-9b9adce499a9/go.mod h1:2lmrmq866uF2tnje75wQHzmPXhmSWUt7Gyx2vgK1RCU=
github.com/drakkan/ftpserverlib v0.0.0-20221203115213-ba73c775a9fd h1:wu/ys+33GwD9PyRO8QDCUpI2WBZtwFiDk8QkFPW8rhQ=
github.com/drakkan/ftpserverlib v0.0.0-20221203115213-ba73c775a9fd/go.mod h1:FHiqwx5L+7z3o7EXRtT6asSd1uO4yTqEljqFU9L+zVA=
github.com/drakkan/sftp v0.0.0-20221225162142-08880975fb1e h1:Eeg6op40DlnZOarl7OWX9t1wdjkhUHT2kPlSkSHOvLA=
github.com/drakkan/sftp v0.0.0-20221225162142-08880975fb1e/go.mod h1:tz1ryNURKu77RL+GuCzmoJYxQczL3wLNNpPWagdg4Qk=
github.com/drakkan/webdav v0.0.0-20221101181759-17ed21f9337b h1:B9z7XyDoVxLO4yEvnXgdvZ+0Uw9NA1qdD4KTSGmKcoQ=
github.com/drakkan/webdav v0.0.0-20221101181759-17ed21f9337b/go.mod h1:8opebuqUyBXrvl7Vo/S1Zzl9U0G1X2Ceud440eVuhUE=
github.com/dustin/go-humanize v0.0.0-20171111073723-bb3d318650d4/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk=
@ -1001,8 +1003,9 @@ github.com/jackc/pgproto3/v2 v2.0.0-rc3.0.20190831210041-4c03ce451f29/go.mod h1:
github.com/jackc/pgproto3/v2 v2.0.6/go.mod h1:WfJCnwN3HIg9Ish/j3sgWXnAfK8A9Y0bwXYU5xKaEdA=
github.com/jackc/pgproto3/v2 v2.1.1/go.mod h1:WfJCnwN3HIg9Ish/j3sgWXnAfK8A9Y0bwXYU5xKaEdA=
github.com/jackc/pgproto3/v2 v2.3.0/go.mod h1:WfJCnwN3HIg9Ish/j3sgWXnAfK8A9Y0bwXYU5xKaEdA=
github.com/jackc/pgservicefile v0.0.0-20200714003250-2b9c44734f2b h1:C8S2+VttkHFdOOCXJe+YGfa4vHYwlt4Zx+IVXQ97jYg=
github.com/jackc/pgservicefile v0.0.0-20200714003250-2b9c44734f2b/go.mod h1:vsD4gTJCa9TptPL8sPkXrLZ+hDuNrZCnj29CQpr4X1E=
github.com/jackc/pgservicefile v0.0.0-20221227161230-091c0ba34f0a h1:bbPeKD0xmW/Y25WS6cokEszi5g+S0QxI/d45PkRi7Nk=
github.com/jackc/pgservicefile v0.0.0-20221227161230-091c0ba34f0a/go.mod h1:5TJZWKEWniPve33vlWYSoGYefn3gLQRzjfDlhSJ9ZKM=
github.com/jackc/pgtype v0.0.0-20190421001408-4ed0de4755e0/go.mod h1:hdSHsc1V01CGwFsrv11mJRHWJ6aifDLfdV3aVjFF0zg=
github.com/jackc/pgtype v0.0.0-20190824184912-ab885b375b90/go.mod h1:KcahbBH1nCMSo2DXpzsoWOAfFkdEtEJpPbVLq8eE+mc=
github.com/jackc/pgtype v0.0.0-20190828014616-a8802b16cc59/go.mod h1:MWlu30kVJrUS8lot6TQqcg7mtthZ9T0EoIBFiJcmcyw=
@ -1335,10 +1338,6 @@ github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINE
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/profile v1.2.1/go.mod h1:hJw3o1OdXxsrSjjVksARp5W95eeEaEfptyVZyv6JUPA=
github.com/pkg/sftp v1.10.1/go.mod h1:lYOWFsE0bwd1+KfKJaKeuokY15vzFx25BLbzYYoAxZI=
github.com/pkg/sftp v1.13.1/go.mod h1:3HaPG6Dq1ILlpPZRO0HVMrsydcdLt6HRDccSgb87qRg=
github.com/pkg/sftp v1.13.6-0.20221020054726-e4133ab7e9bd h1:hg6yeLOCjHz1V8wUATWqczcXyIrm+5Fx5jKDxaB5HBs=
github.com/pkg/sftp v1.13.6-0.20221020054726-e4133ab7e9bd/go.mod h1:tz1ryNURKu77RL+GuCzmoJYxQczL3wLNNpPWagdg4Qk=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/posener/complete v1.1.1/go.mod h1:em0nMJCgc9GFtwrmVmEMR/ZL6WyhyjMBndrE9hABlRI=
@ -1426,8 +1425,8 @@ github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFR
github.com/rogpeppe/go-internal v1.6.1 h1:/FiVV8dS/e+YqF2JvO3yXRFbBLTIuSDkuC7aBOAvL+k=
github.com/rogpeppe/go-internal v1.6.1/go.mod h1:xXDCJY+GAPziupqXw64V24skbSoqbTEfhy4qGm1nDQc=
github.com/rs/cors v1.8.2/go.mod h1:XyqrcTp5zjWr1wsJ8PIRZssZ8b/WMcMf71DJnit4EMU=
github.com/rs/cors v1.8.3-0.20220619195839-da52b0701de5 h1:7PcjxKTsfGXpTMiTNNa1VllbsYSZJN5nhvVEWQMdX8Y=
github.com/rs/cors v1.8.3-0.20220619195839-da52b0701de5/go.mod h1:XyqrcTp5zjWr1wsJ8PIRZssZ8b/WMcMf71DJnit4EMU=
github.com/rs/cors v1.8.3 h1:O+qNyWn7Z+F9M0ILBHgMVPuB1xTOucVd5gtaYyXBpRo=
github.com/rs/cors v1.8.3/go.mod h1:XyqrcTp5zjWr1wsJ8PIRZssZ8b/WMcMf71DJnit4EMU=
github.com/rs/xid v1.2.1/go.mod h1:+uKXf+4Djp6Md1KODXJxgGQPKngRmWyn10oCKFzNHOQ=
github.com/rs/xid v1.4.0 h1:qd7wPTDkN6KQx2VmMBLrpHkiyQwgFXRnkOLacUiaSNY=
github.com/rs/xid v1.4.0/go.mod h1:trrq9SKmegXys3aeAKXMUTdJsYXVwGY3RLcfgqegfbg=

19
internal/common/actions.go
View file

@ -99,7 +99,7 @@ func ExecutePreAction(conn *BaseConnection, operation, filePath, virtualPath str
return handleUnconfiguredPreAction(operation)
}
event = newActionNotification(&conn.User, operation, filePath, virtualPath, "", "", "",
conn.protocol, conn.GetRemoteIP(), conn.ID, fileSize, openFlags, nil)
conn.protocol, conn.GetRemoteIP(), conn.ID, fileSize, openFlags, conn.getNotificationStatus(nil))
if hasNotifiersPlugin {
plugin.Handler.NotifyFsEvent(event)
}
@ -120,7 +120,7 @@ func ExecuteActionNotification(conn *BaseConnection, operation, filePath, virtua
return nil
}
notification := newActionNotification(&conn.User, operation, filePath, virtualPath, target, virtualTarget, sshCmd,
conn.protocol, conn.GetRemoteIP(), conn.ID, fileSize, 0, err)
conn.protocol, conn.GetRemoteIP(), conn.ID, fileSize, 0, conn.getNotificationStatus(err))
if hasNotifiersPlugin {
plugin.Handler.NotifyFsEvent(notification)
}
@ -174,8 +174,7 @@ func newActionNotification(
user *dataprovider.User,
operation, filePath, virtualPath, target, virtualTarget, sshCmd, protocol, ip, sessionID string,
fileSize int64,
openFlags int,
err error,
openFlags, status int,
) *notifier.FsEvent {
var bucket, endpoint string
@ -210,7 +209,7 @@ func newActionNotification(
FsProvider: int(fsConfig.Provider),
Bucket: bucket,
Endpoint: endpoint,
Status: getNotificationStatus(err),
Status: status,
Protocol: protocol,
IP: ip,
SessionID: sessionID,
@ -316,13 +315,3 @@ func notificationAsEnvVars(event *notifier.FsEvent) []string {
fmt.Sprintf("SFTPGO_ACTION_ROLE=%s", event.Role),
}
}
func getNotificationStatus(err error) int {
status := 1
if err == ErrQuotaExceeded {
status = 3
} else if err != nil {
status = 2
}
return status
}

42
internal/common/actions_test.go
View file

@ -35,7 +35,7 @@ import (
)
func TestNewActionNotification(t *testing.T) {
user := &dataprovider.User{
user := dataprovider.User{
BaseUser: sdk.BaseUser{
Username: "username",
},
@ -68,51 +68,57 @@ func TestNewActionNotification(t *testing.T) {
Endpoint: "httpendpoint",
},
}
c := NewBaseConnection("id", ProtocolSSH, "", "", user)
sessionID := xid.New().String()
a := newActionNotification(user, operationDownload, "path", "vpath", "target", "", "", ProtocolSFTP, "", sessionID,
123, 0, errors.New("fake error"))
a := newActionNotification(&user, operationDownload, "path", "vpath", "target", "", "", ProtocolSFTP, "", sessionID,
123, 0, c.getNotificationStatus(errors.New("fake error")))
assert.Equal(t, user.Username, a.Username)
assert.Equal(t, 0, len(a.Bucket))
assert.Equal(t, 0, len(a.Endpoint))
assert.Equal(t, 2, a.Status)
user.FsConfig.Provider = sdk.S3FilesystemProvider
a = newActionNotification(user, operationDownload, "path", "vpath", "target", "", "", ProtocolSSH, "", sessionID,
123, 0, nil)
a = newActionNotification(&user, operationDownload, "path", "vpath", "target", "", "", ProtocolSSH, "", sessionID,
123, 0, c.getNotificationStatus(nil))
assert.Equal(t, "s3bucket", a.Bucket)
assert.Equal(t, "endpoint", a.Endpoint)
assert.Equal(t, 1, a.Status)
user.FsConfig.Provider = sdk.GCSFilesystemProvider
a = newActionNotification(user, operationDownload, "path", "vpath", "target", "", "", ProtocolSCP, "", sessionID,
123, 0, ErrQuotaExceeded)
a = newActionNotification(&user, operationDownload, "path", "vpath", "target", "", "", ProtocolSCP, "", sessionID,
123, 0, c.getNotificationStatus(ErrQuotaExceeded))
assert.Equal(t, "gcsbucket", a.Bucket)
assert.Equal(t, 0, len(a.Endpoint))
assert.Equal(t, 3, a.Status)
a = newActionNotification(&user, operationDownload, "path", "vpath", "target", "", "", ProtocolSCP, "", sessionID,
123, 0, c.getNotificationStatus(fmt.Errorf("wrapper quota error: %w", ErrQuotaExceeded)))
assert.Equal(t, "gcsbucket", a.Bucket)
assert.Equal(t, 0, len(a.Endpoint))
assert.Equal(t, 3, a.Status)
user.FsConfig.Provider = sdk.HTTPFilesystemProvider
a = newActionNotification(user, operationDownload, "path", "vpath", "target", "", "", ProtocolSSH, "", sessionID,
123, 0, nil)
a = newActionNotification(&user, operationDownload, "path", "vpath", "target", "", "", ProtocolSSH, "", sessionID,
123, 0, c.getNotificationStatus(nil))
assert.Equal(t, "httpendpoint", a.Endpoint)
assert.Equal(t, 1, a.Status)
user.FsConfig.Provider = sdk.AzureBlobFilesystemProvider
a = newActionNotification(user, operationDownload, "path", "vpath", "target", "", "", ProtocolSCP, "", sessionID,
123, 0, nil)
a = newActionNotification(&user, operationDownload, "path", "vpath", "target", "", "", ProtocolSCP, "", sessionID,
123, 0, c.getNotificationStatus(nil))
assert.Equal(t, "azcontainer", a.Bucket)
assert.Equal(t, "azendpoint", a.Endpoint)
assert.Equal(t, 1, a.Status)
a = newActionNotification(user, operationDownload, "path", "vpath", "target", "", "", ProtocolSCP, "", sessionID,
123, os.O_APPEND, nil)
a = newActionNotification(&user, operationDownload, "path", "vpath", "target", "", "", ProtocolSCP, "", sessionID,
123, os.O_APPEND, c.getNotificationStatus(nil))
assert.Equal(t, "azcontainer", a.Bucket)
assert.Equal(t, "azendpoint", a.Endpoint)
assert.Equal(t, 1, a.Status)
assert.Equal(t, os.O_APPEND, a.OpenFlags)
user.FsConfig.Provider = sdk.SFTPFilesystemProvider
a = newActionNotification(user, operationDownload, "path", "vpath", "target", "", "", ProtocolSFTP, "", sessionID,
123, 0, nil)
a = newActionNotification(&user, operationDownload, "path", "vpath", "target", "", "", ProtocolSFTP, "", sessionID,
123, 0, c.getNotificationStatus(nil))
assert.Equal(t, "sftpendpoint", a.Endpoint)
}
@ -129,7 +135,7 @@ func TestActionHTTP(t *testing.T) {
},
}
a := newActionNotification(user, operationDownload, "path", "vpath", "target", "", "", ProtocolSFTP, "",
xid.New().String(), 123, 0, nil)
xid.New().String(), 123, 0, 1)
err := actionHandler.Handle(a)
assert.NoError(t, err)
@ -166,7 +172,7 @@ func TestActionCMD(t *testing.T) {
}
sessionID := shortuuid.New()
a := newActionNotification(user, operationDownload, "path", "vpath", "target", "", "", ProtocolSFTP, "", sessionID,
123, 0, nil)
123, 0, 1)
err = actionHandler.Handle(a)
assert.NoError(t, err)
@ -198,7 +204,7 @@ func TestWrongActions(t *testing.T) {
}
a := newActionNotification(user, operationUpload, "", "", "", "", "", ProtocolSFTP, "", xid.New().String(),
123, 0, nil)
123, 0, 1)
err := actionHandler.Handle(a)
assert.Error(t, err, "action with bad command must fail")

2
internal/common/common.go
View file

@ -56,12 +56,14 @@ const (
chownLogSender = "Chown"
chmodLogSender = "Chmod"
chtimesLogSender = "Chtimes"
copyLogSender = "Copy"
truncateLogSender = "Truncate"
operationDownload = "download"
operationUpload = "upload"
operationFirstDownload = "first-download"
operationFirstUpload = "first-upload"
operationDelete = "delete"
operationCopy = "copy"
// Pre-download action name
OperationPreDownload = "pre-download"
// Pre-upload action name

332
internal/common/connection.go
View file

@ -17,6 +17,7 @@ package common
import (
"errors"
"fmt"
"io"
"os"
"path"
"strings"
@ -334,7 +335,7 @@ func (c *BaseConnection) CheckParentDirs(virtualPath string) error {
continue
}
if err = c.createDirIfMissing(dirs[idx]); err != nil {
return fmt.Errorf("unable to check/create missing parent dir %#v for virtual path %#v: %w",
return fmt.Errorf("unable to check/create missing parent dir %q for virtual path %q: %w",
dirs[idx], virtualPath, err)
}
}
@ -352,7 +353,7 @@ func (c *BaseConnection) CreateDir(virtualPath string, checkFilePatterns bool) e
}
}
if c.User.IsVirtualFolder(virtualPath) {
c.Log(logger.LevelWarn, "mkdir not allowed %#v is a virtual folder", virtualPath)
c.Log(logger.LevelWarn, "mkdir not allowed %q is a virtual folder", virtualPath)
return c.GetPermissionDeniedError()
}
fs, fsPath, err := c.GetFsAndResolvedPath(virtualPath)
@ -377,7 +378,7 @@ func (c *BaseConnection) IsRemoveFileAllowed(virtualPath string) error {
return c.GetPermissionDeniedError()
}
if ok, policy := c.User.IsFileAllowed(virtualPath); !ok {
c.Log(logger.LevelDebug, "removing file %#v is not allowed", virtualPath)
c.Log(logger.LevelDebug, "removing file %q is not allowed", virtualPath)
return c.GetErrorForDeniedFile(policy)
}
return nil
@ -392,10 +393,10 @@ func (c *BaseConnection) RemoveFile(fs vfs.Fs, fsPath, virtualPath string, info
size := info.Size()
actionErr := ExecutePreAction(c, operationPreDelete, fsPath, virtualPath, size, 0)
if actionErr == nil {
c.Log(logger.LevelDebug, "remove for path %#v handled by pre-delete action", fsPath)
c.Log(logger.LevelDebug, "remove for path %q handled by pre-delete action", fsPath)
} else {
if err := fs.Remove(fsPath, false); err != nil {
c.Log(logger.LevelError, "failed to remove file/symlink %#v: %+v", fsPath, err)
c.Log(logger.LevelError, "failed to remove file/symlink %q: %+v", fsPath, err)
return c.GetFsError(fs, err)
}
}
@ -421,27 +422,28 @@ func (c *BaseConnection) RemoveFile(fs vfs.Fs, fsPath, virtualPath string, info
// IsRemoveDirAllowed returns an error if removing this directory is not allowed
func (c *BaseConnection) IsRemoveDirAllowed(fs vfs.Fs, fsPath, virtualPath string) error {
if fs.GetRelativePath(fsPath) == "/" {
if virtualPath == "/" || fs.GetRelativePath(fsPath) == "/" {
c.Log(logger.LevelWarn, "removing root dir is not allowed")
return c.GetPermissionDeniedError()
}
if c.User.IsVirtualFolder(virtualPath) {
c.Log(logger.LevelWarn, "removing a virtual folder is not allowed: %#v", virtualPath)
return c.GetPermissionDeniedError()
c.Log(logger.LevelWarn, "removing a virtual folder is not allowed: %q", virtualPath)
return fmt.Errorf("removing virtual folders is not allowed: %w", c.GetPermissionDeniedError())
}
if c.User.HasVirtualFoldersInside(virtualPath) {
c.Log(logger.LevelWarn, "removing a directory with a virtual folder inside is not allowed: %#v", virtualPath)
return c.GetOpUnsupportedError()
return fmt.Errorf("cannot remove directory %q with virtual folders inside: %w", virtualPath, c.GetOpUnsupportedError())
}
if c.User.IsMappedPath(fsPath) {
c.Log(logger.LevelWarn, "removing a directory mapped as virtual folder is not allowed: %#v", fsPath)
return c.GetPermissionDeniedError()
c.Log(logger.LevelWarn, "removing a directory mapped as virtual folder is not allowed: %q", fsPath)
return fmt.Errorf("removing the directory %q mapped as virtual folder is not allowed: %w",
virtualPath, c.GetPermissionDeniedError())
}
if !c.User.HasAnyPerm([]string{dataprovider.PermDeleteDirs, dataprovider.PermDelete}, path.Dir(virtualPath)) {
return c.GetPermissionDeniedError()
}
if ok, policy := c.User.IsFileAllowed(virtualPath); !ok {
c.Log(logger.LevelDebug, "removing directory %#v is not allowed", virtualPath)
c.Log(logger.LevelDebug, "removing directory %q is not allowed", virtualPath)
return c.GetErrorForDeniedFile(policy)
}
return nil
@ -482,112 +484,29 @@ func (c *BaseConnection) RemoveDir(virtualPath string) error {
return nil
}
type objectToRemoveMapping struct {
fsPath string
virtualPath string
info os.FileInfo
}
// orderDirsToRemove orders directories so that the empty ones will be at slice start
func orderDirsToRemove(fs vfs.Fs, dirsToRemove []objectToRemoveMapping) []objectToRemoveMapping {
orderedDirs := make([]objectToRemoveMapping, 0, len(dirsToRemove))
removedDirs := make([]string, 0, len(dirsToRemove))
pathSeparator := "/"
if vfs.IsLocalOsFs(fs) {
pathSeparator = string(os.PathSeparator)
}
for len(orderedDirs) < len(dirsToRemove) {
for idx, d := range dirsToRemove {
if util.Contains(removedDirs, d.fsPath) {
continue
}
isEmpty := true
for idx1, d1 := range dirsToRemove {
if idx == idx1 {
continue
}
if util.Contains(removedDirs, d1.fsPath) {
continue
}
if strings.HasPrefix(d1.fsPath, d.fsPath+pathSeparator) {
isEmpty = false
break
}
}
if isEmpty {
orderedDirs = append(orderedDirs, d)
removedDirs = append(removedDirs, d.fsPath)
}
}
}
return orderedDirs
}
func (c *BaseConnection) removeDirTree(fs vfs.Fs, fsPath, virtualPath string) error {
var dirsToRemove []objectToRemoveMapping
var filesToRemove []objectToRemoveMapping
err := fs.Walk(fsPath, func(walkedPath string, info os.FileInfo, err error) error {
if err != nil {
return err
}
obj := objectToRemoveMapping{
fsPath: walkedPath,
virtualPath: fs.GetRelativePath(walkedPath),
info: info,
}
if info.IsDir() {
err = c.IsRemoveDirAllowed(fs, obj.fsPath, obj.virtualPath)
isDuplicated := false
for _, d := range dirsToRemove {
if d.fsPath == obj.fsPath {
isDuplicated = true
break
}
}
if !isDuplicated {
dirsToRemove = append(dirsToRemove, obj)
}
} else {
err = c.IsRemoveFileAllowed(obj.virtualPath)
filesToRemove = append(filesToRemove, obj)
}
if err != nil {
c.Log(logger.LevelError, "unable to remove dir tree, object %q->%q cannot be removed: %v",
virtualPath, fsPath, err)
return err
}
return nil
})
func (c *BaseConnection) doRecursiveRemoveDirEntry(virtualPath string, info os.FileInfo) error {
fs, fsPath, err := c.GetFsAndResolvedPath(virtualPath)
if err != nil {
c.Log(logger.LevelError, "failed to remove dir tree %q->%q: error: %+v", virtualPath, fsPath, err)
return c.GetFsError(fs, err)
return err
}
return c.doRecursiveRemove(fs, fsPath, virtualPath, info)
}
for _, fileObj := range filesToRemove {
err = c.RemoveFile(fs, fileObj.fsPath, fileObj.virtualPath, fileObj.info)
func (c *BaseConnection) doRecursiveRemove(fs vfs.Fs, fsPath, virtualPath string, info os.FileInfo) error {
if info.IsDir() {
entries, err := c.ListDir(virtualPath)
if err != nil {
c.Log(logger.LevelError, "unable to remove dir tree, error removing file %q->%q: %v",
fileObj.virtualPath, fileObj.fsPath, err)
return err
return fmt.Errorf("unable to get contents for dir %q: %w", virtualPath, err)
}
}
for _, dirObj := range orderDirsToRemove(fs, dirsToRemove) {
err = c.RemoveDir(dirObj.virtualPath)
if err != nil {
c.Log(logger.LevelDebug, "unable to remove dir tree, error removing directory %q->%q: %v",
dirObj.virtualPath, dirObj.fsPath, err)
return err
for _, fi := range entries {
targetPath := path.Join(virtualPath, fi.Name())
if err := c.doRecursiveRemoveDirEntry(targetPath, fi); err != nil {
return err
}
}
return c.RemoveDir(virtualPath)
}
return err
return c.RemoveFile(fs, fsPath, virtualPath, info)
}
// RemoveAll removes the specified path and any children it contains
@ -603,11 +522,150 @@ func (c *BaseConnection) RemoveAll(virtualPath string) error {
return c.GetFsError(fs, err)
}
if fi.IsDir() && fi.Mode()&os.ModeSymlink == 0 {
return c.removeDirTree(fs, fsPath, virtualPath)
if err := c.IsRemoveDirAllowed(fs, fsPath, virtualPath); err != nil {
return err
}
return c.doRecursiveRemove(fs, fsPath, virtualPath, fi)
}
return c.RemoveFile(fs, fsPath, virtualPath, fi)
}
func (c *BaseConnection) checkCopyFolder(srcInfo, dstInfo os.FileInfo, virtualSource, virtualTarget string) error {
if srcInfo.IsDir() {
if dstInfo != nil && !dstInfo.IsDir() {
return fmt.Errorf("cannot overwrite file %q with dir %q: %w", virtualTarget, virtualSource, c.GetOpUnsupportedError())
}
if util.IsDirOverlapped(virtualSource, virtualTarget, true, "/") {
return fmt.Errorf("nested copy %q => %q is not supported: %w", virtualSource, virtualTarget, c.GetOpUnsupportedError())
}
_, fsSourcePath, err := c.GetFsAndResolvedPath(virtualSource)
if err != nil {
return err
}
_, fsTargetPath, err := c.GetFsAndResolvedPath(virtualTarget)
if err != nil {
return err
}
if util.IsDirOverlapped(fsSourcePath, fsTargetPath, true, c.User.FsConfig.GetPathSeparator()) {
c.Log(logger.LevelWarn, "nested fs copy %q => %q not allowed", fsSourcePath, fsTargetPath)
return fmt.Errorf("nested fs copy is not supported: %w", c.GetOpUnsupportedError())
}
return nil
}
if dstInfo != nil && dstInfo.IsDir() {
return fmt.Errorf("cannot overwrite file %q with dir %q: %w", virtualSource, virtualTarget, c.GetOpUnsupportedError())
}
return nil
}
func (c *BaseConnection) copyFile(virtualSourcePath, virtualTargetPath string, srcSize int64) error {
if ok, _ := c.User.IsFileAllowed(virtualTargetPath); !ok {
return fmt.Errorf("file %q is not allowed: %w", virtualTargetPath, c.GetPermissionDeniedError())
}
reader, rCancelFn, err := getFileReader(c, virtualSourcePath)
if err != nil {
return fmt.Errorf("unable to get reader for path %q: %w", virtualSourcePath, err)
}
defer rCancelFn()
defer reader.Close()
writer, numFiles, truncatedSize, wCancelFn, err := getFileWriter(c, virtualTargetPath, srcSize)
if err != nil {
return fmt.Errorf("unable to get writer for path %q: %w", virtualTargetPath, err)
}
defer wCancelFn()
_, err = io.Copy(writer, reader)
return closeWriterAndUpdateQuota(writer, c, virtualSourcePath, virtualTargetPath, numFiles, truncatedSize, err, operationCopy)
}
func (c *BaseConnection) doRecursiveCopy(virtualSourcePath, virtualTargetPath string, srcInfo os.FileInfo,
createTargetDir bool,
) error {
if srcInfo.IsDir() {
if createTargetDir {
if err := c.CreateDir(virtualTargetPath, false); err != nil {
return fmt.Errorf("unable to create directory %q: %w", virtualTargetPath, err)
}
}
entries, err := c.ListDir(virtualSourcePath)
if err != nil {
return fmt.Errorf("unable to get contents for dir %q: %w", virtualSourcePath, err)
}
for _, info := range entries {
sourcePath := path.Join(virtualSourcePath, info.Name())
targetPath := path.Join(virtualTargetPath, info.Name())
targetInfo, err := c.DoStat(targetPath, 1, false)
if err == nil {
if info.IsDir() && targetInfo.IsDir() {
c.Log(logger.LevelDebug, "target copy dir %q already exists", targetPath)
continue
}
}
if err != nil && !c.IsNotExistError(err) {
return err
}
if err := c.checkCopyFolder(info, targetInfo, sourcePath, targetPath); err != nil {
return err
}
if err := c.doRecursiveCopy(sourcePath, targetPath, info, true); err != nil {
return err
}
}
return nil
}
if !srcInfo.Mode().IsRegular() {
c.Log(logger.LevelInfo, "skipping copy for non regular file %q", virtualSourcePath)
return nil
}
return c.copyFile(virtualSourcePath, virtualTargetPath, srcInfo.Size())
}
// Copy virtualSourcePath to virtualTargetPath
func (c *BaseConnection) Copy(virtualSourcePath, virtualTargetPath string) error {
copyFromSource := strings.HasSuffix(virtualSourcePath, "/")
copyInTarget := strings.HasSuffix(virtualTargetPath, "/")
virtualSourcePath = path.Clean(virtualSourcePath)
virtualTargetPath = path.Clean(virtualTargetPath)
if virtualSourcePath == virtualTargetPath {
return fmt.Errorf("the copy source and target cannot be the same: %w", c.GetOpUnsupportedError())
}
srcInfo, err := c.DoStat(virtualSourcePath, 1, false)
if err != nil {
return err
}
if srcInfo.Mode()&os.ModeSymlink != 0 {
return fmt.Errorf("copying symlinks is not supported: %w", c.GetOpUnsupportedError())
}
dstInfo, err := c.DoStat(virtualTargetPath, 1, false)
if err == nil && !copyFromSource {
copyInTarget = dstInfo.IsDir()
}
if err != nil && !c.IsNotExistError(err) {
return err
}
destPath := virtualTargetPath
if copyInTarget {
destPath = path.Join(virtualTargetPath, path.Base(virtualSourcePath))
dstInfo, err = c.DoStat(destPath, 1, false)
if err != nil && !c.IsNotExistError(err) {
return err
}
}
createTargetDir := true
if dstInfo != nil && dstInfo.IsDir() {
createTargetDir = false
}
if err := c.checkCopyFolder(srcInfo, dstInfo, virtualSourcePath, destPath); err != nil {
return err
}
if err := c.CheckParentDirs(path.Dir(destPath)); err != nil {
return err
}
return c.doRecursiveCopy(virtualSourcePath, destPath, srcInfo, createTargetDir)
}
// Rename renames (moves) virtualSourcePath to virtualTargetPath
func (c *BaseConnection) Rename(virtualSourcePath, virtualTargetPath string) error {
if virtualSourcePath == virtualTargetPath {
@ -646,14 +704,7 @@ func (c *BaseConnection) Rename(virtualSourcePath, virtualTargetPath string) err
}
}
if srcInfo.IsDir() {
if c.User.HasVirtualFoldersInside(virtualSourcePath) {
c.Log(logger.LevelDebug, "renaming the folder %#v is not supported: it has virtual folders inside it",
virtualSourcePath)
return c.GetOpUnsupportedError()
}
if err = c.checkRecursiveRenameDirPermissions(fsSrc, fsDst, fsSourcePath, fsTargetPath,
virtualSourcePath, virtualTargetPath, srcInfo); err != nil {
c.Log(logger.LevelDebug, "error checking recursive permissions before renaming %#v: %+v", fsSourcePath, err)
if err := c.checkFolderRename(fsSrc, fsDst, fsSourcePath, fsTargetPath, virtualSourcePath, virtualTargetPath, srcInfo); err != nil {
return err
}
}
@ -662,7 +713,7 @@ func (c *BaseConnection) Rename(virtualSourcePath, virtualTargetPath string) err
return c.GetGenericError(ErrQuotaExceeded)
}
if err := fsDst.Rename(fsSourcePath, fsTargetPath); err != nil {
c.Log(logger.LevelError, "failed to rename %#v -> %#v: %+v", fsSourcePath, fsTargetPath, err)
c.Log(logger.LevelError, "failed to rename %q -> %q: %+v", fsSourcePath, fsTargetPath, err)
return c.GetFsError(fsSrc, err)
}
vfs.SetPathPermissions(fsDst, fsTargetPath, c.User.GetUID(), c.User.GetGID())
@ -768,7 +819,9 @@ func (c *BaseConnection) doStatInternal(virtualPath string, mode int, checkFileP
info, err = fs.Stat(c.getRealFsPath(fsPath))
}
if err != nil {
c.Log(logger.LevelWarn, "stat error for path %#v: %+v", virtualPath, err)
if !fs.IsNotExist(err) {
c.Log(logger.LevelWarn, "stat error for path %q: %+v", virtualPath, err)
}
return info, c.GetFsError(fs, err)
}
if convertResult && vfs.IsCryptOsFs(fs) {
@ -997,6 +1050,31 @@ func (c *BaseConnection) hasRenamePerms(virtualSourcePath, virtualTargetPath str
c.User.HasAnyPerm(perms, path.Dir(virtualTargetPath))
}
func (c *BaseConnection) checkFolderRename(fsSrc, fsDst vfs.Fs, fsSourcePath, fsTargetPath, virtualSourcePath,
virtualTargetPath string, fi os.FileInfo) error {
if util.IsDirOverlapped(virtualSourcePath, virtualTargetPath, true, "/") {
c.Log(logger.LevelDebug, "renaming the folder %q->%q is not supported: nested folders",
virtualSourcePath, virtualTargetPath)
return c.GetOpUnsupportedError()
}
if util.IsDirOverlapped(fsSourcePath, fsTargetPath, true, c.User.FsConfig.GetPathSeparator()) {
c.Log(logger.LevelDebug, "renaming the folder %q->%q is not supported: nested fs folders",
fsSourcePath, fsTargetPath)
return c.GetOpUnsupportedError()
}
if c.User.HasVirtualFoldersInside(virtualSourcePath) {
c.Log(logger.LevelDebug, "renaming the folder %q is not supported: it has virtual folders inside it",
virtualSourcePath)
return c.GetOpUnsupportedError()
}
if err := c.checkRecursiveRenameDirPermissions(fsSrc, fsDst, fsSourcePath, fsTargetPath,
virtualSourcePath, virtualTargetPath, fi); err != nil {
c.Log(logger.LevelDebug, "error checking recursive permissions before renaming %q: %+v", fsSourcePath, err)
return err
}
return nil
}
func (c *BaseConnection) isRenamePermitted(fsSrc, fsDst vfs.Fs, fsSourcePath, fsTargetPath, virtualSourcePath,
virtualTargetPath string, fi os.FileInfo,
) bool {
@ -1533,6 +1611,16 @@ func (c *BaseConnection) GetFsError(fs vfs.Fs, err error) error {
return nil
}
func (c *BaseConnection) getNotificationStatus(err error) int {
if err == nil {
return 1
}
if c.IsQuotaExceededError(err) {
return 3
}
return 2
}
// GetFsAndResolvedPath returns the fs and the fs path matching virtualPath
func (c *BaseConnection) GetFsAndResolvedPath(virtualPath string) (vfs.Fs, string, error) {
fs, err := c.User.GetFilesystemForPath(virtualPath, c.ID)

143
internal/common/connection_test.go
View file

@ -35,8 +35,7 @@ import (
)
var (
errWalkDir = errors.New("err walk dir")
errWalkFile = errors.New("err walk file")
errWalkDir = errors.New("err walk dir")
)
// MockOsFs mockable OsFs
@ -68,6 +67,13 @@ func (fs *MockOsFs) Chtimes(name string, atime, mtime time.Time, isUploading boo
return vfs.ErrVfsUnsupported
}
func (fs *MockOsFs) Lstat(name string) (os.FileInfo, error) {
if fs.err != nil {
return nil, fs.err
}
return fs.Fs.Lstat(name)
}
// Walk returns a duplicate path for testing
func (fs *MockOsFs) Walk(root string, walkFn filepath.WalkFunc) error {
if fs.err == errWalkDir {
@ -272,6 +278,17 @@ func TestRenamePerms(t *testing.T) {
assert.True(t, conn.hasRenamePerms(src, subTarget, info))
}
func TestRenameNestedFolders(t *testing.T) {
u := dataprovider.User{}
conn := NewBaseConnection("", ProtocolSFTP, "", "", u)
err := conn.checkFolderRename(nil, nil, filepath.Clean(os.TempDir()), filepath.Join(os.TempDir(), "subdir"), "/src", "/dst", nil)
assert.Error(t, err)
err = conn.checkFolderRename(nil, nil, filepath.Join(os.TempDir(), "subdir"), filepath.Clean(os.TempDir()), "/src", "/dst", nil)
assert.Error(t, err)
err = conn.checkFolderRename(nil, nil, "", "", "/src/sub", "/src", nil)
assert.Error(t, err)
}
func TestUpdateQuotaAfterRename(t *testing.T) {
user := dataprovider.User{
BaseUser: sdk.BaseUser{
@ -547,97 +564,43 @@ func TestCheckParentDirsErrors(t *testing.T) {
assert.NoError(t, err)
}
func TestRemoveDirTree(t *testing.T) {
user := dataprovider.User{
func TestErrorResolvePath(t *testing.T) {
u := dataprovider.User{
BaseUser: sdk.BaseUser{
HomeDir: filepath.Clean(os.TempDir()),
HomeDir: filepath.Join(os.TempDir(), "u"),
Status: 1,
Permissions: map[string][]string{
"/": {dataprovider.PermAny},
},
},
}
user.Permissions = make(map[string][]string)
user.Permissions["/"] = []string{dataprovider.PermAny}
fs := vfs.NewOsFs("connID", user.HomeDir, "")
connection := NewBaseConnection(fs.ConnectionID(), ProtocolWebDAV, "", "", user)
vpath := path.Join("adir", "missing")
p := filepath.Join(user.HomeDir, "adir", "missing")
err := connection.removeDirTree(fs, p, vpath)
if assert.Error(t, err) {
assert.True(t, fs.IsNotExist(err))
u.FsConfig.Provider = sdk.GCSFilesystemProvider
u.FsConfig.GCSConfig.Bucket = "test"
u.FsConfig.GCSConfig.Credentials = kms.NewPlainSecret("invalid JSON for credentials")
u.VirtualFolders = []vfs.VirtualFolder{
{
BaseVirtualFolder: vfs.BaseVirtualFolder{
Name: "f",
MappedPath: filepath.Join(os.TempDir(), "f"),
},
VirtualPath: "/f",
},
}
fs = newMockOsFs(false, "mockID", user.HomeDir, "", nil)
err = connection.removeDirTree(fs, p, vpath)
if assert.Error(t, err) {
assert.True(t, fs.IsNotExist(err), "unexpected error: %v", err)
}
errFake := errors.New("fake err")
fs = newMockOsFs(false, "mockID", user.HomeDir, "", errFake)
err = connection.removeDirTree(fs, p, vpath)
if assert.Error(t, err) {
assert.EqualError(t, err, ErrGenericFailure.Error())
}
fs = newMockOsFs(true, "mockID", user.HomeDir, "", errWalkDir)
err = connection.removeDirTree(fs, p, vpath)
if assert.Error(t, err) {
assert.True(t, fs.IsPermission(err), "unexpected error: %v", err)
}
fs = newMockOsFs(false, "mockID", user.HomeDir, "", errWalkFile)
err = connection.removeDirTree(fs, p, vpath)
if assert.Error(t, err) {
assert.EqualError(t, err, ErrGenericFailure.Error())
}
connection.User.Permissions["/"] = []string{dataprovider.PermListItems}
fs = newMockOsFs(false, "mockID", user.HomeDir, "", nil)
err = connection.removeDirTree(fs, p, vpath)
if assert.Error(t, err) {
assert.EqualError(t, err, ErrPermissionDenied.Error())
}
}
func TestOrderDirsToRemove(t *testing.T) {
fs := vfs.NewOsFs("id", os.TempDir(), "")
dirsToRemove := []objectToRemoveMapping{}
orderedDirs := orderDirsToRemove(fs, dirsToRemove)
assert.Equal(t, len(dirsToRemove), len(orderedDirs))
dirsToRemove = []objectToRemoveMapping{
{
fsPath: "dir1",
virtualPath: "",
},
}
orderedDirs = orderDirsToRemove(fs, dirsToRemove)
assert.Equal(t, len(dirsToRemove), len(orderedDirs))
dirsToRemove = []objectToRemoveMapping{
{
fsPath: "dir1",
virtualPath: "",
},
{
fsPath: "dir12",
virtualPath: "",
},
{
fsPath: filepath.Join("dir1", "a", "b"),
virtualPath: "",
},
{
fsPath: filepath.Join("dir1", "a"),
virtualPath: "",
},
}
orderedDirs = orderDirsToRemove(fs, dirsToRemove)
if assert.Equal(t, len(dirsToRemove), len(orderedDirs)) {
assert.Equal(t, "dir12", orderedDirs[0].fsPath)
assert.Equal(t, filepath.Join("dir1", "a", "b"), orderedDirs[1].fsPath)
assert.Equal(t, filepath.Join("dir1", "a"), orderedDirs[2].fsPath)
assert.Equal(t, "dir1", orderedDirs[3].fsPath)
}
conn := NewBaseConnection("", ProtocolSFTP, "", "", u)
err := conn.doRecursiveRemoveDirEntry("/vpath", nil)
assert.Error(t, err)
err = conn.checkCopyFolder(vfs.NewFileInfo("name", true, 0, time.Unix(0, 0), false), nil, "/source", "/target")
assert.Error(t, err)
sourceFile := filepath.Join(os.TempDir(), "f", "source")
err = os.MkdirAll(filepath.Dir(sourceFile), os.ModePerm)
assert.NoError(t, err)
err = os.WriteFile(sourceFile, []byte(""), 0666)
assert.NoError(t, err)
err = conn.checkCopyFolder(vfs.NewFileInfo("name", true, 0, time.Unix(0, 0), false), nil, "/f/source", "/target")
assert.Error(t, err)
err = conn.checkCopyFolder(vfs.NewFileInfo("", false, 0, time.Unix(0, 0), false), vfs.NewFileInfo("", true, 0, time.Unix(0, 0), false), "", "")
assert.Error(t, err)
err = os.RemoveAll(filepath.Dir(sourceFile))
assert.NoError(t, err)
}

145
internal/common/eventmanager.go
View file

@ -667,22 +667,40 @@ func getCSVRetentionReport(results []folderRetentionCheckResult) ([]byte, error)
return b.Bytes(), err
}
func closeWriterAndUpdateQuota(w io.WriteCloser, conn *BaseConnection, virtualPath string, numFiles int,
truncatedSize int64, errTransfer error,
func closeWriterAndUpdateQuota(w io.WriteCloser, conn *BaseConnection, virtualSourcePath, virtualTargetPath string,
numFiles int, truncatedSize int64, errTransfer error, operation string,
) error {
errWrite := w.Close()
info, err := conn.doStatInternal(virtualPath, 0, false, false)
targetPath := virtualSourcePath
if virtualTargetPath != "" {
targetPath = virtualTargetPath
}
info, err := conn.doStatInternal(targetPath, 0, false, false)
if err == nil {
updateUserQuotaAfterFileWrite(conn, virtualPath, numFiles, info.Size()-truncatedSize)
_, fsPath, errFs := conn.GetFsAndResolvedPath(virtualPath)
if errFs == nil {
updateUserQuotaAfterFileWrite(conn, targetPath, numFiles, info.Size()-truncatedSize)
var fsSrcPath, fsDstPath string
var errSrcFs, errDstFs error
if virtualSourcePath != "" {
_, fsSrcPath, errSrcFs = conn.GetFsAndResolvedPath(virtualSourcePath)
}
if virtualTargetPath != "" {
_, fsDstPath, errDstFs = conn.GetFsAndResolvedPath(virtualTargetPath)
}
if errSrcFs == nil && errDstFs == nil {
if errTransfer == nil {
errTransfer = errWrite
}
ExecuteActionNotification(conn, operationUpload, fsPath, virtualPath, "", "", "", info.Size(), errTransfer) //nolint:errcheck
if operation == operationCopy {
logger.CommandLog(copyLogSender, fsSrcPath, fsDstPath, conn.User.Username, "", conn.ID, conn.protocol, -1, -1,
"", "", "", info.Size(), conn.localAddr, conn.remoteAddr)
}
ExecuteActionNotification(conn, operation, fsSrcPath, virtualSourcePath, fsDstPath, virtualTargetPath, "", info.Size(), errTransfer) //nolint:errcheck
}
} else {
eventManagerLog(logger.LevelWarn, "unable to update quota after writing %q: %v", virtualPath, err)
eventManagerLog(logger.LevelWarn, "unable to update quota after writing %q: %v", targetPath, err)
}
if errTransfer != nil {
return errTransfer
}
return errWrite
}
@ -699,7 +717,33 @@ func updateUserQuotaAfterFileWrite(conn *BaseConnection, virtualPath string, num
}
}
func getFileWriter(conn *BaseConnection, virtualPath string) (io.WriteCloser, int, int64, func(), error) {
func checkWriterPermsAndQuota(conn *BaseConnection, virtualPath string, numFiles int, expectedSize, truncatedSize int64) error {
if numFiles == 0 {
if !conn.User.HasPerm(dataprovider.PermOverwrite, path.Dir(virtualPath)) {
return conn.GetPermissionDeniedError()
}
} else {
if !conn.User.HasPerm(dataprovider.PermUpload, path.Dir(virtualPath)) {
return conn.GetPermissionDeniedError()
}
}
q, _ := conn.HasSpace(numFiles > 0, false, virtualPath)
if !q.HasSpace {
return conn.GetQuotaExceededError()
}
if expectedSize != -1 {
sizeDiff := expectedSize - truncatedSize
if sizeDiff > 0 {
remainingSize := q.GetRemainingSize()
if remainingSize > 0 && remainingSize < sizeDiff {
return conn.GetQuotaExceededError()
}
}
}
return nil
}
func getFileWriter(conn *BaseConnection, virtualPath string, expectedSize int64) (io.WriteCloser, int, int64, func(), error) {
fs, fsPath, err := conn.GetFsAndResolvedPath(virtualPath)
if err != nil {
return nil, 0, 0, nil, err
@ -723,6 +767,10 @@ func getFileWriter(conn *BaseConnection, virtualPath string) (io.WriteCloser, in
if err != nil && !fs.IsNotExist(err) {
return nil, numFiles, truncatedSize, nil, conn.GetFsError(fs, err)
}
if err := checkWriterPermsAndQuota(conn, virtualPath, numFiles, expectedSize, truncatedSize); err != nil {
return nil, numFiles, truncatedSize, nil, err
}
f, w, cancelFn, err := fs.Create(fsPath, 0)
if err != nil {
return nil, numFiles, truncatedSize, nil, conn.GetFsError(fs, err)
@ -823,6 +871,9 @@ func getZipEntryName(entryPath, baseDir string) (string, error) {
}
func getFileReader(conn *BaseConnection, virtualPath string) (io.ReadCloser, func(), error) {
if !conn.User.HasPerm(dataprovider.PermDownload, path.Dir(virtualPath)) {
return nil, nil, conn.GetPermissionDeniedError()
}
fs, fsPath, err := conn.GetFsAndResolvedPath(virtualPath)
if err != nil {
return nil, nil, err
@ -1427,6 +1478,37 @@ func executeRenameFsActionForUser(renames []dataprovider.KeyValue, replacer *str
return nil
}
func executeCopyFsActionForUser(copy []dataprovider.KeyValue, replacer *strings.Replacer,
user dataprovider.User,
) error {
user, err := getUserForEventAction(user)
if err != nil {
return err
}
connectionID := fmt.Sprintf("%s_%s", protocolEventAction, xid.New().String())
err = user.CheckFsRoot(connectionID)
defer user.CloseFs() //nolint:errcheck
if err != nil {
return fmt.Errorf("copy error, unable to check root fs for user %q: %w", user.Username, err)
}
conn := NewBaseConnection(connectionID, protocolEventAction, "", "", user)
for _, item := range copy {
source := util.CleanPath(replaceWithReplacer(item.Key, replacer))
target := util.CleanPath(replaceWithReplacer(item.Value, replacer))
if strings.HasSuffix(item.Key, "/") {
source += "/"
}
if strings.HasSuffix(item.Value, "/") {
target += "/"
}
if err = conn.Copy(source, target); err != nil {
return fmt.Errorf("unable to copy %q->%q, user %q: %w", source, target, user.Username, err)
}
eventManagerLog(logger.LevelDebug, "copy %q->%q ok, user %q", source, target, user.Username)
}
return nil
}
func executeExistFsActionForUser(exist []string, replacer *strings.Replacer,
user dataprovider.User,
) error {
@ -1485,6 +1567,41 @@ func executeRenameFsRuleAction(renames []dataprovider.KeyValue, replacer *string
return nil
}
func executeCopyFsRuleAction(copy []dataprovider.KeyValue, replacer *strings.Replacer,
conditions dataprovider.ConditionOptions, params *EventParams,
) error {
users, err := params.getUsers()
if err != nil {
return fmt.Errorf("unable to get users: %w", err)
}
var failures []string
var executed int
for _, user := range users {
// if sender is set, the conditions have already been evaluated
if params.sender == "" {
if !checkUserConditionOptions(&user, &conditions) {
eventManagerLog(logger.LevelDebug, "skipping fs copy for user %s, condition options don't match",
user.Username)
continue
}
}
executed++
if err = executeCopyFsActionForUser(copy, replacer, user); err != nil {
failures = append(failures, user.Username)
params.AddError(err)
continue
}
}
if len(failures) > 0 {
return fmt.Errorf("fs copy failed for users: %+v", failures)
}
if executed == 0 {
eventManagerLog(logger.LevelError, "no copy executed")
return errors.New("no copy executed")
}
return nil
}
func getArchiveBaseDir(paths []string) string {
var parentDirs []string
for _, p := range paths {
@ -1521,7 +1638,7 @@ func executeCompressFsActionForUser(c dataprovider.EventActionFsCompress, replac
}
paths = append(paths, p)
}
writer, numFiles, truncatedSize, cancelFn, err := getFileWriter(conn, name)
writer, numFiles, truncatedSize, cancelFn, err := getFileWriter(conn, name, -1)
if err != nil {
eventManagerLog(logger.LevelError, "unable to create archive %q: %v", name, err)
return fmt.Errorf("unable to create archive: %w", err)
@ -1539,16 +1656,16 @@ func executeCompressFsActionForUser(c dataprovider.EventActionFsCompress, replac
}
for _, item := range paths {
if err := addZipEntry(zipWriter, conn, item, baseDir); err != nil {
closeWriterAndUpdateQuota(writer, conn, name, numFiles, truncatedSize, err) //nolint:errcheck
closeWriterAndUpdateQuota(writer, conn, name, "", numFiles, truncatedSize, err, operationUpload) //nolint:errcheck
return err
}
}
if err := zipWriter.Writer.Close(); err != nil {
eventManagerLog(logger.LevelError, "unable to close zip file %q: %v", name, err)
closeWriterAndUpdateQuota(writer, conn, name, numFiles, truncatedSize, err) //nolint:errcheck
closeWriterAndUpdateQuota(writer, conn, name, "", numFiles, truncatedSize, err, operationUpload) //nolint:errcheck
return fmt.Errorf("unable to close zip file %q: %w", name, err)
}
return closeWriterAndUpdateQuota(writer, conn, name, numFiles, truncatedSize, err)
return closeWriterAndUpdateQuota(writer, conn, name, "", numFiles, truncatedSize, err, operationUpload)
}
func executeExistFsRuleAction(exist []string, replacer *strings.Replacer, conditions dataprovider.ConditionOptions,
@ -1638,6 +1755,8 @@ func executeFsRuleAction(c dataprovider.EventActionFilesystemConfig, conditions
return executeExistFsRuleAction(c.Exist, replacer, conditions, params)
case dataprovider.FilesystemActionCompress:
return executeCompressFsRuleAction(c.Compress, replacer, conditions, params)
case dataprovider.FilesystemActionCopy:
return executeCopyFsRuleAction(c.Copy, replacer, conditions, params)
default:
return fmt.Errorf("unsupported filesystem action %d", c.Type)
}

19
internal/common/eventmanager_test.go
View file

@ -406,6 +406,8 @@ func TestEventManagerErrors(t *testing.T) {
assert.Error(t, err)
err = executeExistFsRuleAction(nil, nil, dataprovider.ConditionOptions{}, &EventParams{})
assert.Error(t, err)
err = executeCopyFsRuleAction(nil, nil, dataprovider.ConditionOptions{}, &EventParams{})
assert.Error(t, err)
err = executeCompressFsRuleAction(dataprovider.EventActionFsCompress{}, nil, dataprovider.ConditionOptions{}, &EventParams{})
assert.Error(t, err)
err = executePwdExpirationCheckRuleAction(dataprovider.EventActionPasswordExpiration{},
@ -476,6 +478,15 @@ func TestEventManagerErrors(t *testing.T) {
},
})
assert.Error(t, err)
err = executeCopyFsActionForUser(nil, nil, dataprovider.User{
Groups: []sdk.GroupMapping{
{
Name: groupName,
Type: sdk.GroupTypePrimary,
},
},
})
assert.Error(t, err)
err = executeCompressFsActionForUser(dataprovider.EventActionFsCompress{}, nil, dataprovider.User{
Groups: []sdk.GroupMapping{
{
@ -1163,6 +1174,10 @@ func TestEventRuleActionsNoGroupMatching(t *testing.T) {
if assert.Error(t, err) {
assert.Contains(t, err.Error(), "no existence check executed")
}
err = executeCopyFsRuleAction(nil, nil, conditions, &EventParams{})
if assert.Error(t, err) {
assert.Contains(t, err.Error(), "no copy executed")
}
err = executeUsersQuotaResetRuleAction(conditions, &EventParams{})
if assert.Error(t, err) {
assert.Contains(t, err.Error(), "no user quota reset executed")
@ -1297,9 +1312,11 @@ func TestFilesystemActionErrors(t *testing.T) {
assert.Error(t, err)
err = executeExistFsActionForUser(nil, testReplacer, user)
assert.Error(t, err)
err = executeCopyFsActionForUser(nil, testReplacer, user)
assert.Error(t, err)
err = executeCompressFsActionForUser(dataprovider.EventActionFsCompress{}, testReplacer, user)
assert.Error(t, err)
_, _, _, _, err = getFileWriter(conn, "/path.txt") //nolint:dogsled
_, _, _, _, err = getFileWriter(conn, "/path.txt", -1) //nolint:dogsled
assert.Error(t, err)
err = executeEmailRuleAction(dataprovider.EventActionEmailConfig{
Recipients: []string{"test@example.net"},

463
internal/common/protocol_test.go
View file

@ -139,6 +139,7 @@ func TestMain(m *testing.M) {
sftpdConf := config.GetSFTPDConfig()
sftpdConf.Bindings[0].Port = 4022
sftpdConf.EnabledSSHCommands = []string{"*"}
sftpdConf.Bindings = append(sftpdConf.Bindings, sftpd.Binding{
Port: 4024,
})
@ -273,6 +274,10 @@ func TestBaseConnection(t *testing.T) {
if assert.Error(t, err) {
assert.Contains(t, err.Error(), "the rename source and target cannot be the same")
}
err = client.Rename(testDir, path.Join(testDir, "sub"))
if assert.Error(t, err) {
assert.Contains(t, err.Error(), "SSH_FX_OP_UNSUPPORTED")
}
err = client.RemoveDirectory(testDir)
assert.NoError(t, err)
err = client.Remove(testFileName)
@ -4190,6 +4195,80 @@ func TestEventRuleFsActions(t *testing.T) {
assert.NoError(t, err)
}
func TestFsActionCopy(t *testing.T) {
a1 := dataprovider.BaseEventAction{
Name: "a1",
Type: dataprovider.ActionTypeFilesystem,
Options: dataprovider.BaseEventActionOptions{
FsConfig: dataprovider.EventActionFilesystemConfig{
Type: dataprovider.FilesystemActionCopy,
Copy: []dataprovider.KeyValue{
{
Key: "/{{VirtualPath}}/",
Value: "/dircopy/",
},
},
},
},
}
action1, resp, err := httpdtest.AddEventAction(a1, http.StatusCreated)
assert.NoError(t, err, string(resp))
r1 := dataprovider.EventRule{
Name: "rule1",
Trigger: dataprovider.EventTriggerFsEvent,
Conditions: dataprovider.EventConditions{
FsEvents: []string{"upload"},
},
Actions: []dataprovider.EventAction{
{
BaseEventAction: dataprovider.BaseEventAction{
Name: action1.Name,
},
Order: 1,
Options: dataprovider.EventActionOptions{
ExecuteSync: true,
},
},
},
}
rule1, _, err := httpdtest.AddEventRule(r1, http.StatusCreated)
assert.NoError(t, err)
u := getTestUser()
user, _, err := httpdtest.AddUser(u, http.StatusCreated)
assert.NoError(t, err)
conn, client, err := getSftpClient(user)
if assert.NoError(t, err) {
defer conn.Close()
defer client.Close()
err = writeSFTPFile(testFileName, 100, client)
assert.NoError(t, err)
_, err = client.Stat(path.Join("dircopy", testFileName))
assert.NoError(t, err)
action1.Options.FsConfig.Copy = []dataprovider.KeyValue{
{
Key: "/missing path",
Value: "/copied path",
},
}
_, _, err = httpdtest.UpdateEventAction(action1, http.StatusOK)
assert.NoError(t, err)
// copy a missing path will fail
err = writeSFTPFile(testFileName, 100, client)
assert.Error(t, err)
}
_, err = httpdtest.RemoveEventRule(rule1, http.StatusOK)
assert.NoError(t, err)
_, err = httpdtest.RemoveEventAction(action1, http.StatusOK)
assert.NoError(t, err)
_, err = httpdtest.RemoveUser(user, http.StatusOK)
assert.NoError(t, err)
err = os.RemoveAll(user.GetHomeDir())
assert.NoError(t, err)
}
func TestEventFsActionsGroupFilters(t *testing.T) {
smtpCfg := smtp.Config{
Host: "127.0.0.1",
@ -6824,6 +6903,355 @@ func TestNonLocalCrossRenameNonLocalBaseUser(t *testing.T) {
assert.NoError(t, err)
}
func TestCopyAndRemoveSSHCommands(t *testing.T) {
u := getTestUser()
u.QuotaFiles = 1000
user, _, err := httpdtest.AddUser(u, http.StatusCreated)
assert.NoError(t, err)
conn, client, err := getSftpClient(user)
if assert.NoError(t, err) {
defer conn.Close()
defer client.Close()
fileSize := int64(32)
err = writeSFTPFile(testFileName, fileSize, client)
assert.NoError(t, err)
testFileNameCopy := testFileName + "_copy"
out, err := runSSHCommand(fmt.Sprintf("sftpgo-copy %s %s", testFileName, testFileNameCopy), user)
assert.NoError(t, err, string(out))
info, err := client.Stat(testFileNameCopy)
if assert.NoError(t, err) {
assert.Equal(t, fileSize, info.Size())
}
testDir := "test dir"
err = client.Mkdir(testDir)
assert.NoError(t, err)
out, err = runSSHCommand(fmt.Sprintf(`sftpgo-copy %s '%s'`, testFileName, testDir), user)
assert.NoError(t, err, string(out))
info, err = client.Stat(path.Join(testDir, testFileName))
if assert.NoError(t, err) {
assert.Equal(t, fileSize, info.Size())
}
user, _, err = httpdtest.GetUserByUsername(user.Username, http.StatusOK)
assert.NoError(t, err)
assert.Equal(t, 3*fileSize, user.UsedQuotaSize)
assert.Equal(t, 3, user.UsedQuotaFiles)
out, err = runSSHCommand(fmt.Sprintf("sftpgo-remove %s", testFileNameCopy), user)
assert.NoError(t, err, string(out))
out, err = runSSHCommand(fmt.Sprintf(`sftpgo-remove '%s'`, testDir), user)
assert.NoError(t, err, string(out))
user, _, err = httpdtest.GetUserByUsername(user.Username, http.StatusOK)
assert.NoError(t, err)
assert.Equal(t, fileSize, user.UsedQuotaSize)
assert.Equal(t, 1, user.UsedQuotaFiles)
_, err = client.Stat(testFileNameCopy)
assert.ErrorIs(t, err, os.ErrNotExist)
// create a dir tree
dir1 := "dir1"
dir2 := "dir 2"
err = client.MkdirAll(path.Join(dir1, dir2))
assert.NoError(t, err)
toCreate := []string{
path.Join(dir1, testFileName),
path.Join(dir1, dir2, testFileName),
}
for _, p := range toCreate {
err = writeSFTPFile(p, fileSize, client)
assert.NoError(t, err)
}
// create a symlink, copying a symlink is not supported
err = client.Symlink(path.Join("/", dir1, testFileName), path.Join("/", dir1, testFileName+"_link"))
assert.NoError(t, err)
out, err = runSSHCommand(fmt.Sprintf("sftpgo-copy %s %s", path.Join("/", dir1, testFileName+"_link"),
path.Join("/", testFileName+"_link")), user)
assert.Error(t, err, string(out))
// copying a dir inside itself should fail
out, err = runSSHCommand(fmt.Sprintf("sftpgo-copy %s %s", path.Join("/", dir1),
path.Join("/", dir1, "sub")), user)
assert.Error(t, err, string(out))
// copy source and dest must differ
out, err = runSSHCommand(fmt.Sprintf("sftpgo-copy %s %s", path.Join("/", dir1),
path.Join("/", dir1)), user)
assert.Error(t, err, string(out))
// copy a missing file/dir should fail
out, err = runSSHCommand(fmt.Sprintf("sftpgo-copy %s %s", path.Join("/", "missing_entry"),
path.Join("/", dir1)), user)
assert.Error(t, err, string(out))
// try to overwrite a file with a dir
out, err = runSSHCommand(fmt.Sprintf("sftpgo-copy %s %s", path.Join("/", dir1), testFileName), user)
assert.Error(t, err, string(out))
out, err = runSSHCommand(fmt.Sprintf(`sftpgo-copy %s "%s"`, dir1, dir2), user)
assert.NoError(t, err, string(out))
user, _, err = httpdtest.GetUserByUsername(user.Username, http.StatusOK)
assert.NoError(t, err)
assert.Equal(t, 5*fileSize, user.UsedQuotaSize)
assert.Equal(t, 5, user.UsedQuotaFiles)
// copy again, quota must remain unchanged
out, err = runSSHCommand(fmt.Sprintf(`sftpgo-copy %s/ "%s"`, dir1, dir2), user)
assert.NoError(t, err, string(out))
_, err = client.Stat(dir2)
assert.NoError(t, err)
user, _, err = httpdtest.GetUserByUsername(user.Username, http.StatusOK)
assert.NoError(t, err)
assert.Equal(t, 5*fileSize, user.UsedQuotaSize)
assert.Equal(t, 5, user.UsedQuotaFiles)
// now copy inside target
out, err = runSSHCommand(fmt.Sprintf(`sftpgo-copy %s "%s"`, dir1, dir2), user)
assert.NoError(t, err, string(out))
user, _, err = httpdtest.GetUserByUsername(user.Username, http.StatusOK)
assert.NoError(t, err)
assert.Equal(t, 7*fileSize, user.UsedQuotaSize)
assert.Equal(t, 7, user.UsedQuotaFiles)
for _, p := range []string{dir1, dir2} {
out, err = runSSHCommand(fmt.Sprintf(`sftpgo-remove "%s"`, p), user)
assert.NoError(t, err, string(out))
_, err = client.Stat(p)
assert.ErrorIs(t, err, os.ErrNotExist)
}
user, _, err = httpdtest.GetUserByUsername(user.Username, http.StatusOK)
assert.NoError(t, err)
assert.Equal(t, fileSize, user.UsedQuotaSize)
assert.Equal(t, 1, user.UsedQuotaFiles)
// test quota errors
user.QuotaFiles = 1
_, _, err = httpdtest.UpdateUser(user, http.StatusOK, "")
assert.NoError(t, err)
// quota files exceeded
out, err = runSSHCommand(fmt.Sprintf("sftpgo-copy %s %s", testFileName, testFileNameCopy), user)
assert.Error(t, err, string(out))
user.QuotaFiles = 1000
user.QuotaSize = fileSize + 1
_, _, err = httpdtest.UpdateUser(user, http.StatusOK, "")
assert.NoError(t, err)
// quota size exceeded after the copy
out, err = runSSHCommand(fmt.Sprintf("sftpgo-copy %s %s", testFileName, testFileNameCopy), user)
assert.Error(t, err, string(out))
user.QuotaSize = fileSize - 1
_, _, err = httpdtest.UpdateUser(user, http.StatusOK, "")
assert.NoError(t, err)
// quota size exceeded
out, err = runSSHCommand(fmt.Sprintf("sftpgo-copy %s %s", testFileName, testFileNameCopy), user)
assert.Error(t, err, string(out))
}
_, err = httpdtest.RemoveUser(user, http.StatusOK)
assert.NoError(t, err)
err = os.RemoveAll(user.GetHomeDir())
assert.NoError(t, err)
}
func TestCopyAndRemovePermissions(t *testing.T) {
u := getTestUser()
restrictedPath := "/dir/path"
patternFilterPath := "/patterns"
u.Filters.FilePatterns = []sdk.PatternsFilter{
{
Path: patternFilterPath,
DeniedPatterns: []string{"*.dat"},
},
}
u.Permissions[restrictedPath] = []string{}
user, _, err := httpdtest.AddUser(u, http.StatusCreated)
assert.NoError(t, err)
conn, client, err := getSftpClient(user)
if assert.NoError(t, err) {
defer conn.Close()
defer client.Close()
err = client.MkdirAll(restrictedPath)
assert.NoError(t, err)
err = client.MkdirAll(patternFilterPath)
assert.NoError(t, err)
err = writeSFTPFile(testFileName, 100, client)
assert.NoError(t, err)
// getting file writer will fail
out, err := runSSHCommand(fmt.Sprintf(`sftpgo-copy %s %s`, testFileName, restrictedPath), user)
assert.Error(t, err, string(out))
// file pattern not allowed
out, err = runSSHCommand(fmt.Sprintf(`sftpgo-copy %s %s`, testFileName, patternFilterPath), user)
assert.Error(t, err, string(out))
testDir := path.Join("/", path.Base(restrictedPath))
err = client.Mkdir(testDir)
assert.NoError(t, err)
err = writeSFTPFile(path.Join(testDir, testFileName), 100, client)
assert.NoError(t, err)
// creating target dir will fail
out, err = runSSHCommand(fmt.Sprintf(`sftpgo-copy %s %s/`, testDir, restrictedPath), user)
assert.Error(t, err, string(out))
// get dir contents will fail
out, err = runSSHCommand(fmt.Sprintf(`sftpgo-copy %s /`, restrictedPath), user)
assert.Error(t, err, string(out))
// get dir contents will fail
out, err = runSSHCommand(fmt.Sprintf(`sftpgo-remove %s`, restrictedPath), user)
assert.Error(t, err, string(out))
// give list dir permissions and retry, now delete will fail
user.Permissions[restrictedPath] = []string{dataprovider.PermListItems, dataprovider.PermUpload}
user.Permissions[testDir] = []string{dataprovider.PermListItems}
_, _, err = httpdtest.UpdateUser(user, http.StatusOK, "")
assert.NoError(t, err)
out, err = runSSHCommand(fmt.Sprintf(`sftpgo-copy %s %s`, testFileName, restrictedPath), user)
assert.NoError(t, err, string(out))
// overwrite will fail, no permission
out, err = runSSHCommand(fmt.Sprintf(`sftpgo-copy %s %s`, testFileName, restrictedPath), user)
assert.Error(t, err, string(out))
out, err = runSSHCommand(fmt.Sprintf(`sftpgo-remove %s`, restrictedPath), user)
assert.Error(t, err, string(out))
// try to copy a file from testDir, we have only list permissions so getFileReader will fail
out, err = runSSHCommand(fmt.Sprintf(`sftpgo-copy %s %s`, path.Join(testDir, testFileName), testFileName+".copy"), user)
assert.Error(t, err, string(out))
}
_, err = httpdtest.RemoveUser(user, http.StatusOK)
assert.NoError(t, err)
err = os.RemoveAll(user.GetHomeDir())
assert.NoError(t, err)
}
func TestCrossFoldersCopy(t *testing.T) {
baseUser, resp, err := httpdtest.AddUser(getTestUser(), http.StatusCreated)
assert.NoError(t, err, string(resp))
u := getTestUser()
u.Username += "_1"
u.HomeDir = filepath.Join(os.TempDir(), u.Username)
u.QuotaFiles = 1000
mappedPath1 := filepath.Join(os.TempDir(), "mapped1")
folderName1 := filepath.Base(mappedPath1)
vpath1 := "/vdirs/vdir1"
u.VirtualFolders = append(u.VirtualFolders, vfs.VirtualFolder{
BaseVirtualFolder: vfs.BaseVirtualFolder{
Name: folderName1,
MappedPath: mappedPath1,
},
VirtualPath: vpath1,
QuotaSize: -1,
QuotaFiles: -1,
})
mappedPath2 := filepath.Join(os.TempDir(), "mapped1", "dir", "mapped2")
folderName2 := filepath.Base(mappedPath2)
vpath2 := "/vdirs/vdir2"
u.VirtualFolders = append(u.VirtualFolders, vfs.VirtualFolder{
BaseVirtualFolder: vfs.BaseVirtualFolder{
Name: folderName2,
MappedPath: mappedPath2,
},
VirtualPath: vpath2,
QuotaSize: -1,
QuotaFiles: -1,
})
mappedPath3 := filepath.Join(os.TempDir(), "mapped3")
folderName3 := filepath.Base(mappedPath3)
vpath3 := "/vdirs/vdir3"
u.VirtualFolders = append(u.VirtualFolders, vfs.VirtualFolder{
BaseVirtualFolder: vfs.BaseVirtualFolder{
Name: folderName3,
MappedPath: mappedPath3,
FsConfig: vfs.Filesystem{
Provider: sdk.CryptedFilesystemProvider,
CryptConfig: vfs.CryptFsConfig{
Passphrase: kms.NewPlainSecret(defaultPassword),
},
},
},
VirtualPath: vpath3,
QuotaSize: -1,
QuotaFiles: -1,
})
mappedPath4 := filepath.Join(os.TempDir(), "mapped4")
folderName4 := filepath.Base(mappedPath4)
vpath4 := "/vdirs/vdir4"
u.VirtualFolders = append(u.VirtualFolders, vfs.VirtualFolder{
BaseVirtualFolder: vfs.BaseVirtualFolder{
Name: folderName4,
MappedPath: mappedPath4,
FsConfig: vfs.Filesystem{
Provider: sdk.SFTPFilesystemProvider,
SFTPConfig: vfs.SFTPFsConfig{
BaseSFTPFsConfig: sdk.BaseSFTPFsConfig{
Endpoint: sftpServerAddr,
Username: baseUser.Username,
},
Password: kms.NewPlainSecret(defaultPassword),
},
},
},
VirtualPath: vpath4,
QuotaSize: -1,
QuotaFiles: -1,
})
user, resp, err := httpdtest.AddUser(u, http.StatusCreated)
assert.NoError(t, err, string(resp))
conn, client, err := getSftpClient(user)
if assert.NoError(t, err) {
defer conn.Close()
defer client.Close()
baseFileSize := int64(100)
err = writeSFTPFile(path.Join(vpath1, testFileName), baseFileSize+1, client)
assert.NoError(t, err)
err = writeSFTPFile(path.Join(vpath2, testFileName), baseFileSize+2, client)
assert.NoError(t, err)
err = writeSFTPFile(path.Join(vpath3, testFileName), baseFileSize+3, client)
assert.NoError(t, err)
err = writeSFTPFile(path.Join(vpath4, testFileName), baseFileSize+4, client)
assert.NoError(t, err)
// cannot remove a directory with virtual folders inside
out, err := runSSHCommand(fmt.Sprintf(`sftpgo-remove %s`, path.Dir(vpath1)), user)
assert.Error(t, err, string(out))
// copy across virtual folders
copyDir := "/copy"
out, err = runSSHCommand(fmt.Sprintf(`sftpgo-copy %s %s/`, path.Dir(vpath1), copyDir), user)
assert.NoError(t, err, string(out))
// check the copy
info, err := client.Stat(path.Join(copyDir, vpath1, testFileName))
if assert.NoError(t, err) {
assert.Equal(t, baseFileSize+1, info.Size())
}
info, err = client.Stat(path.Join(copyDir, vpath2, testFileName))
if assert.NoError(t, err) {
assert.Equal(t, baseFileSize+2, info.Size())
}
info, err = client.Stat(path.Join(copyDir, vpath3, testFileName))
if assert.NoError(t, err) {
assert.Equal(t, baseFileSize+3, info.Size())
}
info, err = client.Stat(path.Join(copyDir, vpath4, testFileName))
if assert.NoError(t, err) {
assert.Equal(t, baseFileSize+4, info.Size())
}
// nested fs paths
out, err = runSSHCommand(fmt.Sprintf(`sftpgo-copy %s %s`, vpath1, vpath2), user)
assert.Error(t, err, string(out))
}
_, err = httpdtest.RemoveUser(user, http.StatusOK)
assert.NoError(t, err)
err = os.RemoveAll(user.GetHomeDir())
assert.NoError(t, err)
_, err = httpdtest.RemoveUser(baseUser, http.StatusOK)
assert.NoError(t, err)
err = os.RemoveAll(baseUser.GetHomeDir())
assert.NoError(t, err)
for _, folderName := range []string{folderName1, folderName2, folderName3, folderName4} {
_, err = httpdtest.RemoveFolder(vfs.BaseVirtualFolder{Name: folderName}, http.StatusOK)
assert.NoError(t, err)
err = os.RemoveAll(filepath.Join(os.TempDir(), folderName))
assert.NoError(t, err)
}
}
func TestProxyProtocol(t *testing.T) {
resp, err := httpclient.Get(fmt.Sprintf("http://%v", httpProxyAddr))
if assert.NoError(t, err) {
@ -6918,6 +7346,41 @@ func getSftpClient(user dataprovider.User) (*ssh.Client, *sftp.Client, error) {
return conn, sftpClient, err
}
func runSSHCommand(command string, user dataprovider.User) ([]byte, error) {
var sshSession *ssh.Session
var output []byte
config := &ssh.ClientConfig{
User: user.Username,
HostKeyCallback: func(hostname string, remote net.Addr, key ssh.PublicKey) error {
return nil
},
Timeout: 5 * time.Second,
}
if user.Password != "" {
config.Auth = []ssh.AuthMethod{ssh.Password(user.Password)}
} else {
config.Auth = []ssh.AuthMethod{ssh.Password(defaultPassword)}
}
conn, err := ssh.Dial("tcp", sftpServerAddr, config)
if err != nil {
return output, err
}
defer conn.Close()
sshSession, err = conn.NewSession()
if err != nil {
return output, err
}
var stdout, stderr bytes.Buffer
sshSession.Stdout = &stdout
sshSession.Stderr = &stderr
err = sshSession.Run(command)
if err != nil {
return nil, fmt.Errorf("failed to run command %v: %v", command, stderr.Bytes())
}
return stdout.Bytes(), err
}
func getWebDavClient(user dataprovider.User) *gowebdav.Client {
rootPath := fmt.Sprintf("http://localhost:%d/", webDavServerPort)
pwd := defaultPassword

25
internal/dataprovider/dataprovider.go
View file

@ -2383,25 +2383,6 @@ func buildUserHomeDir(user *User) {
}
}
func isVirtualDirOverlapped(dir1, dir2 string, fullCheck bool) bool {
if dir1 == dir2 {
return true
}
if fullCheck {
if len(dir1) > len(dir2) {
if strings.HasPrefix(dir1, dir2+"/") {
return true
}
}
if len(dir2) > len(dir1) {
if strings.HasPrefix(dir2, dir1+"/") {
return true
}
}
}
return false
}
func validateFolderQuotaLimits(folder vfs.VirtualFolder) error {
if folder.QuotaSize < -1 {
return util.NewValidationError(fmt.Sprintf("invalid quota_size: %v folder path %#v", folder.QuotaSize, folder.MappedPath))
@ -2481,11 +2462,11 @@ func validateAssociatedVirtualFolders(vfolders []vfs.VirtualFolder) ([]vfs.Virtu
return nil, err
}
if folderNames[folder.Name] {
return nil, util.NewValidationError(fmt.Sprintf("the folder %#v is duplicated", folder.Name))
return nil, util.NewValidationError(fmt.Sprintf("the folder %q is duplicated", folder.Name))
}
for _, vFolder := range virtualFolders {
if isVirtualDirOverlapped(vFolder.VirtualPath, cleanedVPath, false) {
return nil, util.NewValidationError(fmt.Sprintf("invalid virtual folder %#v, it overlaps with virtual folder %#v",
if util.IsDirOverlapped(vFolder.VirtualPath, cleanedVPath, false, "/") {
return nil, util.NewValidationError(fmt.Sprintf("invalid virtual folder %q, it overlaps with virtual folder %q",
v.VirtualPath, vFolder.VirtualPath))
}
}

56
internal/dataprovider/eventrule.go
View file

@ -127,6 +127,7 @@ const (
FilesystemActionMkdirs
FilesystemActionExist
FilesystemActionCompress
FilesystemActionCopy
)
const (
@ -136,7 +137,7 @@ const (
var (
supportedFsActions = []int{FilesystemActionRename, FilesystemActionDelete, FilesystemActionMkdirs,
FilesystemActionCompress, FilesystemActionExist}
FilesystemActionCopy, FilesystemActionCompress, FilesystemActionExist}
)
func isFilesystemActionValid(value int) bool {
@ -153,6 +154,8 @@ func getFsActionTypeAsString(value int) string {
return "Paths exist"
case FilesystemActionCompress:
return "Compress"
case FilesystemActionCopy:
return "Copy"
default:
return "Create directories"
}
@ -162,7 +165,7 @@ func getFsActionTypeAsString(value int) string {
var (
// SupportedFsEvents defines the supported filesystem events
SupportedFsEvents = []string{"upload", "first-upload", "download", "first-download", "delete", "rename",
"mkdir", "rmdir", "ssh_cmd"}
"mkdir", "rmdir", "copy", "ssh_cmd"}
// SupportedProviderEvents defines the supported provider events
SupportedProviderEvents = []string{operationAdd, operationUpdate, operationDelete}
// SupportedRuleConditionProtocols defines the supported protcols for rule conditions
@ -587,6 +590,8 @@ type EventActionFilesystemConfig struct {
Deletes []string `json:"deletes,omitempty"`
// file/dirs to check for existence
Exist []string `json:"exist,omitempty"`
// files/dirs to copy, key is the source and target the value
Copy []KeyValue `json:"copy,omitempty"`
// paths to compress and archive name
Compress EventActionFsCompress `json:"compress"`
}
@ -641,6 +646,38 @@ func (c *EventActionFilesystemConfig) validateRenames() error {
return nil
}
func (c *EventActionFilesystemConfig) validateCopy() error {
if len(c.Copy) == 0 {
return util.NewValidationError("no path to copy specified")
}
for idx, kv := range c.Copy {
key := strings.TrimSpace(kv.Key)
value := strings.TrimSpace(kv.Value)
if key == "" || value == "" {
return util.NewValidationError("invalid paths to copy")
}
key = util.CleanPath(key)
value = util.CleanPath(value)
if key == value {
return util.NewValidationError("copy source and target cannot be equal")
}
if key == "/" || value == "/" {
return util.NewValidationError("copying the root directory is not allowed")
}
if strings.HasSuffix(c.Copy[idx].Key, "/") {
key += "/"
}
if strings.HasSuffix(c.Copy[idx].Value, "/") {
value += "/"
}
c.Copy[idx] = KeyValue{
Key: key,
Value: value,
}
}
return nil
}
func (c *EventActionFilesystemConfig) validateDeletes() error {
if len(c.Deletes) == 0 {
return util.NewValidationError("no path to delete specified")
@ -695,6 +732,7 @@ func (c *EventActionFilesystemConfig) validate() error {
c.MkDirs = nil
c.Deletes = nil
c.Exist = nil
c.Copy = nil
c.Compress = EventActionFsCompress{}
if err := c.validateRenames(); err != nil {
return err
@ -703,6 +741,7 @@ func (c *EventActionFilesystemConfig) validate() error {
c.Renames = nil
c.MkDirs = nil
c.Exist = nil
c.Copy = nil
c.Compress = EventActionFsCompress{}
if err := c.validateDeletes(); err != nil {
return err
@ -711,6 +750,7 @@ func (c *EventActionFilesystemConfig) validate() error {
c.Renames = nil
c.Deletes = nil
c.Exist = nil
c.Copy = nil
c.Compress = EventActionFsCompress{}
if err := c.validateMkdirs(); err != nil {
return err
@ -719,6 +759,7 @@ func (c *EventActionFilesystemConfig) validate() error {
c.Renames = nil
c.Deletes = nil
c.MkDirs = nil
c.Copy = nil
c.Compress = EventActionFsCompress{}
if err := c.validateExist(); err != nil {
return err
@ -728,9 +769,19 @@ func (c *EventActionFilesystemConfig) validate() error {
c.MkDirs = nil
c.Deletes = nil
c.Exist = nil
c.Copy = nil
if err := c.Compress.validate(); err != nil {
return err
}
case FilesystemActionCopy:
c.Renames = nil
c.Deletes = nil
c.MkDirs = nil
c.Exist = nil
c.Compress = EventActionFsCompress{}
if err := c.validateCopy(); err != nil {
return err
}
}
return nil
}
@ -751,6 +802,7 @@ func (c *EventActionFilesystemConfig) getACopy() EventActionFilesystemConfig {
MkDirs: mkdirs,
Deletes: deletes,
Exist: exist,
Copy: cloneKeyValues(c.Copy),
Compress: EventActionFsCompress{
Paths: compressPaths,
Name: c.Compress.Name,

2
internal/dataprovider/share.go
View file

@ -221,7 +221,7 @@ func (s *Share) validatePaths() error {
if idx == innerIdx {
continue
}
if isVirtualDirOverlapped(s.Paths[idx], s.Paths[innerIdx], true) {
if util.IsDirOverlapped(s.Paths[idx], s.Paths[innerIdx], true, "/") {
return util.NewGenericError("shared paths cannot be nested")
}
}

1
internal/httpd/webadmin.go
View file

@ -2179,6 +2179,7 @@ func getEventActionOptionsFromPostFields(r *http.Request) (dataprovider.BaseEven
Deletes: strings.Split(strings.ReplaceAll(r.Form.Get("fs_delete_paths"), " ", ""), ","),
MkDirs: strings.Split(strings.ReplaceAll(r.Form.Get("fs_mkdir_paths"), " ", ""), ","),
Exist: strings.Split(strings.ReplaceAll(r.Form.Get("fs_exist_paths"), " ", ""), ","),
Copy: getKeyValsFromPostFields(r, "fs_copy_source", "fs_copy_target"),
Compress: dataprovider.EventActionFsCompress{
Name: r.Form.Get("fs_compress_name"),
Paths: strings.Split(strings.ReplaceAll(r.Form.Get("fs_compress_paths"), " ", ""), ","),

3
internal/httpdtest/httpdtest.go
View file

@ -2504,6 +2504,9 @@ func compareEventActionFsConfigFields(expected, actual dataprovider.EventActionF
if err := compareKeyValues(expected.Renames, actual.Renames); err != nil {
return errors.New("fs renames mismatch")
}
if err := compareKeyValues(expected.Copy, actual.Copy); err != nil {
return errors.New("fs copy mismatch")
}
if len(expected.Deletes) != len(actual.Deletes) {
return errors.New("fs deletes mismatch")
}

79
internal/sftpd/internal_test.go
View file

@ -533,7 +533,7 @@ func TestSSHCommandErrors(t *testing.T) {
user.QuotaFiles = 1
user.UsedQuotaFiles = 2
cmd.connection.User = user
fs, err := cmd.connection.User.GetFilesystem("123")
_, err = cmd.connection.User.GetFilesystem("123")
assert.NoError(t, err)
err = cmd.handle()
assert.EqualError(t, err, common.ErrQuotaExceeded.Error())
@ -599,22 +599,6 @@ func TestSSHCommandErrors(t *testing.T) {
cmd.connection.User.Permissions = make(map[string][]string)
cmd.connection.User.Permissions["/"] = []string{dataprovider.PermAny}
if runtime.GOOS != osWindows {
aDir := filepath.Join(os.TempDir(), "adir")
err = os.MkdirAll(aDir, os.ModePerm)
assert.NoError(t, err)
tmpFile := filepath.Join(aDir, "testcopy")
err = os.WriteFile(tmpFile, []byte("aaa"), os.ModePerm)
assert.NoError(t, err)
err = os.Chmod(aDir, 0001)
assert.NoError(t, err)
err = cmd.checkCopyDestination(fs, tmpFile)
assert.Error(t, err)
err = os.Chmod(aDir, os.ModePerm)
assert.NoError(t, err)
err = os.Remove(tmpFile)
assert.NoError(t, err)
}
common.WaitForTransfers(1)
_, err = cmd.getSystemCommand()
@ -743,8 +727,6 @@ func TestSSHCommandsRemoteFs(t *testing.T) {
}
err = cmd.handleSFTPGoRemove()
assert.Error(t, err)
// the user has no permissions
assert.False(t, cmd.hasCopyPermissions("", "", nil))
}
func TestSSHCmdGetFsErrors(t *testing.T) {
@ -778,30 +760,10 @@ func TestSSHCmdGetFsErrors(t *testing.T) {
connection: connection,
args: []string{"path1", "path2"},
}
_, _, _, _, _, _, err = cmd.getFsAndCopyPaths() //nolint:dogsled
assert.Error(t, err)
user = dataprovider.User{}
user.HomeDir = filepath.Join(os.TempDir(), "home")
user.VirtualFolders = append(connection.User.VirtualFolders, vfs.VirtualFolder{
BaseVirtualFolder: vfs.BaseVirtualFolder{
MappedPath: "relative",
},
VirtualPath: "/vpath",
})
connection.User = user
err = os.MkdirAll(user.GetHomeDir(), os.ModePerm)
assert.NoError(t, err)
cmd = sshCommand{
command: "sftpgo-copy",
connection: connection,
args: []string{"path1", "/vpath/path2"},
}
_, _, _, _, _, _, err = cmd.getFsAndCopyPaths() //nolint:dogsled
err = cmd.handleSFTPGoCopy()
assert.Error(t, err)
err = os.Remove(user.GetHomeDir())
err = os.RemoveAll(user.GetHomeDir())
assert.NoError(t, err)
}
@ -2021,41 +1983,6 @@ func TestCertCheckerInitErrors(t *testing.T) {
assert.NoError(t, err)
}
func TestRecursiveCopyErrors(t *testing.T) {
permissions := make(map[string][]string)
permissions["/"] = []string{dataprovider.PermAny}
user := dataprovider.User{
BaseUser: sdk.BaseUser{
Permissions: permissions,
HomeDir: os.TempDir(),
},
}
fs, err := user.GetFilesystem("123")
assert.NoError(t, err)
conn := &Connection{
BaseConnection: common.NewBaseConnection("", common.ProtocolSFTP, "", "", user),
}
sshCmd := sshCommand{
command: "sftpgo-copy",
connection: conn,
args: []string{"adir", "another"},
}
// try to copy a missing directory
sshCmd.connection.User.Permissions["/another"] = []string{
dataprovider.PermCreateDirs,
dataprovider.PermCreateSymlinks,
dataprovider.PermListItems,
}
err = sshCmd.checkRecursiveCopyPermissions(fs, fs, "adir", "another/sub", "/adir", "/another/sub")
assert.Error(t, err)
sshCmd.connection.User.Permissions["/another"] = []string{
dataprovider.PermListItems,
dataprovider.PermCreateDirs,
}
err = sshCmd.checkRecursiveCopyPermissions(fs, fs, "adir", "another", "/adir/sub", "/another/sub/dir")
assert.Error(t, err)
}
func TestSFTPSubSystem(t *testing.T) {
permissions := make(map[string][]string)
permissions["/"] = []string{dataprovider.PermAny}

136
internal/sftpd/sftpd_test.go
View file

@ -8805,13 +8805,13 @@ func TestSSHCopy(t *testing.T) {
_, err = client.Stat(testDir1)
assert.Error(t, err)
_, err = runSSHCommand(fmt.Sprintf("sftpgo-copy %v", path.Join(vdirPath1, testDir1)), user, usePubKey)
_, err = runSSHCommand(fmt.Sprintf("sftpgo-copy %s", path.Join(vdirPath1, testDir1)), user, usePubKey)
assert.Error(t, err)
_, err = runSSHCommand("sftpgo-copy", user, usePubKey)
assert.Error(t, err)
_, err = runSSHCommand(fmt.Sprintf("sftpgo-copy %v %v", testFileName, testFileName+".linkcopy"), user, usePubKey)
_, err = runSSHCommand(fmt.Sprintf("sftpgo-copy %s %s", testFileName, testFileName+".linkcopy"), user, usePubKey)
assert.Error(t, err)
out, err := runSSHCommand(fmt.Sprintf("sftpgo-copy %v %v", path.Join(vdirPath1, testDir1), "."), user, usePubKey)
out, err := runSSHCommand(fmt.Sprintf("sftpgo-copy %s %s", path.Join(vdirPath1, testDir1), "."), user, usePubKey)
if assert.NoError(t, err) {
assert.Equal(t, "OK\n", string(out))
fi, err := client.Stat(testDir1)
@ -8826,7 +8826,13 @@ func TestSSHCopy(t *testing.T) {
_, err = runSSHCommand(fmt.Sprintf("sftpgo-copy %v %v", "missing\\ dir", "."), user, usePubKey)
assert.Error(t, err)
_, err = runSSHCommand(fmt.Sprintf("sftpgo-copy %v %v", path.Join(vdirPath1, testDir1), "."), user, usePubKey)
assert.Error(t, err)
if assert.NoError(t, err) {
// all files are overwritten, quota must remain unchanged
user, _, err = httpdtest.GetUserByUsername(user.Username, http.StatusOK)
assert.NoError(t, err)
assert.Equal(t, 6, user.UsedQuotaFiles)
assert.Equal(t, 3*testFileSize+3*testFileSize1, user.UsedQuotaSize)
}
out, err = runSSHCommand(fmt.Sprintf("sftpgo-copy %v %v", path.Join(vdirPath2, testDir1, testFileName), testFileName+".copy"),
user, usePubKey)
if assert.NoError(t, err) {
@ -8872,9 +8878,13 @@ func TestSSHCopy(t *testing.T) {
assert.Equal(t, 2*testFileSize+2*testFileSize1, f.UsedQuotaSize)
assert.Equal(t, 4, f.UsedQuotaFiles)
}
_, err = runSSHCommand(fmt.Sprintf("sftpgo-copy %v %v", path.Join(vdirPath2, ".."), "newdir"), user, usePubKey)
assert.Error(t, err)
// cross folder copy
newDir := "newdir"
_, err = runSSHCommand(fmt.Sprintf("sftpgo-copy %v %v", path.Join(vdirPath2, ".."), newDir), user, usePubKey)
assert.NoError(t, err)
_, err = client.Stat(newDir)
assert.NoError(t, err)
// denied pattern
_, err = runSSHCommand(fmt.Sprintf("sftpgo-copy %v %v", path.Join(testDir, testFileName), testFileName+".denied"), user, usePubKey)
assert.Error(t, err)
if runtime.GOOS != osWindows {
@ -8883,9 +8893,8 @@ func TestSSHCopy(t *testing.T) {
assert.NoError(t, err)
err = os.Chmod(subPath, 0001)
assert.NoError(t, err)
// c.connection.fs.GetDirSize(fsSourcePath) will fail scanning subdirs
// checkRecursiveCopyPermissions will work since it will skip subdirs with no permissions
_, err = runSSHCommand(fmt.Sprintf("sftpgo-copy %v %v", vdirPath1, "newdir"), user, usePubKey)
// listing contents for subdirs with no permissions will fail
_, err = runSSHCommand(fmt.Sprintf("sftpgo-copy %v %v", vdirPath1, "newdir1"), user, usePubKey)
assert.Error(t, err)
err = os.Chmod(subPath, os.ModePerm)
assert.NoError(t, err)
@ -8897,12 +8906,6 @@ func TestSSHCopy(t *testing.T) {
err = os.Chmod(filepath.Join(user.GetHomeDir(), testDir1), os.ModePerm)
assert.NoError(t, err)
err = os.RemoveAll(filepath.Join(user.GetHomeDir(), "vdir1"))
assert.NoError(t, err)
err = os.Chmod(user.GetHomeDir(), 0555)
assert.NoError(t, err)
_, err = runSSHCommand(fmt.Sprintf("sftpgo-copy %v %v", path.Join(vdirPath2), "/vdir1"), user, usePubKey)
assert.Error(t, err)
err = os.Chmod(user.GetHomeDir(), os.ModePerm)
assert.NoError(t, err)
}
@ -8967,16 +8970,17 @@ func TestSSHCopyPermissions(t *testing.T) {
if assert.NoError(t, err) {
assert.True(t, info.Mode().IsRegular())
}
// now create a symlink, dir2 has no create symlink permission
// now create a symlink, dir2 has no create symlink permission, but symlinks will be ignored
err = client.Symlink(path.Join("/", testDir, testFileName), path.Join("/", testDir, testFileName+".link"))
assert.NoError(t, err)
_, err = runSSHCommand(fmt.Sprintf("sftpgo-copy %v %v", path.Join("/", testDir), "/dir2/sub"), user, usePubKey)
assert.Error(t, err)
assert.NoError(t, err)
_, err = runSSHCommand(fmt.Sprintf("sftpgo-copy %v %v", path.Join("/", testDir), "/newdir"), user, usePubKey)
assert.NoError(t, err)
// now delete the file and copy inside /dir3
err = client.Remove(path.Join("/", testDir, testFileName))
assert.NoError(t, err)
// the symlink will be skipped, so no errors
_, err = runSSHCommand(fmt.Sprintf("sftpgo-copy %v %v", path.Join("/", testDir), "/dir3"), user, usePubKey)
assert.NoError(t, err)
@ -9073,6 +9077,11 @@ func TestSSHCopyQuotaLimits(t *testing.T) {
assert.NoError(t, err)
_, err = runSSHCommand(fmt.Sprintf("sftpgo-remove %v", path.Join(vdirPath2, testDir)), user, usePubKey)
assert.NoError(t, err)
// remove partially copied dirs
_, err = runSSHCommand(fmt.Sprintf("sftpgo-remove %v", testDir+"_copy"), user, usePubKey)
assert.NoError(t, err)
_, err = runSSHCommand(fmt.Sprintf("sftpgo-remove %v", path.Join(vdirPath2, testDir+"_copy")), user, usePubKey)
assert.NoError(t, err)
user, _, err = httpdtest.GetUserByUsername(user.Username, http.StatusOK)
assert.NoError(t, err)
assert.Equal(t, 0, user.UsedQuotaFiles)
@ -9150,33 +9159,6 @@ func TestSSHCopyQuotaLimits(t *testing.T) {
assert.NoError(t, err)
}
func TestSSHCopyRemoveNonLocalFs(t *testing.T) {
usePubKey := true
localUser, _, err := httpdtest.AddUser(getTestUser(usePubKey), http.StatusCreated)
assert.NoError(t, err)
sftpUser, _, err := httpdtest.AddUser(getTestSFTPUser(usePubKey), http.StatusCreated)
assert.NoError(t, err)
conn, client, err := getSftpClient(sftpUser, usePubKey)
if assert.NoError(t, err) {
defer conn.Close()
defer client.Close()
testDir := "test"
err = client.Mkdir(testDir)
assert.NoError(t, err)
_, err = runSSHCommand(fmt.Sprintf("sftpgo-copy %v %v", testDir, testDir+"_copy"), sftpUser, usePubKey)
assert.Error(t, err)
_, err = runSSHCommand(fmt.Sprintf("sftpgo-remove %v", testDir), sftpUser, usePubKey)
assert.Error(t, err)
}
_, err = httpdtest.RemoveUser(sftpUser, http.StatusOK)
assert.NoError(t, err)
_, err = httpdtest.RemoveUser(localUser, http.StatusOK)
assert.NoError(t, err)
err = os.RemoveAll(localUser.GetHomeDir())
assert.NoError(t, err)
}
func TestSSHRemove(t *testing.T) {
usePubKey := false
u := getTestUser(usePubKey)
@ -9244,7 +9226,7 @@ func TestSSHRemove(t *testing.T) {
err = client.Symlink(testFileName, testFileName+".link")
assert.NoError(t, err)
_, err = runSSHCommand(fmt.Sprintf("sftpgo-remove %v", testFileName+".link"), user, usePubKey)
assert.Error(t, err)
assert.NoError(t, err)
_, err = runSSHCommand("sftpgo-remove /vdir1", user, usePubKey)
assert.Error(t, err)
_, err = runSSHCommand("sftpgo-remove /", user, usePubKey)
@ -9490,6 +9472,67 @@ func TestBasicGitCommands(t *testing.T) {
assert.NoError(t, err)
}
func TestGitIncludedVirtualFolders(t *testing.T) {
if len(gitPath) == 0 || len(sshPath) == 0 || runtime.GOOS == osWindows {
t.Skip("git and/or ssh command not found or OS is windows, unable to execute this test")
}
usePubKey := true
repoName := "trepo"
u := getTestUser(usePubKey)
u.QuotaFiles = 10000
mappedPath := filepath.Join(os.TempDir(), "repo")
folderName := filepath.Base(mappedPath)
u.VirtualFolders = append(u.VirtualFolders, vfs.VirtualFolder{
BaseVirtualFolder: vfs.BaseVirtualFolder{
Name: folderName,
MappedPath: mappedPath,
},
VirtualPath: "/" + repoName,
QuotaFiles: -1,
QuotaSize: -1,
})
user, _, err := httpdtest.AddUser(u, http.StatusCreated)
assert.NoError(t, err)
clonePath := filepath.Join(homeBasePath, repoName)
err = os.RemoveAll(user.GetHomeDir())
assert.NoError(t, err)
err = os.RemoveAll(filepath.Join(homeBasePath, repoName))
assert.NoError(t, err)
out, err := initGitRepo(mappedPath)
assert.NoError(t, err, "unexpected error, out: %v", string(out))
out, err = cloneGitRepo(homeBasePath, "/"+repoName, user.Username)
assert.NoError(t, err, "unexpected error, out: %v", string(out))
out, err = addFileToGitRepo(clonePath, 128)
assert.NoError(t, err, "unexpected error, out: %v", string(out))
out, err = pushToGitRepo(clonePath)
assert.NoError(t, err, "unexpected error, out: %v", string(out))
user, _, err = httpdtest.GetUserByUsername(user.Username, http.StatusOK)
assert.NoError(t, err)
assert.Greater(t, user.UsedQuotaFiles, 0)
assert.Greater(t, user.UsedQuotaSize, int64(0))
folder, _, err := httpdtest.GetFolderByName(folderName, http.StatusOK)
assert.NoError(t, err)
assert.Equal(t, user.UsedQuotaFiles, folder.UsedQuotaFiles)
assert.Equal(t, user.UsedQuotaSize, folder.UsedQuotaSize)
_, err = httpdtest.RemoveUser(user, http.StatusOK)
assert.NoError(t, err)
err = os.RemoveAll(user.GetHomeDir())
assert.NoError(t, err)
_, err = httpdtest.RemoveFolder(vfs.BaseVirtualFolder{Name: folderName}, http.StatusOK)
assert.NoError(t, err)
err = os.RemoveAll(mappedPath)
assert.NoError(t, err)
err = os.RemoveAll(clonePath)
assert.NoError(t, err)
}
func TestGitQuotaVirtualFolders(t *testing.T) {
if len(gitPath) == 0 || len(sshPath) == 0 || runtime.GOOS == osWindows {
t.Skip("git and/or ssh command not found or OS is windows, unable to execute this test")
@ -9547,7 +9590,6 @@ func TestGitQuotaVirtualFolders(t *testing.T) {
_, err = httpdtest.RemoveUser(user, http.StatusOK)
assert.NoError(t, err)
err = os.RemoveAll(user.GetHomeDir())
assert.NoError(t, err)
_, err = httpdtest.RemoveFolder(vfs.BaseVirtualFolder{Name: folderName}, http.StatusOK)

244
internal/sftpd/ssh_cmd.go
View file

@ -31,7 +31,6 @@ import (
"sync"
"github.com/google/shlex"
fscopy "github.com/otiai10/copy"
"github.com/sftpgo/sdk"
"golang.org/x/crypto/ssh"
@ -88,7 +87,7 @@ func processSSHCommand(payload []byte, connection *Connection, enabledSSHCommand
var msg sshSubsystemExecMsg
if err := ssh.Unmarshal(payload, &msg); err == nil {
name, args, err := parseCommandPayload(msg.Command)
connection.Log(logger.LevelDebug, "new ssh command: %#v args: %v num args: %v user: %v, error: %v",
connection.Log(logger.LevelDebug, "new ssh command: %q args: %v num args: %d user: %s, error: %v",
name, args, len(args), connection.User.Username, err)
if err == nil && util.Contains(enabledSSHCommands, name) {
connection.command = msg.Command
@ -147,7 +146,7 @@ func (c *sshCommand) handle() (err error) {
} else if c.command == "cd" {
c.sendExitStatus(nil)
} else if c.command == "pwd" {
// hard coded response to "/"
// hard coded response to the start directory
c.connection.channel.Write([]byte(util.CleanPath(c.connection.User.Filters.StartDirectory) + "\n")) //nolint:errcheck
c.sendExitStatus(nil)
} else if c.command == "sftpgo-copy" {
@ -159,67 +158,15 @@ func (c *sshCommand) handle() (err error) {
}
func (c *sshCommand) handleSFTPGoCopy() error {
fsSrc, fsDst, sshSourcePath, sshDestPath, fsSourcePath, fsDestPath, err := c.getFsAndCopyPaths()
if err != nil {
sshSourcePath := c.getSourcePath()
sshDestPath := c.getDestPath()
if sshSourcePath == "" || sshDestPath == "" || len(c.args) != 2 {
return c.sendErrorResponse(errors.New("usage sftpgo-copy <source dir path> <destination dir path>"))
}
c.connection.Log(logger.LevelDebug, "requested copy %q -> %q", sshSourcePath, sshDestPath)
if err := c.connection.Copy(sshSourcePath, sshDestPath); err != nil {
return c.sendErrorResponse(err)
}
if !c.isLocalCopy(sshSourcePath, sshDestPath) {
return c.sendErrorResponse(errUnsupportedConfig)
}
if err := c.checkCopyDestination(fsDst, fsDestPath); err != nil {
return c.sendErrorResponse(c.connection.GetFsError(fsDst, err))
}
c.connection.Log(logger.LevelDebug, "requested copy %#v -> %#v sftp paths %#v -> %#v",
fsSourcePath, fsDestPath, sshSourcePath, sshDestPath)
fi, err := fsSrc.Lstat(fsSourcePath)
if err != nil {
return c.sendErrorResponse(c.connection.GetFsError(fsSrc, err))
}
if err := c.checkCopyPermissions(fsSrc, fsDst, fsSourcePath, fsDestPath, sshSourcePath, sshDestPath, fi); err != nil {
return c.sendErrorResponse(err)
}
filesNum := 0
filesSize := int64(0)
if fi.IsDir() {
filesNum, filesSize, err = fsSrc.GetDirSize(fsSourcePath)
if err != nil {
return c.sendErrorResponse(c.connection.GetFsError(fsSrc, err))
}
if c.connection.User.HasVirtualFoldersInside(sshSourcePath) {
err := errors.New("unsupported copy source: the source directory contains virtual folders")
return c.sendErrorResponse(err)
}
if c.connection.User.HasVirtualFoldersInside(sshDestPath) {
err := errors.New("unsupported copy source: the destination directory contains virtual folders")
return c.sendErrorResponse(err)
}
} else if fi.Mode().IsRegular() {
if ok, _ := c.connection.User.IsFileAllowed(sshDestPath); !ok {
err := errors.New("unsupported copy destination: this file is not allowed")
return c.sendErrorResponse(err)
}
filesNum = 1
filesSize = fi.Size()
} else {
err := errors.New("unsupported copy source: only files and directories are supported")
return c.sendErrorResponse(err)
}
if err := c.checkCopyQuota(filesNum, filesSize, sshDestPath); err != nil {
return c.sendErrorResponse(err)
}
c.connection.Log(logger.LevelDebug, "start copy %#v -> %#v", fsSourcePath, fsDestPath)
err = fscopy.Copy(fsSourcePath, fsDestPath, fscopy.Options{
OnSymlink: func(src string) fscopy.SymlinkAction {
return fscopy.Skip
},
})
if err != nil {
return c.sendErrorResponse(c.connection.GetFsError(fsSrc, err))
}
c.updateQuota(sshDestPath, filesNum, filesSize)
c.connection.channel.Write([]byte("OK\n")) //nolint:errcheck
c.sendExitStatus(nil)
return nil
@ -230,52 +177,9 @@ func (c *sshCommand) handleSFTPGoRemove() error {
if err != nil {
return c.sendErrorResponse(err)
}
if !c.connection.User.HasPerm(dataprovider.PermDelete, path.Dir(sshDestPath)) {
return c.sendErrorResponse(common.ErrPermissionDenied)
}
fs, fsDestPath, err := c.connection.GetFsAndResolvedPath(sshDestPath)
if err != nil {
if err := c.connection.RemoveAll(sshDestPath); err != nil {
return c.sendErrorResponse(err)
}
if !vfs.IsLocalOrCryptoFs(fs) {
return c.sendErrorResponse(errUnsupportedConfig)
}
fi, err := fs.Lstat(fsDestPath)
if err != nil {
return c.sendErrorResponse(c.connection.GetFsError(fs, err))
}
filesNum := 0
filesSize := int64(0)
if fi.IsDir() {
filesNum, filesSize, err = fs.GetDirSize(fsDestPath)
if err != nil {
return c.sendErrorResponse(c.connection.GetFsError(fs, err))
}
if sshDestPath == "/" {
err := errors.New("removing root dir is not allowed")
return c.sendErrorResponse(err)
}
if c.connection.User.HasVirtualFoldersInside(sshDestPath) {
err := errors.New("unsupported remove source: this directory contains virtual folders")
return c.sendErrorResponse(err)
}
if c.connection.User.IsVirtualFolder(sshDestPath) {
err := errors.New("unsupported remove source: this directory is a virtual folder")
return c.sendErrorResponse(err)
}
} else if fi.Mode().IsRegular() {
filesNum = 1
filesSize = fi.Size()
} else {
err := errors.New("unsupported remove source: only files and directories are supported")
return c.sendErrorResponse(err)
}
err = os.RemoveAll(fsDestPath)
if err != nil {
return c.sendErrorResponse(err)
}
c.updateQuota(sshDestPath, -filesNum, -filesSize)
c.connection.channel.Write([]byte("OK\n")) //nolint:errcheck
c.sendExitStatus(nil)
return nil
@ -572,89 +476,6 @@ func (c *sshCommand) cleanCommandPath(name string) string {
return result
}
func (c *sshCommand) getFsAndCopyPaths() (vfs.Fs, vfs.Fs, string, string, string, string, error) {
sshSourcePath := strings.TrimSuffix(c.getSourcePath(), "/")
sshDestPath := c.getDestPath()
if strings.HasSuffix(sshDestPath, "/") {
sshDestPath = path.Join(sshDestPath, path.Base(sshSourcePath))
}
if sshSourcePath == "" || sshDestPath == "" || len(c.args) != 2 {
err := errors.New("usage sftpgo-copy <source dir path> <destination dir path>")
return nil, nil, "", "", "", "", err
}
fsSrc, fsSourcePath, err := c.connection.GetFsAndResolvedPath(sshSourcePath)
if err != nil {
return nil, nil, "", "", "", "", err
}
fsDst, fsDestPath, err := c.connection.GetFsAndResolvedPath(sshDestPath)
if err != nil {
return nil, nil, "", "", "", "", err
}
return fsSrc, fsDst, sshSourcePath, sshDestPath, fsSourcePath, fsDestPath, nil
}
func (c *sshCommand) hasCopyPermissions(sshSourcePath, sshDestPath string, srcInfo os.FileInfo) bool {
if !c.connection.User.HasPerm(dataprovider.PermListItems, path.Dir(sshSourcePath)) {
return false
}
if srcInfo.IsDir() {
return c.connection.User.HasPerm(dataprovider.PermCreateDirs, path.Dir(sshDestPath))
} else if srcInfo.Mode()&os.ModeSymlink != 0 {
return c.connection.User.HasPerm(dataprovider.PermCreateSymlinks, path.Dir(sshDestPath))
}
return c.connection.User.HasPerm(dataprovider.PermUpload, path.Dir(sshDestPath))
}
// fsSourcePath must be a directory
func (c *sshCommand) checkRecursiveCopyPermissions(fsSrc vfs.Fs, fsDst vfs.Fs, fsSourcePath, fsDestPath,
sshSourcePath, sshDestPath string,
) error {
if !c.connection.User.HasPerm(dataprovider.PermCreateDirs, path.Dir(sshDestPath)) {
return common.ErrPermissionDenied
}
if !c.connection.User.HasPermissionsInside(sshSourcePath) &&
!c.connection.User.HasPermissionsInside(sshDestPath) {
// if there are no subdirs with defined permissions we can just check source and destination paths
dstPerms := []string{
dataprovider.PermCreateDirs,
dataprovider.PermCreateSymlinks,
dataprovider.PermUpload,
}
if c.connection.User.HasPerm(dataprovider.PermListItems, sshSourcePath) &&
c.connection.User.HasPerms(dstPerms, sshDestPath) {
return nil
}
// we don't return an error here because we checked all the required permissions above
// for example the directory could not have symlinks inside, so we have to walk to check
// permissions for each item
}
return fsSrc.Walk(fsSourcePath, func(walkedPath string, info os.FileInfo, err error) error {
if err != nil {
return c.connection.GetFsError(fsSrc, err)
}
fsDstSubPath := strings.Replace(walkedPath, fsSourcePath, fsDestPath, 1)
sshSrcSubPath := fsSrc.GetRelativePath(walkedPath)
sshDstSubPath := fsDst.GetRelativePath(fsDstSubPath)
if !c.hasCopyPermissions(sshSrcSubPath, sshDstSubPath, info) {
return common.ErrPermissionDenied
}
return nil
})
}
func (c *sshCommand) checkCopyPermissions(fsSrc vfs.Fs, fsDst vfs.Fs, fsSourcePath, fsDestPath, sshSourcePath,
sshDestPath string, info os.FileInfo,
) error {
if info.IsDir() {
return c.checkRecursiveCopyPermissions(fsSrc, fsDst, fsSourcePath, fsDestPath, sshSourcePath, sshDestPath)
}
if !c.hasCopyPermissions(sshSourcePath, sshDestPath, info) {
return c.connection.GetPermissionDeniedError()
}
return nil
}
func (c *sshCommand) getRemovePath() (string, error) {
sshDestPath := c.getDestPath()
if sshDestPath == "" || len(c.args) != 1 {
@ -675,49 +496,6 @@ func (c *sshCommand) isLocalPath(virtualPath string) bool {
return folder.FsConfig.Provider == sdk.LocalFilesystemProvider
}
func (c *sshCommand) isLocalCopy(virtualSourcePath, virtualTargetPath string) bool {
if !c.isLocalPath(virtualSourcePath) {
return false
}
return c.isLocalPath(virtualTargetPath)
}
func (c *sshCommand) checkCopyDestination(fs vfs.Fs, fsDestPath string) error {
_, err := fs.Lstat(fsDestPath)
if err == nil {
err := errors.New("invalid copy destination: cannot overwrite an existing file or directory")
return err
} else if !fs.IsNotExist(err) {
return err
}
return nil
}
func (c *sshCommand) checkCopyQuota(numFiles int, filesSize int64, requestPath string) error {
quotaResult, _ := c.connection.HasSpace(true, false, requestPath)
if !quotaResult.HasSpace {
return common.ErrQuotaExceeded
}
if quotaResult.QuotaFiles > 0 {
remainingFiles := quotaResult.GetRemainingFiles()
if remainingFiles < numFiles {
c.connection.Log(logger.LevelDebug, "copy not allowed, file limit will be exceeded, "+
"remaining files: %v to copy: %v", remainingFiles, numFiles)
return common.ErrQuotaExceeded
}
}
if quotaResult.QuotaSize > 0 {
remainingSize := quotaResult.GetRemainingSize()
if remainingSize < filesSize {
c.connection.Log(logger.LevelDebug, "copy not allowed, size limit will be exceeded, "+
"remaining size: %v to copy: %v", remainingSize, filesSize)
return common.ErrQuotaExceeded
}
}
return nil
}
func (c *sshCommand) getSizeForPath(fs vfs.Fs, name string) (int, int64, error) {
if dataprovider.GetQuotaTracking() > 0 {
fi, err := fs.Lstat(name)
@ -760,7 +538,7 @@ func (c *sshCommand) sendExitStatus(err error) {
}
if err != nil {
status = uint32(1)
c.connection.Log(logger.LevelError, "command failed: %#v args: %v user: %v err: %v",
c.connection.Log(logger.LevelError, "command failed: %q args: %v user: %s err: %v",
c.command, c.args, c.connection.User.Username, err)
}
exitStatus := sshSubsystemExitStatus{

20
internal/util/util.go
View file

@ -422,6 +422,26 @@ func GenerateEd25519Keys(file string) error {
return os.WriteFile(file+".pub", ssh.MarshalAuthorizedKey(pub), 0600)
}
// IsDirOverlapped returns true if dir1 and dir2 overlap
func IsDirOverlapped(dir1, dir2 string, fullCheck bool, separator string) bool {
if dir1 == dir2 {
return true
}
if fullCheck {
if len(dir1) > len(dir2) {
if strings.HasPrefix(dir1, dir2+separator) {
return true
}
}
if len(dir2) > len(dir1) {
if strings.HasPrefix(dir2, dir1+separator) {
return true
}
}
}
return false
}
// GetDirsForVirtualPath returns all the directory for the given path in reverse order
// for example if the path is: /1/2/3/4 it returns:
// [ "/1/2/3/4", "/1/2/3", "/1/2", "/1", "/" ]

12
internal/vfs/filesystem.go
View file

@ -15,6 +15,8 @@
package vfs
import (
"os"
"github.com/sftpgo/sdk"
"github.com/drakkan/sftpgo/v2/internal/kms"
@ -149,6 +151,16 @@ func (f *Filesystem) IsSameResource(other Filesystem) bool {
}
}
// GetPathSeparator returns the path separator
func (f *Filesystem) GetPathSeparator() string {
switch f.Provider {
case sdk.LocalFilesystemProvider, sdk.CryptedFilesystemProvider:
return string(os.PathSeparator)
default:
return "/"
}
}
// Validate verifies the FsConfig matching the configured provider and sets all other
// Filesystem.*Config to their zero value if successful
func (f *Filesystem) Validate(additionalData string) error {

2
internal/vfs/osfs.go
View file

@ -448,7 +448,7 @@ func (fs *OsFs) findFirstExistingDir(path string) (string, error) {
return "", err
}
if !fileInfo.IsDir() {
return "", fmt.Errorf("resolved path is not a dir: %#v", p)
return "", fmt.Errorf("resolved path is not a dir: %q", p)
}
err = fs.isSubDir(p)
return p, err

5
openapi/openapi.yaml
View file

@ -6502,6 +6502,10 @@ components:
type: array
items:
type: string
copy:
type: array
items:
$ref: '#/components/schemas/KeyValue'
compress:
$ref: '#/components/schemas/EventActionFsCompress'
EventActionPasswordExpiration:
@ -6661,6 +6665,7 @@ components:
- rename
- mkdir
- rmdir
- copy
- ssh_cmd
provider_events:
type: array

80
templates/webadmin/eventaction.html
View file

@ -617,6 +617,56 @@ along with this program. If not, see <https://www.gnu.org/licenses/>.
</div>
</div>
<div class="card bg-light mb-3 action-type action-fs-type action-fs-copy">
<div class="card-header">
<b>Copy</b>
</div>
<div class="card-body">
<h6 class="card-title mb-4">Paths to copy as seen by SFTPGo users. Placeholders are supported. The required permissions are granted automatically</h6>
<div class="form-group row">
<div class="col-md-12 form_field_fs_copy_outer">
{{range $idx, $val := .Action.Options.FsConfig.Copy}}
<div class="row form_field_fs_copy_outer_row">
<div class="form-group col-md-5">
<input type="text" class="form-control" id="idFsCopySource{{$idx}}" name="fs_copy_source{{$idx}}" placeholder="Source path" value="{{$val.Key}}">
</div>
<div class="form-group col-md-5">
<input type="text" class="form-control" id="idFsCopyTarget{{$idx}}" name="fs_copy_target{{$idx}}" placeholder="Target path" value="{{$val.Value}}">
</div>
<div class="form-group col-md-1"></div>
<div class="form-group col-md-1">
<button class="btn btn-circle btn-danger remove_fs_copy_btn_frm_field">
<i class="fas fa-trash"></i>
</button>
</div>
</div>
{{else}}
<div class="row form_field_fs_copy_outer_row">
<div class="form-group col-md-5">
<input type="text" class="form-control" id="idFsCopySource0" name="fs_copy_source0" placeholder="Source path" value="">
</div>
<div class="form-group col-md-5">
<input type="text" class="form-control" id="idFsCopyTarget0" name="fs_copy_target0" placeholder="Target path" value="">
</div>
<div class="form-group col-md-1"></div>
<div class="form-group col-md-1">
<button class="btn btn-circle btn-danger remove_fs_copy_btn_frm_field">
<i class="fas fa-trash"></i>
</button>
</div>
</div>
{{end}}
</div>
</div>
<div class="row mx-1">
<button type="button" class="btn btn-secondary add_new_fs_copy_field_btn">
<i class="fas fa-plus"></i> Add new
</button>
</div>
</div>
</div>
<div class="form-group row action-type action-fs-type action-fs-compress">
<label for="idFsCompressName" class="col-sm-2 col-form-label">Archive path</label>
<div class="col-sm-10">
@ -879,6 +929,33 @@ along with this program. If not, see <https://www.gnu.org/licenses/>.
$(this).closest(".form_field_fs_rename_outer_row").remove();
});
$("body").on("click", ".add_new_fs_copy_field_btn", function () {
var index = $(".form_field_fs_copy_outer").find(".form_field_fs_copy_outer_row").length;
while (document.getElementById("idFsCopySource"+index) != null){
index++;
}
$(".form_field_fs_copy_outer").append(`
<div class="row form_field_fs_copy_outer_row">
<div class="form-group col-md-5">
<input type="text" class="form-control" id="idFsCopySource${index}" name="fs_copy_source${index}" placeholder="Source path" value="">
</div>
<div class="form-group col-md-5">
<input type="text" class="form-control" id="idFsCopyTarget${index}" name="fs_copy_target${index}" placeholder="Target path" value="">
</div>
<div class="form-group col-md-1"></div>
<div class="form-group col-md-1">
<button class="btn btn-circle btn-danger remove_fs_copy_btn_frm_field">
<i class="fas fa-trash"></i>
</button>
</div>
</div>
`);
});
$("body").on("click", ".remove_fs_copy_btn_frm_field", function () {
$(this).closest(".form_field_fs_copy_outer_row").remove();
});
$("body").on("click", ".add_new_http_part_field_btn", function () {
var index = $(".form_field_http_part_outer").find(".form_field_http_part_outer_row").length;
while (document.getElementById("idHTTPPartName"+index) != null){
@ -966,6 +1043,9 @@ along with this program. If not, see <https://www.gnu.org/licenses/>.
case '5':
$('.action-fs-compress').show();
break;
case '6':
$('.action-fs-copy').show();
break;
}
}