merge main

This commit is contained in:
martabal 2023-10-11 20:50:36 +02:00
commit 00d7052b84
No known key found for this signature in database
GPG key ID: C00196E3148A52BD
162 changed files with 18072 additions and 14112 deletions

View file

@ -45,7 +45,7 @@ jobs:
uses: subosito/flutter-action@v2
with:
channel: "stable"
flutter-version: "3.13.3"
flutter-version: "3.13.6"
cache: true
- name: Create the Keystore

View file

@ -23,7 +23,7 @@ jobs:
uses: subosito/flutter-action@v2
with:
channel: "stable"
flutter-version: "3.13.3"
flutter-version: "3.13.6"
- name: Install dependencies
run: dart pub get

View file

@ -144,7 +144,7 @@ jobs:
uses: subosito/flutter-action@v2
with:
channel: "stable"
flutter-version: "3.13.3"
flutter-version: "3.13.6"
- name: Run tests
working-directory: ./mobile
run: flutter test -j 1

6808
cli/package-lock.json generated

File diff suppressed because it is too large Load diff

View file

@ -20,22 +20,9 @@ Immich doesn't have two-way synchronization ([yet](https://github.com/immich-app
The initial approach of Immich is to become a backup tool, primarily for mobile device usage. Thus, all the assets must be uploaded from the mobile client. The app was architectured to perform that job well.
### Why does my uploaded photo show up with the wrong date or time in Immich?
When a photo is initially uploaded Immich uses the create date of the file to determine where it belongs in the timeline. After that, background jobs will run that extract [exif metadata](https://en.wikipedia.org/wiki/Exif), including the CreateDate, to provide a more accurate date for the photo. If that is not available it will fallback to the modified date. If you want to ensure your photo has the right date, check the exif metadata before uploading.
If the timezone is incorrect in an uploaded photo, check the `DateTimeOriginal` exif field of the uploaded file. Immich uses the very competent library [exiftool-vendored.js](https://github.com/photostructure/exiftool-vendored.js#dates) to handle timezone parsing, but in some cases (like photos taken with DSLR cameras) it has to fallback on the local timezone. If you are using docker, this fallback will be UTC. (Note that even the photo backup app that can't be named [has the same bug!](https://photo.stackexchange.com/a/126978)) In Immich, it is possible to change this assumed fallback timezone system-wide by setting the timezone in the microservices docker container. You might need to run the "Extract Metadata" job after to effect the change.
As an example, the following modification of `docker-compose.yml` will set the timezone of the microservices container to be `Europe/Stockholm`
```
environment:
- TZ=Europe/Stockholm # <---- Add this line in the microservices config
```
### Why are only photos and not videos being uploaded to Immich?
This often happens when using a reverse proxy or cloudflare tunnel in front of Immich. Make sure to set your reverse proxy to allow large POST requests. In `nginx`, set `client_max_body_size 50000M;` or similar. Cloudflare tunnels are limited to 100 mb file sizes.
This often happens when using a reverse proxy or cloudflare tunnel in front of Immich. Make sure to set your reverse proxy to allow large POST requests. In `nginx`, set `client_max_body_size 50000M;` or similar. Cloudflare tunnels are limited to 100 mb file sizes. Also check the disk space of your reverse proxy, in some cases proxies caches requests to disk before passing them on, and if disk space runs out the request fails.
### Why is Immich slow on low-memory systems like the Raspberry Pi?

View file

@ -42,8 +42,26 @@ Finally, files can be deleted from Immich via the `Remove Offline Files` job. An
External libraries use import paths to determine which files to scan. Each library can have multiple import paths so that files from different locations can be added to the same library. Import paths are scanned recursively, and if a file is in multiple import paths, it will only be added once. If the import paths are edited in a way that an external file is no longer in any import path, it will be removed from the library in the same way a deleted file would. If the file is moved back to an import path, it will be added again as if it was a new file.
### Troubleshooting
Sometimes, an external library will not scan correctly. This can happen if the immich_server or immich_microservices can't access the files. Here are some things to check:
- Is the external path set correctly?
- In the docker-compose file, are the volumes mounted correctly?
- Are the volumes identical between the `server` and `microservices` container?
- Are the import paths set correctly, and do they match the path set in docker-compose file?
- Are the permissions set correctly?
If all else fails, you can always start a shell inside the container and check if the path is accessible. For example, `docker exec -it immich_microservices /bin/bash` will start a bash shell. If your import path, for instance, is `/data/import/photos`, you can check if the files are accessible by running `ls /data/import/photos`. Also check the `immich_server` container in the same way.
### Security Considerations
:::caution
Please read and understand this section before setting external paths, as there are important security considerations.
:::
For security purposes, each Immich user is disallowed to add external files by default. This is to prevent devastating [path traversal attacks](https://owasp.org/www-community/attacks/Path_Traversal). An admin can allow individual users to use external path feature via the `external path` setting found in the admin panel. Without the external path restriction, a user can add any image or video file on the Immich host filesystem to be imported into Immich, potentially allowing sensitive data to be accessed. If you are running Immich as root in your Docker setup (which is the default), all external file reads are done with root privileges. This is particularly dangerous if the Immich host is a shared server.
With the `external path` set, a user is restricted to accessing external files to files or directories within that path. The Immich admin should still be careful not set the external path too generously. For example, `user1` wants to read their photos in to `/home/user1`. A lazy admin sets that user's external path to `/home/` since it "gets the job done". However, that user will then be able to read all photos in `/home/user2/private-photos`, too! Please set the external path as specific as possible. If multiple folders must be added, do this using the docker volume mount feature described below.
@ -59,6 +77,10 @@ Some basic examples:
- `**/Raw/**` will exclude all files in any directory named `Raw`
- `*.(tif,jpg)` will exclude all files with the extension `.tif` or `.jpg`
### Nightly job
There is an automatic job that's run once a day and refreshes all modified files in all libraries as well as cleans up any libraries stuck in deletion.
## Usage
Let's show a concrete example where we add an existing gallery to Immich. Here, we have the following folders we want to add:

View file

@ -66,6 +66,10 @@ ORDER BY
"users"."email";
```
```sql title="Failed file movements"
SELECT * FROM "move_history";
```
## Users
```sql title="List"

9112
docs/package-lock.json generated

File diff suppressed because it is too large Load diff

View file

@ -17,10 +17,11 @@
"check": "tsc"
},
"dependencies": {
"@docusaurus/core": "^2.4.1",
"@docusaurus/preset-classic": "^2.4.1",
"@docusaurus/core": "^2.4.3",
"@docusaurus/preset-classic": "^2.4.3",
"@mdx-js/react": "^1.6.22",
"autoprefixer": "^10.4.13",
"classnames": "^2.3.2",
"clsx": "^1.2.1",
"docusaurus-lunr-search": "^2.3.2",
"docusaurus-preset-openapi": "^0.6.3",

View file

@ -1,7 +1,9 @@
{
// This file is not used in compilation. It is here just for a nice editor experience.
"extends": "@tsconfig/docusaurus/tsconfig.json",
"compilerOptions": {
"baseUrl": "."
"baseUrl": ".",
"module": "Node16"
}
}

View file

@ -16,13 +16,6 @@ from ..config import log
from ..schemas import ModelType
from .base import InferenceModel
_ST_TO_JINA_MODEL_NAME = {
"clip-ViT-B-16": "ViT-B-16::openai",
"clip-ViT-B-32": "ViT-B-32::openai",
"clip-ViT-B-32-multilingual-v1": "M-CLIP/XLM-Roberta-Large-Vit-B-32",
"clip-ViT-L-14": "ViT-L-14::openai",
}
class CLIPEncoder(InferenceModel):
_model_type = ModelType.CLIP
@ -36,11 +29,10 @@ class CLIPEncoder(InferenceModel):
) -> None:
if mode is not None and mode not in ("text", "vision"):
raise ValueError(f"Mode must be 'text', 'vision', or omitted; got '{mode}'")
if "vit-b" not in model_name.lower():
raise ValueError(f"Only ViT-B models are currently supported; got '{model_name}'")
if model_name not in _MODELS:
raise ValueError(f"Unknown model name {model_name}.")
self.mode = mode
jina_model_name = self._get_jina_model_name(model_name)
super().__init__(jina_model_name, cache_dir, **model_kwargs)
super().__init__(model_name, cache_dir, **model_kwargs)
def _download(self) -> None:
models: tuple[tuple[str, str], tuple[str, str]] = _MODELS[self.model_name]
@ -104,20 +96,6 @@ class CLIPEncoder(InferenceModel):
return outputs[0][0].tolist()
def _get_jina_model_name(self, model_name: str) -> str:
if model_name in _MODELS:
return model_name
elif model_name in _ST_TO_JINA_MODEL_NAME:
log.warn(
(
f"Sentence-Transformer models like '{model_name}' are not supported."
f"Using '{_ST_TO_JINA_MODEL_NAME[model_name]}' instead as it is the best match for '{model_name}'."
),
)
return _ST_TO_JINA_MODEL_NAME[model_name]
else:
raise ValueError(f"Unknown model name {model_name}.")
def _download_model(self, model_name: str, model_md5: str) -> bool:
# downloading logic is adapted from clip-server's CLIPOnnxModel class
download_model(

View file

@ -34,6 +34,7 @@ class ServerInfoNotifier extends StateNotifier<ServerInfoState> {
mapTileUrl: "https://tile.openstreetmap.org/{z}/{x}/{y}.png",
oauthButtonText: "",
trashDays: 30,
isInitialized: false,
),
isVersionMismatch: false,
versionMismatchErrorMessage: "",

View file

@ -11,6 +11,7 @@ Method | HTTP request | Description
------------- | ------------- | -------------
[**getExploreData**](SearchApi.md#getexploredata) | **GET** /search/explore |
[**search**](SearchApi.md#search) | **GET** /search |
[**searchPerson**](SearchApi.md#searchperson) | **GET** /search/person |
# **getExploreData**
@ -149,3 +150,58 @@ Name | Type | Description | Notes
[[Back to top]](#) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to Model list]](../README.md#documentation-for-models) [[Back to README]](../README.md)
# **searchPerson**
> List<PersonResponseDto> searchPerson(name)
### Example
```dart
import 'package:openapi/api.dart';
// TODO Configure API key authorization: cookie
//defaultApiClient.getAuthentication<ApiKeyAuth>('cookie').apiKey = 'YOUR_API_KEY';
// uncomment below to setup prefix (e.g. Bearer) for API key, if needed
//defaultApiClient.getAuthentication<ApiKeyAuth>('cookie').apiKeyPrefix = 'Bearer';
// TODO Configure API key authorization: api_key
//defaultApiClient.getAuthentication<ApiKeyAuth>('api_key').apiKey = 'YOUR_API_KEY';
// uncomment below to setup prefix (e.g. Bearer) for API key, if needed
//defaultApiClient.getAuthentication<ApiKeyAuth>('api_key').apiKeyPrefix = 'Bearer';
// TODO Configure HTTP Bearer authorization: bearer
// Case 1. Use String Token
//defaultApiClient.getAuthentication<HttpBearerAuth>('bearer').setAccessToken('YOUR_ACCESS_TOKEN');
// Case 2. Use Function which generate token.
// String yourTokenGeneratorFunction() { ... }
//defaultApiClient.getAuthentication<HttpBearerAuth>('bearer').setAccessToken(yourTokenGeneratorFunction);
final api_instance = SearchApi();
final name = name_example; // String |
try {
final result = api_instance.searchPerson(name);
print(result);
} catch (e) {
print('Exception when calling SearchApi->searchPerson: $e\n');
}
```
### Parameters
Name | Type | Description | Notes
------------- | ------------- | ------------- | -------------
**name** | **String**| |
### Return type
[**List<PersonResponseDto>**](PersonResponseDto.md)
### Authorization
[cookie](../README.md#cookie), [api_key](../README.md#api_key), [bearer](../README.md#bearer)
### HTTP request headers
- **Content-Type**: Not defined
- **Accept**: application/json
[[Back to top]](#) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to Model list]](../README.md#documentation-for-models) [[Back to README]](../README.md)

View file

@ -8,6 +8,7 @@ import 'package:openapi/api.dart';
## Properties
Name | Type | Description | Notes
------------ | ------------- | ------------- | -------------
**isInitialized** | **bool** | |
**loginPageMessage** | **String** | |
**mapTileUrl** | **String** | |
**oauthButtonText** | **String** | |

View file

@ -410,6 +410,20 @@ Name | Type | Description | Notes
### Example
```dart
import 'package:openapi/api.dart';
// TODO Configure API key authorization: cookie
//defaultApiClient.getAuthentication<ApiKeyAuth>('cookie').apiKey = 'YOUR_API_KEY';
// uncomment below to setup prefix (e.g. Bearer) for API key, if needed
//defaultApiClient.getAuthentication<ApiKeyAuth>('cookie').apiKeyPrefix = 'Bearer';
// TODO Configure API key authorization: api_key
//defaultApiClient.getAuthentication<ApiKeyAuth>('api_key').apiKey = 'YOUR_API_KEY';
// uncomment below to setup prefix (e.g. Bearer) for API key, if needed
//defaultApiClient.getAuthentication<ApiKeyAuth>('api_key').apiKeyPrefix = 'Bearer';
// TODO Configure HTTP Bearer authorization: bearer
// Case 1. Use String Token
//defaultApiClient.getAuthentication<HttpBearerAuth>('bearer').setAccessToken('YOUR_ACCESS_TOKEN');
// Case 2. Use Function which generate token.
// String yourTokenGeneratorFunction() { ... }
//defaultApiClient.getAuthentication<HttpBearerAuth>('bearer').setAccessToken(yourTokenGeneratorFunction);
final api_instance = UserApi();
final admin = true; // bool |
@ -434,7 +448,7 @@ Name | Type | Description | Notes
### Authorization
No authorization required
[cookie](../README.md#cookie), [api_key](../README.md#api_key), [bearer](../README.md#bearer)
### HTTP request headers

View file

@ -215,4 +215,56 @@ class SearchApi {
}
return null;
}
/// Performs an HTTP 'GET /search/person' operation and returns the [Response].
/// Parameters:
///
/// * [String] name (required):
Future<Response> searchPersonWithHttpInfo(String name,) async {
// ignore: prefer_const_declarations
final path = r'/search/person';
// ignore: prefer_final_locals
Object? postBody;
final queryParams = <QueryParam>[];
final headerParams = <String, String>{};
final formParams = <String, String>{};
queryParams.addAll(_queryParams('', 'name', name));
const contentTypes = <String>[];
return apiClient.invokeAPI(
path,
'GET',
queryParams,
postBody,
headerParams,
formParams,
contentTypes.isEmpty ? null : contentTypes.first,
);
}
/// Parameters:
///
/// * [String] name (required):
Future<List<PersonResponseDto>?> searchPerson(String name,) async {
final response = await searchPersonWithHttpInfo(name,);
if (response.statusCode >= HttpStatus.badRequest) {
throw ApiException(response.statusCode, await _decodeBodyBytes(response));
}
// When a remote server returns no body with a status of 204, we shall not decode it.
// At the time of writing this, `dart:convert` will throw an "Unexpected end of input"
// FormatException when trying to decode an empty string.
if (response.body.isNotEmpty && response.statusCode != HttpStatus.noContent) {
final responseBody = await _decodeBodyBytes(response);
return (await apiClient.deserializeAsync(responseBody, 'List<PersonResponseDto>') as List)
.cast<PersonResponseDto>()
.toList();
}
return null;
}
}

View file

@ -13,12 +13,15 @@ part of openapi.api;
class ServerConfigDto {
/// Returns a new [ServerConfigDto] instance.
ServerConfigDto({
required this.isInitialized,
required this.loginPageMessage,
required this.mapTileUrl,
required this.oauthButtonText,
required this.trashDays,
});
bool isInitialized;
String loginPageMessage;
String mapTileUrl;
@ -29,6 +32,7 @@ class ServerConfigDto {
@override
bool operator ==(Object other) => identical(this, other) || other is ServerConfigDto &&
other.isInitialized == isInitialized &&
other.loginPageMessage == loginPageMessage &&
other.mapTileUrl == mapTileUrl &&
other.oauthButtonText == oauthButtonText &&
@ -37,16 +41,18 @@ class ServerConfigDto {
@override
int get hashCode =>
// ignore: unnecessary_parenthesis
(isInitialized.hashCode) +
(loginPageMessage.hashCode) +
(mapTileUrl.hashCode) +
(oauthButtonText.hashCode) +
(trashDays.hashCode);
@override
String toString() => 'ServerConfigDto[loginPageMessage=$loginPageMessage, mapTileUrl=$mapTileUrl, oauthButtonText=$oauthButtonText, trashDays=$trashDays]';
String toString() => 'ServerConfigDto[isInitialized=$isInitialized, loginPageMessage=$loginPageMessage, mapTileUrl=$mapTileUrl, oauthButtonText=$oauthButtonText, trashDays=$trashDays]';
Map<String, dynamic> toJson() {
final json = <String, dynamic>{};
json[r'isInitialized'] = this.isInitialized;
json[r'loginPageMessage'] = this.loginPageMessage;
json[r'mapTileUrl'] = this.mapTileUrl;
json[r'oauthButtonText'] = this.oauthButtonText;
@ -62,6 +68,7 @@ class ServerConfigDto {
final json = value.cast<String, dynamic>();
return ServerConfigDto(
isInitialized: mapValueOfType<bool>(json, r'isInitialized')!,
loginPageMessage: mapValueOfType<String>(json, r'loginPageMessage')!,
mapTileUrl: mapValueOfType<String>(json, r'mapTileUrl')!,
oauthButtonText: mapValueOfType<String>(json, r'oauthButtonText')!,
@ -113,6 +120,7 @@ class ServerConfigDto {
/// The list of required keys that must be present in a JSON.
static const requiredKeys = <String>{
'isInitialized',
'loginPageMessage',
'mapTileUrl',
'oauthButtonText',

View file

@ -27,5 +27,10 @@ void main() {
// TODO
});
//Future<List<PersonResponseDto>> searchPerson(String name) async
test('test searchPerson', () async {
// TODO
});
});
}

View file

@ -16,6 +16,11 @@ void main() {
// final instance = ServerConfigDto();
group('test ServerConfigDto', () {
// bool isInitialized
test('to test the property `isInitialized`', () async {
// TODO
});
// String loginPageMessage
test('to test the property `loginPageMessage`', () async {
// TODO

View file

@ -1071,10 +1071,10 @@ packages:
dependency: "direct main"
description:
name: photo_manager
sha256: b2d81bd197323697d1b335e2e04cea2f67e11624ced77cfd02917a10afaeba73
sha256: "41eaa1d1fa51bac1c8f2f6debfd34074edcc6b330aa96bb3d33c3bc2fc6c8a5c"
url: "https://pub.dev"
source: hosted
version: "2.7.1"
version: "2.7.2"
platform:
dependency: transitive
description:

View file

@ -13,7 +13,7 @@ dependencies:
sdk: flutter
path_provider_ios:
photo_manager: ^2.5.0
photo_manager: ^2.7.2
flutter_hooks: ^0.18.6
hooks_riverpod: ^2.2.0
cached_network_image: ^3.2.2

View file

@ -3934,6 +3934,50 @@
]
}
},
"/search/person": {
"get": {
"operationId": "searchPerson",
"parameters": [
{
"name": "name",
"required": true,
"in": "query",
"schema": {
"type": "string"
}
}
],
"responses": {
"200": {
"content": {
"application/json": {
"schema": {
"items": {
"$ref": "#/components/schemas/PersonResponseDto"
},
"type": "array"
}
}
},
"description": ""
}
},
"security": [
{
"bearer": []
},
{
"cookie": []
},
{
"api_key": []
}
],
"tags": [
"Search"
]
}
},
"/server-info": {
"get": {
"operationId": "getServerInfo",
@ -5105,6 +5149,17 @@
"description": ""
}
},
"security": [
{
"bearer": []
},
{
"cookie": []
},
{
"api_key": []
}
],
"tags": [
"User"
]
@ -7501,6 +7556,9 @@
},
"ServerConfigDto": {
"properties": {
"isInitialized": {
"type": "boolean"
},
"loginPageMessage": {
"type": "string"
},
@ -7518,7 +7576,8 @@
"trashDays",
"oauthButtonText",
"loginPageMessage",
"mapTileUrl"
"mapTileUrl",
"isInitialized"
],
"type": "object"
},

7710
server/package-lock.json generated

File diff suppressed because it is too large Load diff

View file

@ -1,6 +1,6 @@
import { BadRequestException, UnauthorizedException } from '@nestjs/common';
import { AuthUserDto } from '../auth';
import { IAccessRepository } from './access.repository';
import { IAccessRepository } from '../repositories';
export enum Permission {
// ASSET_CREATE = 'asset.create',

View file

@ -1,2 +1 @@
export * from './access.core';
export * from './access.repository';

View file

@ -12,10 +12,9 @@ import {
userStub,
} from '@test';
import _ from 'lodash';
import { BulkIdErrorReason, IAssetRepository } from '../asset';
import { IJobRepository, JobName } from '../job';
import { IUserRepository } from '../user';
import { IAlbumRepository } from './album.repository';
import { BulkIdErrorReason } from '../asset';
import { JobName } from '../job';
import { IAlbumRepository, IAssetRepository, IJobRepository, IUserRepository } from '../repositories';
import { AlbumService } from './album.service';
describe(AlbumService.name, () => {

View file

@ -1,10 +1,17 @@
import { AlbumEntity, AssetEntity, UserEntity } from '@app/infra/entities';
import { BadRequestException, Inject, Injectable } from '@nestjs/common';
import { AccessCore, IAccessRepository, Permission } from '../access';
import { BulkIdErrorReason, BulkIdResponseDto, BulkIdsDto, IAssetRepository } from '../asset';
import { AccessCore, Permission } from '../access';
import { BulkIdErrorReason, BulkIdResponseDto, BulkIdsDto } from '../asset';
import { AuthUserDto } from '../auth';
import { IJobRepository, JobName } from '../job';
import { IUserRepository } from '../user';
import { JobName } from '../job';
import {
AlbumInfoOptions,
IAccessRepository,
IAlbumRepository,
IAssetRepository,
IJobRepository,
IUserRepository,
} from '../repositories';
import {
AlbumCountResponseDto,
AlbumResponseDto,
@ -12,7 +19,6 @@ import {
mapAlbumWithAssets,
mapAlbumWithoutAssets,
} from './album-response.dto';
import { AlbumInfoOptions, IAlbumRepository } from './album.repository';
import { AddUsersDto, AlbumInfoDto, CreateAlbumDto, GetAlbumsDto, UpdateAlbumDto } from './dto';
@Injectable()

View file

@ -1,4 +1,3 @@
export * from './album-response.dto';
export * from './album.repository';
export * from './album.service';
export * from './dto';

View file

@ -1,7 +1,6 @@
import { BadRequestException } from '@nestjs/common';
import { authStub, keyStub, newCryptoRepositoryMock, newKeyRepositoryMock } from '@test';
import { ICryptoRepository } from '../crypto';
import { IKeyRepository } from './api-key.repository';
import { ICryptoRepository, IKeyRepository } from '../repositories';
import { APIKeyService } from './api-key.service';
describe(APIKeyService.name, () => {

View file

@ -1,9 +1,8 @@
import { APIKeyEntity } from '@app/infra/entities';
import { BadRequestException, Inject, Injectable } from '@nestjs/common';
import { AuthUserDto } from '../auth';
import { ICryptoRepository } from '../crypto';
import { ICryptoRepository, IKeyRepository } from '../repositories';
import { APIKeyCreateDto, APIKeyCreateResponseDto, APIKeyResponseDto } from './api-key.dto';
import { IKeyRepository } from './api-key.repository';
@Injectable()
export class APIKeyService {

View file

@ -1,3 +1,2 @@
export * from './api-key.dto';
export * from './api-key.repository';
export * from './api-key.service';

View file

@ -10,17 +10,27 @@ import {
newCommunicationRepositoryMock,
newCryptoRepositoryMock,
newJobRepositoryMock,
newMoveRepositoryMock,
newPersonRepositoryMock,
newStorageRepositoryMock,
newSystemConfigRepositoryMock,
} from '@test';
import { when } from 'jest-when';
import { Readable } from 'stream';
import { ICommunicationRepository } from '../communication';
import { ICryptoRepository } from '../crypto';
import { IJobRepository, JobItem, JobName } from '../job';
import { IStorageRepository } from '../storage';
import { ISystemConfigRepository } from '../system-config';
import { AssetStats, IAssetRepository, TimeBucketSize } from './asset.repository';
import { JobName } from '../job';
import {
AssetStats,
IAssetRepository,
ICommunicationRepository,
ICryptoRepository,
IJobRepository,
IMoveRepository,
IPersonRepository,
IStorageRepository,
ISystemConfigRepository,
JobItem,
TimeBucketSize,
} from '../repositories';
import { AssetService, UploadFieldName } from './asset.service';
import { AssetJobName, AssetStatsResponseDto, DownloadResponseDto } from './dto';
import { mapAsset } from './response-dto';
@ -154,6 +164,8 @@ describe(AssetService.name, () => {
let assetMock: jest.Mocked<IAssetRepository>;
let cryptoMock: jest.Mocked<ICryptoRepository>;
let jobMock: jest.Mocked<IJobRepository>;
let moveMock: jest.Mocked<IMoveRepository>;
let personMock: jest.Mocked<IPersonRepository>;
let storageMock: jest.Mocked<IStorageRepository>;
let communicationMock: jest.Mocked<ICommunicationRepository>;
let configMock: jest.Mocked<ISystemConfigRepository>;
@ -168,9 +180,21 @@ describe(AssetService.name, () => {
communicationMock = newCommunicationRepositoryMock();
cryptoMock = newCryptoRepositoryMock();
jobMock = newJobRepositoryMock();
moveMock = newMoveRepositoryMock();
personMock = newPersonRepositoryMock();
storageMock = newStorageRepositoryMock();
configMock = newSystemConfigRepositoryMock();
sut = new AssetService(accessMock, assetMock, cryptoMock, jobMock, configMock, storageMock, communicationMock);
sut = new AssetService(
accessMock,
assetMock,
cryptoMock,
jobMock,
configMock,
moveMock,
personMock,
storageMock,
communicationMock,
);
when(assetMock.getById)
.calledWith(assetStub.livePhotoStillAsset.id)

View file

@ -4,16 +4,26 @@ import _ from 'lodash';
import { DateTime, Duration } from 'luxon';
import { extname } from 'path';
import sanitize from 'sanitize-filename';
import { AccessCore, IAccessRepository, Permission } from '../access';
import { AccessCore, Permission } from '../access';
import { AuthUserDto } from '../auth';
import { CommunicationEvent, ICommunicationRepository } from '../communication';
import { ICryptoRepository } from '../crypto';
import { mimeTypes } from '../domain.constant';
import { HumanReadableSize, usePagination } from '../domain.util';
import { IAssetDeletionJob, IJobRepository, JOBS_ASSET_PAGINATION_SIZE, JobName } from '../job';
import { IStorageRepository, ImmichReadStream, StorageCore, StorageFolder } from '../storage';
import { ISystemConfigRepository, SystemConfigCore } from '../system-config';
import { IAssetRepository } from './asset.repository';
import { IAssetDeletionJob, JOBS_ASSET_PAGINATION_SIZE, JobName } from '../job';
import {
CommunicationEvent,
IAccessRepository,
IAssetRepository,
ICommunicationRepository,
ICryptoRepository,
IJobRepository,
IMoveRepository,
IPersonRepository,
IStorageRepository,
ISystemConfigRepository,
ImmichReadStream,
} from '../repositories';
import { StorageCore, StorageFolder } from '../storage';
import { SystemConfigCore } from '../system-config';
import {
AssetBulkDeleteDto,
AssetBulkUpdateDto,
@ -72,12 +82,14 @@ export class AssetService {
@Inject(ICryptoRepository) private cryptoRepository: ICryptoRepository,
@Inject(IJobRepository) private jobRepository: IJobRepository,
@Inject(ISystemConfigRepository) configRepository: ISystemConfigRepository,
@Inject(IMoveRepository) moveRepository: IMoveRepository,
@Inject(IPersonRepository) personRepository: IPersonRepository,
@Inject(IStorageRepository) private storageRepository: IStorageRepository,
@Inject(ICommunicationRepository) private communicationRepository: ICommunicationRepository,
) {
this.access = new AccessCore(accessRepository);
this.storageCore = new StorageCore(storageRepository);
this.configCore = SystemConfigCore.create(configRepository);
this.storageCore = new StorageCore(storageRepository, assetRepository, moveRepository, personRepository);
}
canUploadFile({ authUser, fieldName, file }: UploadRequest): true {
@ -169,13 +181,15 @@ export class AssetService {
private async timeBucketChecks(authUser: AuthUserDto, dto: TimeBucketDto) {
if (dto.albumId) {
await this.access.requirePermission(authUser, Permission.ALBUM_READ, [dto.albumId]);
} else if (dto.userId) {
} else {
dto.userId = dto.userId || authUser.id;
}
if (dto.userId) {
await this.access.requirePermission(authUser, Permission.TIMELINE_READ, [dto.userId]);
if (dto.isArchived !== false) {
await this.access.requirePermission(authUser, Permission.ARCHIVE_READ, [dto.userId]);
}
await this.access.requirePermission(authUser, Permission.TIMELINE_READ, [dto.userId]);
} else {
dto.userId = authUser.id;
}
}

View file

@ -3,7 +3,7 @@ import { ApiProperty } from '@nestjs/swagger';
import { Transform } from 'class-transformer';
import { IsBoolean } from 'class-validator';
import { Optional, toBoolean } from '../../domain.util';
import { AssetStats } from '../asset.repository';
import { AssetStats } from '../../repositories';
export class AssetStatsDto {
@IsBoolean()

View file

@ -2,7 +2,7 @@ import { ApiProperty } from '@nestjs/swagger';
import { Transform } from 'class-transformer';
import { IsBoolean, IsEnum, IsNotEmpty, IsString } from 'class-validator';
import { Optional, ValidateUUID, toBoolean } from '../../domain.util';
import { TimeBucketSize } from '../asset.repository';
import { TimeBucketSize } from '../../repositories';
export class TimeBucketDto {
@IsNotEmpty()

View file

@ -1,4 +1,3 @@
export * from './asset.repository';
export * from './asset.service';
export * from './dto';
export * from './response-dto';

View file

@ -1,6 +1,6 @@
import { DatabaseAction, EntityType } from '@app/infra/entities';
import { IAccessRepositoryMock, auditStub, authStub, newAccessRepositoryMock, newAuditRepositoryMock } from '@test';
import { IAuditRepository } from './audit.repository';
import { IAuditRepository } from '../repositories';
import { AuditService } from './audit.service';
describe(AuditService.name, () => {

View file

@ -1,11 +1,11 @@
import { DatabaseAction } from '@app/infra/entities';
import { Inject, Injectable } from '@nestjs/common';
import { DateTime } from 'luxon';
import { AccessCore, IAccessRepository, Permission } from '../access';
import { AccessCore, Permission } from '../access';
import { AuthUserDto } from '../auth';
import { AUDIT_LOG_MAX_DURATION } from '../domain.constant';
import { IAccessRepository, IAuditRepository } from '../repositories';
import { AuditDeletesDto, AuditDeletesResponseDto } from './audit.dto';
import { IAuditRepository } from './audit.repository';
@Injectable()
export class AuditService {

View file

@ -1,3 +1,2 @@
export * from './audit.dto';
export * from './audit.repository';
export * from './audit.service';

View file

@ -19,16 +19,18 @@ import {
import { IncomingHttpHeaders } from 'http';
import { Issuer, generators } from 'openid-client';
import { Socket } from 'socket.io';
import { IKeyRepository } from '../api-key';
import { ICryptoRepository } from '../crypto/crypto.repository';
import { ILibraryRepository } from '../library';
import { ISharedLinkRepository } from '../shared-link';
import { ISystemConfigRepository } from '../system-config';
import { IUserRepository } from '../user';
import {
ICryptoRepository,
IKeyRepository,
ILibraryRepository,
ISharedLinkRepository,
ISystemConfigRepository,
IUserRepository,
IUserTokenRepository,
} from '../repositories';
import { AuthType } from './auth.constant';
import { AuthService } from './auth.service';
import { AuthUserDto, SignUpDto } from './dto';
import { IUserTokenRepository } from './user-token.repository';
// const token = Buffer.from('my-api-key', 'utf8').toString('base64');

View file

@ -11,13 +11,17 @@ import cookieParser from 'cookie';
import { IncomingHttpHeaders } from 'http';
import { DateTime } from 'luxon';
import { ClientMetadata, Issuer, UserinfoResponse, custom, generators } from 'openid-client';
import { IKeyRepository } from '../api-key';
import { ICryptoRepository } from '../crypto/crypto.repository';
import { ILibraryRepository } from '../library';
import { ISharedLinkRepository } from '../shared-link';
import { ISystemConfigRepository } from '../system-config';
import {
ICryptoRepository,
IKeyRepository,
ILibraryRepository,
ISharedLinkRepository,
ISystemConfigRepository,
IUserRepository,
IUserTokenRepository,
} from '../repositories';
import { SystemConfigCore } from '../system-config/system-config.core';
import { IUserRepository, UserCore, UserResponseDto } from '../user';
import { UserCore, UserResponseDto } from '../user';
import {
AuthType,
IMMICH_ACCESS_COOKIE,
@ -38,7 +42,6 @@ import {
mapLoginResponse,
mapUserToken,
} from './response-dto';
import { IUserTokenRepository } from './user-token.repository';
export interface LoginDetails {
isSecure: boolean;

View file

@ -2,4 +2,3 @@ export * from './auth.constant';
export * from './auth.service';
export * from './dto';
export * from './response-dto';
export * from './user-token.repository';

View file

@ -1 +0,0 @@
export * from './communication.repository';

View file

@ -1 +0,0 @@
export * from './crypto.repository';

View file

@ -4,8 +4,6 @@ export * from './api-key';
export * from './asset';
export * from './audit';
export * from './auth';
export * from './communication';
export * from './crypto';
export * from './domain.config';
export * from './domain.constant';
export * from './domain.module';
@ -16,6 +14,7 @@ export * from './media';
export * from './metadata';
export * from './partner';
export * from './person';
export * from './repositories';
export * from './search';
export * from './server-info';
export * from './shared-link';

View file

@ -1,5 +1,4 @@
export * from './job.constants';
export * from './job.dto';
export * from './job.interface';
export * from './job.repository';
export * from './job.service';

View file

@ -69,7 +69,6 @@ export enum JobName {
LIBRARY_SCAN = 'library-refresh',
LIBRARY_SCAN_ASSET = 'library-refresh-asset',
LIBRARY_REMOVE_OFFLINE = 'library-remove-offline',
LIBRARY_MARK_ASSET_OFFLINE = 'library-mark-asset-offline',
LIBRARY_DELETE = 'library-delete',
LIBRARY_QUEUE_SCAN_ALL = 'library-queue-all-refresh',
LIBRARY_QUEUE_CLEANUP = 'library-queue-cleanup',
@ -172,7 +171,6 @@ export const JOBS_TO_QUEUE: Record<JobName, QueueName> = {
// Library managment
[JobName.LIBRARY_SCAN_ASSET]: QueueName.LIBRARY,
[JobName.LIBRARY_MARK_ASSET_OFFLINE]: QueueName.LIBRARY,
[JobName.LIBRARY_SCAN]: QueueName.LIBRARY,
[JobName.LIBRARY_DELETE]: QueueName.LIBRARY,
[JobName.LIBRARY_REMOVE_OFFLINE]: QueueName.LIBRARY,

View file

@ -16,10 +16,6 @@ export interface IAssetDeletionJob extends IEntityJob {
fromExternal?: boolean;
}
export interface IOfflineLibraryFileJob extends IEntityJob {
assetPath: string;
}
export interface ILibraryFileJob extends IEntityJob {
ownerId: string;
assetPath: string;

View file

@ -9,13 +9,17 @@ import {
newPersonRepositoryMock,
newSystemConfigRepositoryMock,
} from '@test';
import { IAssetRepository } from '../asset';
import { ICommunicationRepository } from '../communication';
import { IPersonRepository } from '../person';
import { ISystemConfigRepository } from '../system-config';
import {
IAssetRepository,
ICommunicationRepository,
IJobRepository,
IPersonRepository,
ISystemConfigRepository,
JobHandler,
JobItem,
} from '../repositories';
import { SystemConfigCore } from '../system-config/system-config.core';
import { JobCommand, JobName, QueueName } from './job.constants';
import { IJobRepository, JobHandler, JobItem } from './job.repository';
import { JobService } from './job.service';
const makeMockHandlers = (success: boolean) => {

View file

@ -1,13 +1,19 @@
import { AssetType } from '@app/infra/entities';
import { BadRequestException, Inject, Injectable, Logger } from '@nestjs/common';
import { IAssetRepository, mapAsset } from '../asset';
import { CommunicationEvent, ICommunicationRepository } from '../communication';
import { IPersonRepository } from '../person';
import { FeatureFlag, ISystemConfigRepository } from '../system-config';
import { SystemConfigCore } from '../system-config/system-config.core';
import { mapAsset } from '../asset';
import {
CommunicationEvent,
IAssetRepository,
ICommunicationRepository,
IJobRepository,
IPersonRepository,
ISystemConfigRepository,
JobHandler,
JobItem,
} from '../repositories';
import { FeatureFlag, SystemConfigCore } from '../system-config/system-config.core';
import { JobCommand, JobName, QueueName } from './job.constants';
import { AllJobStatusResponseDto, JobCommandDto, JobStatusDto } from './job.dto';
import { IJobRepository, JobHandler, JobItem } from './job.repository';
@Injectable()
export class JobService {

View file

@ -1,3 +1,2 @@
export * from './library.dto';
export * from './library.repository';
export * from './library.service';

View file

@ -16,10 +16,15 @@ import {
userStub,
} from '@test';
import { Stats } from 'fs';
import { IJobRepository, ILibraryFileJob, ILibraryRefreshJob, IOfflineLibraryFileJob, JobName } from '../job';
import { IAssetRepository, ICryptoRepository, IStorageRepository, IUserRepository } from '..';
import { ILibraryRepository } from './library.repository';
import { ILibraryFileJob, ILibraryRefreshJob, JobName } from '../job';
import {
IAssetRepository,
ICryptoRepository,
IJobRepository,
ILibraryRepository,
IStorageRepository,
IUserRepository,
} from '../repositories';
import { LibraryService } from './library.service';
describe(LibraryService.name, () => {
@ -121,14 +126,11 @@ describe(LibraryService.name, () => {
await sut.handleQueueAssetRefresh(mockLibraryJob);
expect(jobMock.queue.mock.calls).toEqual([
expect(assetMock.updateAll.mock.calls).toEqual([
[
[assetStub.external.id],
{
name: JobName.LIBRARY_MARK_ASSET_OFFLINE,
data: {
id: libraryStub.externalLibrary1.id,
assetPath: '/data/user1/photo.jpg',
},
isOffline: true,
},
],
]);
@ -145,7 +147,7 @@ describe(LibraryService.name, () => {
userMock.get.mockResolvedValue(userStub.user1);
expect(sut.handleQueueAssetRefresh(mockLibraryJob)).resolves.toBe(false);
await expect(sut.handleQueueAssetRefresh(mockLibraryJob)).resolves.toBe(false);
});
it('should not scan upload libraries', async () => {
@ -157,7 +159,7 @@ describe(LibraryService.name, () => {
libraryMock.get.mockResolvedValue(libraryStub.uploadLibrary1);
expect(sut.handleQueueAssetRefresh(mockLibraryJob)).resolves.toBe(false);
await expect(sut.handleQueueAssetRefresh(mockLibraryJob)).resolves.toBe(false);
});
});
@ -410,61 +412,6 @@ describe(LibraryService.name, () => {
});
});
it('should skip an asset if the user cannot be found', async () => {
userMock.get.mockResolvedValue(null);
const mockLibraryJob: ILibraryFileJob = {
id: libraryStub.externalLibrary1.id,
ownerId: mockUser.id,
assetPath: '/data/user1/photo.jpg',
force: false,
};
expect(sut.handleAssetRefresh(mockLibraryJob)).resolves.toBe(false);
});
it('should skip an asset if external path is not set', async () => {
mockUser = userStub.admin;
userMock.get.mockResolvedValue(mockUser);
const mockLibraryJob: ILibraryFileJob = {
id: libraryStub.externalLibrary1.id,
ownerId: mockUser.id,
assetPath: '/data/user1/photo.jpg',
force: false,
};
expect(sut.handleAssetRefresh(mockLibraryJob)).resolves.toBe(false);
});
it("should skip an asset if it isn't in the external path", async () => {
mockUser = userStub.externalPath1;
userMock.get.mockResolvedValue(mockUser);
const mockLibraryJob: ILibraryFileJob = {
id: libraryStub.externalLibrary1.id,
ownerId: mockUser.id,
assetPath: '/etc/rootpassword.jpg',
force: false,
};
expect(sut.handleAssetRefresh(mockLibraryJob)).resolves.toBe(false);
});
it('should skip an asset if directory traversal is attempted', async () => {
mockUser = userStub.externalPath1;
userMock.get.mockResolvedValue(mockUser);
const mockLibraryJob: ILibraryFileJob = {
id: libraryStub.externalLibrary1.id,
ownerId: mockUser.id,
assetPath: '/data/user1/../../etc/rootpassword.jpg',
force: false,
};
expect(sut.handleAssetRefresh(mockLibraryJob)).resolves.toBe(false);
});
it('should set a missing asset to offline', async () => {
storageMock.stat.mockRejectedValue(new Error());
@ -595,24 +542,6 @@ describe(LibraryService.name, () => {
});
});
describe('handleOfflineAsset', () => {
it('should mark an asset as offline', async () => {
const offlineJob: IOfflineLibraryFileJob = {
id: libraryStub.externalLibrary1.id,
assetPath: '/data/user1/photo.jpg',
};
assetMock.getByLibraryIdAndOriginalPath.mockResolvedValue(assetStub.image);
await expect(sut.handleOfflineAsset(offlineJob)).resolves.toBe(true);
expect(assetMock.save).toHaveBeenCalledWith({
id: assetStub.image.id,
isOffline: true,
});
});
});
describe('delete', () => {
it('should delete a library', async () => {
assetMock.getByLibraryIdAndOriginalPath.mockResolvedValue(assetStub.image);

View file

@ -4,24 +4,22 @@ import { R_OK } from 'node:constants';
import { Stats } from 'node:fs';
import path from 'node:path';
import { basename, parse } from 'path';
import { AccessCore, IAccessRepository, Permission } from '../access';
import { IAssetRepository, WithProperty } from '../asset';
import { AccessCore, Permission } from '../access';
import { AuthUserDto } from '../auth';
import { ICryptoRepository } from '../crypto';
import { mimeTypes } from '../domain.constant';
import { usePagination } from '../domain.util';
import { IBaseJob, IEntityJob, ILibraryFileJob, ILibraryRefreshJob, JOBS_ASSET_PAGINATION_SIZE, JobName } from '../job';
import {
IBaseJob,
IEntityJob,
IAccessRepository,
IAssetRepository,
ICryptoRepository,
IJobRepository,
ILibraryFileJob,
ILibraryRefreshJob,
IOfflineLibraryFileJob,
JOBS_ASSET_PAGINATION_SIZE,
JobName,
} from '../job';
import { IStorageRepository } from '../storage';
import { IUserRepository } from '../user';
ILibraryRepository,
IStorageRepository,
IUserRepository,
WithProperty,
} from '../repositories';
import {
CreateLibraryDto,
LibraryResponseDto,
@ -30,7 +28,6 @@ import {
UpdateLibraryDto,
mapLibrary,
} from './library.dto';
import { ILibraryRepository } from './library.repository';
@Injectable()
export class LibraryService {
@ -152,17 +149,6 @@ export class LibraryService {
async handleAssetRefresh(job: ILibraryFileJob) {
const assetPath = path.normalize(job.assetPath);
const user = await this.userRepository.get(job.ownerId);
if (!user?.externalPath) {
this.logger.warn('User has no external path set, cannot import asset');
return false;
}
if (!path.normalize(assetPath).match(new RegExp(`^${path.normalize(user.externalPath)}`))) {
this.logger.error("Asset must be within the user's external path");
return false;
}
const existingAssetEntity = await this.assetRepository.getByLibraryIdAndOriginalPath(job.id, assetPath);
let stats: Stats;
@ -363,8 +349,6 @@ export class LibraryService {
return false;
}
const normalizedExternalPath = path.normalize(user.externalPath);
this.logger.verbose(`Refreshing library: ${job.id}`);
const crawledAssetPaths = (
await this.storageRepository.crawl({
@ -375,33 +359,31 @@ export class LibraryService {
.map(path.normalize)
.filter((assetPath) =>
// Filter out paths that are not within the user's external path
assetPath.match(new RegExp(`^${normalizedExternalPath}`)),
assetPath.match(new RegExp(`^${user.externalPath}`)),
);
this.logger.debug(`Found ${crawledAssetPaths.length} assets when crawling import paths ${library.importPaths}`);
const assetsInLibrary = await this.assetRepository.getByLibraryId([job.id]);
const offlineAssets = assetsInLibrary.filter((asset) => !crawledAssetPaths.includes(asset.originalPath));
this.logger.debug(`${offlineAssets.length} assets in library are not present on disk and will be marked offline`);
const onlineFiles = new Set(crawledAssetPaths);
const offlineAssetIds = assetsInLibrary
.filter((asset) => !onlineFiles.has(asset.originalPath))
.filter((asset) => !asset.isOffline)
.map((asset) => asset.id);
this.logger.debug(`Marking ${offlineAssetIds.length} assets as offline`);
for (const offlineAsset of offlineAssets) {
const offlineJobData: IOfflineLibraryFileJob = {
id: job.id,
assetPath: offlineAsset.originalPath,
};
await this.jobRepository.queue({ name: JobName.LIBRARY_MARK_ASSET_OFFLINE, data: offlineJobData });
}
await this.assetRepository.updateAll(offlineAssetIds, { isOffline: true });
if (crawledAssetPaths.length > 0) {
let filteredPaths: string[] = [];
if (job.refreshAllFiles || job.refreshModifiedFiles) {
filteredPaths = crawledAssetPaths;
} else {
const existingPaths = await this.repository.getOnlineAssetPaths(job.id);
this.logger.debug(`Found ${existingPaths.length} existing asset(s) in library ${job.id}`);
const onlinePathsInLibrary = new Set(
assetsInLibrary.filter((asset) => !asset.isOffline).map((asset) => asset.originalPath),
);
filteredPaths = crawledAssetPaths.filter((assetPath) => !onlinePathsInLibrary.has(assetPath));
filteredPaths = crawledAssetPaths.filter((assetPath) => !existingPaths.includes(assetPath));
this.logger.debug(`After db comparison, ${filteredPaths.length} asset(s) remain to be imported`);
this.logger.debug(`Will import ${filteredPaths.length} new asset(s)`);
}
for (const assetPath of filteredPaths) {
@ -421,17 +403,6 @@ export class LibraryService {
return true;
}
async handleOfflineAsset(job: IOfflineLibraryFileJob): Promise<boolean> {
const existingAssetEntity = await this.assetRepository.getByLibraryIdAndOriginalPath(job.id, job.assetPath);
if (existingAssetEntity) {
this.logger.verbose(`Marking asset as offline: ${job.assetPath}`);
await this.assetRepository.save({ id: existingAssetEntity.id, isOffline: true });
}
return true;
}
private async findOrFail(id: string) {
const library = await this.repository.get(id);
if (!library) {

View file

@ -1,3 +1,2 @@
export * from './media.constant';
export * from './media.repository';
export * from './media.service';

View file

@ -14,18 +14,24 @@ import {
newAssetRepositoryMock,
newJobRepositoryMock,
newMediaRepositoryMock,
newMoveRepositoryMock,
newPersonRepositoryMock,
newStorageRepositoryMock,
newSystemConfigRepositoryMock,
personStub,
probeStub,
} from '@test';
import { IAssetRepository, WithoutProperty } from '../asset';
import { IJobRepository, JobName } from '../job';
import { IPersonRepository } from '../person';
import { IStorageRepository } from '../storage';
import { ISystemConfigRepository } from '../system-config';
import { IMediaRepository } from './media.repository';
import { JobName } from '../job';
import {
IAssetRepository,
IJobRepository,
IMediaRepository,
IMoveRepository,
IPersonRepository,
IStorageRepository,
ISystemConfigRepository,
WithoutProperty,
} from '../repositories';
import { MediaService } from './media.service';
describe(MediaService.name, () => {
@ -34,6 +40,7 @@ describe(MediaService.name, () => {
let configMock: jest.Mocked<ISystemConfigRepository>;
let jobMock: jest.Mocked<IJobRepository>;
let mediaMock: jest.Mocked<IMediaRepository>;
let moveMock: jest.Mocked<IMoveRepository>;
let personMock: jest.Mocked<IPersonRepository>;
let storageMock: jest.Mocked<IStorageRepository>;
@ -42,10 +49,11 @@ describe(MediaService.name, () => {
configMock = newSystemConfigRepositoryMock();
jobMock = newJobRepositoryMock();
mediaMock = newMediaRepositoryMock();
moveMock = newMoveRepositoryMock();
personMock = newPersonRepositoryMock();
storageMock = newStorageRepositoryMock();
sut = new MediaService(assetMock, personMock, jobMock, mediaMock, storageMock, configMock);
sut = new MediaService(assetMock, personMock, jobMock, mediaMock, storageMock, configMock, moveMock);
});
it('should be defined', () => {

View file

@ -1,13 +1,31 @@
import { AssetEntity, AssetType, Colorspace, TranscodeHWAccel, TranscodePolicy, VideoCodec } from '@app/infra/entities';
import {
AssetEntity,
AssetPathType,
AssetType,
Colorspace,
TranscodeHWAccel,
TranscodePolicy,
VideoCodec,
} from '@app/infra/entities';
import { Inject, Injectable, Logger, UnsupportedMediaTypeException } from '@nestjs/common';
import { IAssetRepository, WithoutProperty } from '../asset';
import { usePagination } from '../domain.util';
import { IBaseJob, IEntityJob, IJobRepository, JOBS_ASSET_PAGINATION_SIZE, JobName, QueueName } from '../job';
import { IPersonRepository } from '../person';
import { IStorageRepository, StorageCore, StorageFolder } from '../storage';
import { ISystemConfigRepository, SystemConfigFFmpegDto } from '../system-config';
import { IBaseJob, IEntityJob, JOBS_ASSET_PAGINATION_SIZE, JobName, QueueName } from '../job';
import {
AudioStreamInfo,
IAssetRepository,
IJobRepository,
IMediaRepository,
IMoveRepository,
IPersonRepository,
IStorageRepository,
ISystemConfigRepository,
VideoCodecHWConfig,
VideoStreamInfo,
WithoutProperty,
} from '../repositories';
import { StorageCore, StorageFolder } from '../storage';
import { SystemConfigFFmpegDto } from '../system-config';
import { SystemConfigCore } from '../system-config/system-config.core';
import { AudioStreamInfo, IMediaRepository, VideoCodecHWConfig, VideoStreamInfo } from './media.repository';
import { H264Config, HEVCConfig, NVENCConfig, QSVConfig, ThumbnailConfig, VAAPIConfig, VP9Config } from './media.util';
@Injectable()
@ -23,9 +41,10 @@ export class MediaService {
@Inject(IMediaRepository) private mediaRepository: IMediaRepository,
@Inject(IStorageRepository) private storageRepository: IStorageRepository,
@Inject(ISystemConfigRepository) configRepository: ISystemConfigRepository,
@Inject(IMoveRepository) moveRepository: IMoveRepository,
) {
this.configCore = SystemConfigCore.create(configRepository);
this.storageCore = new StorageCore(this.storageRepository);
this.storageCore = new StorageCore(this.storageRepository, assetRepository, moveRepository, personRepository);
}
async handleQueueGenerateThumbnails({ force }: IBaseJob) {
@ -99,29 +118,9 @@ export class MediaService {
return false;
}
if (asset.resizePath) {
const resizePath = this.ensureThumbnailPath(asset, 'jpeg');
if (asset.resizePath !== resizePath) {
await this.storageRepository.moveFile(asset.resizePath, resizePath);
await this.assetRepository.save({ id: asset.id, resizePath });
}
}
if (asset.webpPath) {
const webpPath = this.ensureThumbnailPath(asset, 'webp');
if (asset.webpPath !== webpPath) {
await this.storageRepository.moveFile(asset.webpPath, webpPath);
await this.assetRepository.save({ id: asset.id, webpPath });
}
}
if (asset.encodedVideoPath) {
const encodedVideoPath = this.ensureEncodedVideoPath(asset, 'mp4');
if (asset.encodedVideoPath !== encodedVideoPath) {
await this.storageRepository.moveFile(asset.encodedVideoPath, encodedVideoPath);
await this.assetRepository.save({ id: asset.id, encodedVideoPath });
}
}
await this.storageCore.moveAssetFile(asset, AssetPathType.JPEG_THUMBNAIL);
await this.storageCore.moveAssetFile(asset, AssetPathType.WEBP_THUMBNAIL);
await this.storageCore.moveAssetFile(asset, AssetPathType.ENCODED_VIDEO);
return true;
}
@ -137,15 +136,33 @@ export class MediaService {
return true;
}
async generateThumbnail(asset: AssetEntity, format: 'jpeg' | 'webp') {
let path;
private async generateThumbnail(asset: AssetEntity, format: 'jpeg' | 'webp') {
const { thumbnail, ffmpeg } = await this.configCore.getConfig();
const size = format === 'jpeg' ? thumbnail.jpegSize : thumbnail.webpSize;
const path =
format === 'jpeg' ? this.storageCore.getLargeThumbnailPath(asset) : this.storageCore.getSmallThumbnailPath(asset);
this.storageCore.ensureFolders(path);
switch (asset.type) {
case AssetType.IMAGE:
path = await this.generateImageThumbnail(asset, format);
const colorspace = this.isSRGB(asset) ? Colorspace.SRGB : thumbnail.colorspace;
const thumbnailOptions = { format, size, colorspace, quality: thumbnail.quality };
await this.mediaRepository.resize(asset.originalPath, path, thumbnailOptions);
break;
case AssetType.VIDEO:
path = await this.generateVideoThumbnail(asset, format);
const { audioStreams, videoStreams } = await this.mediaRepository.probe(asset.originalPath);
const mainVideoStream = this.getMainStream(videoStreams);
if (!mainVideoStream) {
this.logger.warn(`Skipped thumbnail generation for asset ${asset.id}: no video streams found`);
return;
}
const mainAudioStream = this.getMainStream(audioStreams);
const config = { ...ffmpeg, targetResolution: size.toString() };
const options = new ThumbnailConfig(config).getOptions(mainVideoStream, mainAudioStream);
await this.mediaRepository.transcode(asset.originalPath, path, options);
break;
default:
throw new UnsupportedMediaTypeException(`Unsupported asset type for thumbnail generation: ${asset.type}`);
}
@ -155,33 +172,6 @@ export class MediaService {
return path;
}
async generateImageThumbnail(asset: AssetEntity, format: 'jpeg' | 'webp') {
const { thumbnail } = await this.configCore.getConfig();
const size = format === 'jpeg' ? thumbnail.jpegSize : thumbnail.webpSize;
const path = this.ensureThumbnailPath(asset, format);
const colorspace = this.isSRGB(asset) ? Colorspace.SRGB : thumbnail.colorspace;
const thumbnailOptions = { format, size, colorspace, quality: thumbnail.quality };
await this.mediaRepository.resize(asset.originalPath, path, thumbnailOptions);
return path;
}
async generateVideoThumbnail(asset: AssetEntity, format: 'jpeg' | 'webp') {
const { ffmpeg, thumbnail } = await this.configCore.getConfig();
const size = format === 'jpeg' ? thumbnail.jpegSize : thumbnail.webpSize;
const { audioStreams, videoStreams } = await this.mediaRepository.probe(asset.originalPath);
const mainVideoStream = this.getMainStream(videoStreams);
if (!mainVideoStream) {
this.logger.warn(`Skipped thumbnail generation for asset ${asset.id}: no video streams found`);
return;
}
const mainAudioStream = this.getMainStream(audioStreams);
const path = this.ensureThumbnailPath(asset, format);
const config = { ...ffmpeg, targetResolution: size.toString() };
const options = new ThumbnailConfig(config).getOptions(mainVideoStream, mainAudioStream);
await this.mediaRepository.transcode(asset.originalPath, path, options);
return path;
}
async handleGenerateWebpThumbnail({ id }: IEntityJob) {
const [asset] = await this.assetRepository.getByIds([id]);
if (!asset) {
@ -230,7 +220,8 @@ export class MediaService {
}
const input = asset.originalPath;
const output = this.ensureEncodedVideoPath(asset, 'mp4');
const output = this.storageCore.getEncodedVideoPath(asset);
this.storageCore.ensureFolders(output);
const { videoStreams, audioStreams, format } = await this.mediaRepository.probe(input);
const mainVideoStream = this.getMainStream(videoStreams);
@ -373,14 +364,6 @@ export class MediaService {
return handler;
}
ensureThumbnailPath(asset: AssetEntity, extension: string): string {
return this.storageCore.ensurePath(StorageFolder.THUMBNAILS, asset.ownerId, `${asset.id}.${extension}`);
}
ensureEncodedVideoPath(asset: AssetEntity, extension: string): string {
return this.storageCore.ensurePath(StorageFolder.ENCODED_VIDEO, asset.ownerId, `${asset.id}.${extension}`);
}
isSRGB(asset: AssetEntity): boolean {
const { colorspace, profileDescription, bitsPerSample } = asset.exifInfo ?? {};
if (colorspace || profileDescription) {

View file

@ -1,5 +1,4 @@
import { CQMode, ToneMapping, TranscodeHWAccel, VideoCodec } from '@app/infra/entities';
import { SystemConfigFFmpegDto } from '../system-config/dto';
import {
AudioStreamInfo,
BitrateDistribution,
@ -7,7 +6,8 @@ import {
VideoCodecHWConfig,
VideoCodecSWConfig,
VideoStreamInfo,
} from './media.repository';
} from '../repositories';
import { SystemConfigFFmpegDto } from '../system-config/dto';
class BaseConfig implements VideoCodecSWConfig {
presets = ['veryslow', 'slower', 'slow', 'medium', 'fast', 'faster', 'veryfast', 'superfast', 'ultrafast'];
constructor(protected config: SystemConfigFFmpegDto) {}

View file

@ -1,2 +1 @@
export * from './metadata.repository';
export * from './metadata.service';

View file

@ -6,19 +6,29 @@ import {
newCryptoRepositoryMock,
newJobRepositoryMock,
newMetadataRepositoryMock,
newMoveRepositoryMock,
newPersonRepositoryMock,
newStorageRepositoryMock,
newSystemConfigRepositoryMock,
} from '@test';
import { randomBytes } from 'crypto';
import { Stats } from 'fs';
import { constants } from 'fs/promises';
import { IAlbumRepository } from '../album';
import { IAssetRepository, WithProperty, WithoutProperty } from '../asset';
import { ICryptoRepository } from '../crypto';
import { IJobRepository, JobName, QueueName } from '../job';
import { IStorageRepository } from '../storage';
import { ISystemConfigRepository } from '../system-config';
import { IMetadataRepository, ImmichTags } from './metadata.repository';
import { JobName, QueueName } from '../job';
import {
IAlbumRepository,
IAssetRepository,
ICryptoRepository,
IJobRepository,
IMetadataRepository,
IMoveRepository,
IPersonRepository,
IStorageRepository,
ISystemConfigRepository,
ImmichTags,
WithProperty,
WithoutProperty,
} from '../repositories';
import { MetadataService } from './metadata.service';
describe(MetadataService.name, () => {
@ -28,6 +38,8 @@ describe(MetadataService.name, () => {
let cryptoRepository: jest.Mocked<ICryptoRepository>;
let jobMock: jest.Mocked<IJobRepository>;
let metadataMock: jest.Mocked<IMetadataRepository>;
let moveMock: jest.Mocked<IMoveRepository>;
let personMock: jest.Mocked<IPersonRepository>;
let storageMock: jest.Mocked<IStorageRepository>;
let sut: MetadataService;
@ -38,9 +50,21 @@ describe(MetadataService.name, () => {
cryptoRepository = newCryptoRepositoryMock();
jobMock = newJobRepositoryMock();
metadataMock = newMetadataRepositoryMock();
moveMock = newMoveRepositoryMock();
personMock = newPersonRepositoryMock();
storageMock = newStorageRepositoryMock();
sut = new MetadataService(albumMock, assetMock, cryptoRepository, jobMock, metadataMock, storageMock, configMock);
sut = new MetadataService(
albumMock,
assetMock,
cryptoRepository,
jobMock,
metadataMock,
storageMock,
configMock,
moveMock,
personMock,
);
});
it('should be defined', () => {

View file

@ -4,14 +4,24 @@ import { ExifDateTime, Tags } from 'exiftool-vendored';
import { firstDateTime } from 'exiftool-vendored/dist/FirstDateTime';
import { constants } from 'fs/promises';
import { Duration } from 'luxon';
import { IAlbumRepository } from '../album';
import { IAssetRepository, WithProperty, WithoutProperty } from '../asset';
import { ICryptoRepository } from '../crypto';
import { usePagination } from '../domain.util';
import { IBaseJob, IEntityJob, IJobRepository, JOBS_ASSET_PAGINATION_SIZE, JobName, QueueName } from '../job';
import { IStorageRepository, StorageCore, StorageFolder } from '../storage';
import { FeatureFlag, ISystemConfigRepository, SystemConfigCore } from '../system-config';
import { IMetadataRepository, ImmichTags } from './metadata.repository';
import { IBaseJob, IEntityJob, JOBS_ASSET_PAGINATION_SIZE, JobName, QueueName } from '../job';
import {
IAlbumRepository,
IAssetRepository,
ICryptoRepository,
IJobRepository,
IMetadataRepository,
IMoveRepository,
IPersonRepository,
IStorageRepository,
ISystemConfigRepository,
ImmichTags,
WithProperty,
WithoutProperty,
} from '../repositories';
import { StorageCore } from '../storage';
import { FeatureFlag, SystemConfigCore } from '../system-config';
interface DirectoryItem {
Length?: number;
@ -65,9 +75,11 @@ export class MetadataService {
@Inject(IMetadataRepository) private repository: IMetadataRepository,
@Inject(IStorageRepository) private storageRepository: IStorageRepository,
@Inject(ISystemConfigRepository) configRepository: ISystemConfigRepository,
@Inject(IMoveRepository) moveRepository: IMoveRepository,
@Inject(IPersonRepository) personRepository: IPersonRepository,
) {
this.storageCore = new StorageCore(storageRepository);
this.configCore = SystemConfigCore.create(configRepository);
this.storageCore = new StorageCore(storageRepository, assetRepository, moveRepository, personRepository);
this.configCore.config$.subscribe(() => this.init());
}
@ -288,7 +300,7 @@ export class MetadataService {
localDateTime: createdAt,
checksum,
ownerId: asset.ownerId,
originalPath: this.storageCore.ensurePath(StorageFolder.ENCODED_VIDEO, asset.ownerId, `${asset.id}-MP.mp4`),
originalPath: this.storageCore.getAndroidMotionPath(asset),
originalFileName: asset.originalFileName,
isVisible: false,
isReadOnly: false,

View file

@ -1,2 +1 @@
export * from './partner.repository';
export * from './partner.service';

View file

@ -1,7 +1,7 @@
import { BadRequestException } from '@nestjs/common';
import { authStub, newPartnerRepositoryMock, partnerStub } from '@test';
import { UserResponseDto } from '../index';
import { IPartnerRepository, PartnerDirection } from './partner.repository';
import { IPartnerRepository, PartnerDirection } from '../repositories';
import { PartnerService } from './partner.service';
const responseDto = {

View file

@ -1,7 +1,7 @@
import { PartnerEntity } from '@app/infra/entities';
import { BadRequestException, Inject, Injectable } from '@nestjs/common';
import { IPartnerRepository, PartnerDirection, PartnerIds } from '.';
import { AuthUserDto } from '../auth';
import { IPartnerRepository, PartnerDirection, PartnerIds } from '../repositories';
import { UserResponseDto, mapUser } from '../user';
@Injectable()

View file

@ -1,3 +1,2 @@
export * from './person.dto';
export * from './person.repository';
export * from './person.service';

View file

@ -10,21 +10,28 @@ import {
newJobRepositoryMock,
newMachineLearningRepositoryMock,
newMediaRepositoryMock,
newMoveRepositoryMock,
newPersonRepositoryMock,
newSearchRepositoryMock,
newStorageRepositoryMock,
newSystemConfigRepositoryMock,
personStub,
} from '@test';
import { BulkIdErrorReason, IAssetRepository, WithoutProperty } from '../asset';
import { IJobRepository, JobName } from '../job';
import { IMediaRepository } from '../media';
import { ISearchRepository } from '../search';
import { IMachineLearningRepository } from '../smart-info';
import { IStorageRepository } from '../storage';
import { ISystemConfigRepository } from '../system-config';
import { BulkIdErrorReason } from '../asset';
import { JobName } from '../job';
import {
IAssetRepository,
IJobRepository,
IMachineLearningRepository,
IMediaRepository,
IMoveRepository,
IPersonRepository,
ISearchRepository,
IStorageRepository,
ISystemConfigRepository,
WithoutProperty,
} from '../repositories';
import { PersonResponseDto } from './person.dto';
import { IPersonRepository } from './person.repository';
import { PersonService } from './person.service';
const responseDto: PersonResponseDto = {
@ -86,6 +93,7 @@ describe(PersonService.name, () => {
let jobMock: jest.Mocked<IJobRepository>;
let machineLearningMock: jest.Mocked<IMachineLearningRepository>;
let mediaMock: jest.Mocked<IMediaRepository>;
let moveMock: jest.Mocked<IMoveRepository>;
let personMock: jest.Mocked<IPersonRepository>;
let searchMock: jest.Mocked<ISearchRepository>;
let storageMock: jest.Mocked<IStorageRepository>;
@ -97,6 +105,7 @@ describe(PersonService.name, () => {
configMock = newSystemConfigRepositoryMock();
jobMock = newJobRepositoryMock();
machineLearningMock = newMachineLearningRepositoryMock();
moveMock = newMoveRepositoryMock();
mediaMock = newMediaRepositoryMock();
personMock = newPersonRepositoryMock();
searchMock = newSearchRepositoryMock();
@ -105,6 +114,7 @@ describe(PersonService.name, () => {
accessMock,
assetMock,
machineLearningMock,
moveMock,
mediaMock,
personMock,
searchMock,
@ -542,19 +552,19 @@ describe(PersonService.name, () => {
it('should generate a thumbnail', async () => {
personMock.getById.mockResolvedValue({ ...personStub.primaryPerson, faceAssetId: faceStub.middle.assetId });
personMock.getFacesByIds.mockResolvedValue([faceStub.middle]);
assetMock.getByIds.mockResolvedValue([assetStub.image]);
assetMock.getByIds.mockResolvedValue([assetStub.primaryImage]);
await sut.handleGeneratePersonThumbnail({ id: 'person-1' });
expect(assetMock.getByIds).toHaveBeenCalledWith([faceStub.middle.assetId]);
expect(storageMock.mkdirSync).toHaveBeenCalledWith('upload/thumbs/user-id/pe/rs');
expect(mediaMock.crop).toHaveBeenCalledWith('/uploads/user-id/thumbs/path.jpg', {
expect(storageMock.mkdirSync).toHaveBeenCalledWith('upload/thumbs/admin_id/pe/rs');
expect(mediaMock.crop).toHaveBeenCalledWith('/uploads/admin-id/thumbs/path.jpg', {
left: 95,
top: 95,
width: 110,
height: 110,
});
expect(mediaMock.resize).toHaveBeenCalledWith(croppedFace, 'upload/thumbs/user-id/pe/rs/person-1.jpeg', {
expect(mediaMock.resize).toHaveBeenCalledWith(croppedFace, 'upload/thumbs/admin_id/pe/rs/person-1.jpeg', {
format: 'jpeg',
size: 250,
quality: 80,
@ -562,7 +572,7 @@ describe(PersonService.name, () => {
});
expect(personMock.update).toHaveBeenCalledWith({
id: 'person-1',
thumbnailPath: 'upload/thumbs/user-id/pe/rs/person-1.jpeg',
thumbnailPath: 'upload/thumbs/admin_id/pe/rs/person-1.jpeg',
});
});
@ -579,7 +589,7 @@ describe(PersonService.name, () => {
width: 510,
height: 510,
});
expect(mediaMock.resize).toHaveBeenCalledWith(croppedFace, 'upload/thumbs/user-id/pe/rs/person-1.jpeg', {
expect(mediaMock.resize).toHaveBeenCalledWith(croppedFace, 'upload/thumbs/admin_id/pe/rs/person-1.jpeg', {
format: 'jpeg',
size: 250,
quality: 80,
@ -590,17 +600,17 @@ describe(PersonService.name, () => {
it('should generate a thumbnail without overflowing', async () => {
personMock.getById.mockResolvedValue({ ...personStub.primaryPerson, faceAssetId: faceStub.end.assetId });
personMock.getFacesByIds.mockResolvedValue([faceStub.end]);
assetMock.getByIds.mockResolvedValue([assetStub.image]);
assetMock.getByIds.mockResolvedValue([assetStub.primaryImage]);
await sut.handleGeneratePersonThumbnail({ id: 'person-1' });
expect(mediaMock.crop).toHaveBeenCalledWith('/uploads/user-id/thumbs/path.jpg', {
expect(mediaMock.crop).toHaveBeenCalledWith('/uploads/admin-id/thumbs/path.jpg', {
left: 297,
top: 297,
width: 202,
height: 202,
});
expect(mediaMock.resize).toHaveBeenCalledWith(croppedFace, 'upload/thumbs/user-id/pe/rs/person-1.jpeg', {
expect(mediaMock.resize).toHaveBeenCalledWith(croppedFace, 'upload/thumbs/admin_id/pe/rs/person-1.jpeg', {
format: 'jpeg',
size: 250,
quality: 80,

View file

@ -1,23 +1,32 @@
import { PersonEntity } from '@app/infra/entities';
import { PersonPathType } from '@app/infra/entities/move.entity';
import { BadRequestException, Inject, Injectable, Logger, NotFoundException } from '@nestjs/common';
import { AccessCore, IAccessRepository, Permission } from '../access';
import {
AssetResponseDto,
BulkIdErrorReason,
BulkIdResponseDto,
IAssetRepository,
WithoutProperty,
mapAsset,
} from '../asset';
import { AccessCore, Permission } from '../access';
import { AssetResponseDto, BulkIdErrorReason, BulkIdResponseDto, mapAsset } from '../asset';
import { AuthUserDto } from '../auth';
import { mimeTypes } from '../domain.constant';
import { usePagination } from '../domain.util';
import { IBaseJob, IEntityJob, IJobRepository, JOBS_ASSET_PAGINATION_SIZE, JobName } from '../job';
import { CropOptions, FACE_THUMBNAIL_SIZE, IMediaRepository } from '../media';
import { ISearchRepository } from '../search';
import { IMachineLearningRepository } from '../smart-info';
import { IStorageRepository, ImmichReadStream, StorageCore, StorageFolder } from '../storage';
import { ISystemConfigRepository, SystemConfigCore } from '../system-config';
import { IBaseJob, IEntityJob, JOBS_ASSET_PAGINATION_SIZE, JobName } from '../job';
import { FACE_THUMBNAIL_SIZE } from '../media';
import {
AssetFaceId,
CropOptions,
IAccessRepository,
IAssetRepository,
IJobRepository,
IMachineLearningRepository,
IMediaRepository,
IMoveRepository,
IPersonRepository,
ISearchRepository,
IStorageRepository,
ISystemConfigRepository,
ImmichReadStream,
UpdateFacesData,
WithoutProperty,
} from '../repositories';
import { StorageCore } from '../storage';
import { SystemConfigCore } from '../system-config';
import {
AssetFaceBoxDto,
AssetFaceUpdateDto,
@ -29,7 +38,6 @@ import {
PersonUpdateDto,
mapPerson,
} from './person.dto';
import { AssetFaceId, IPersonRepository, UpdateFacesData } from './person.repository';
@Injectable()
export class PersonService {
@ -42,6 +50,7 @@ export class PersonService {
@Inject(IAccessRepository) accessRepository: IAccessRepository,
@Inject(IAssetRepository) private assetRepository: IAssetRepository,
@Inject(IMachineLearningRepository) private machineLearningRepository: IMachineLearningRepository,
@Inject(IMoveRepository) moveRepository: IMoveRepository,
@Inject(IMediaRepository) private mediaRepository: IMediaRepository,
@Inject(IPersonRepository) private repository: IPersonRepository,
@Inject(ISearchRepository) private searchRepository: ISearchRepository,
@ -50,8 +59,8 @@ export class PersonService {
@Inject(IJobRepository) private jobRepository: IJobRepository,
) {
this.access = new AccessCore(accessRepository);
this.storageCore = new StorageCore(storageRepository);
this.configCore = SystemConfigCore.create(configRepository);
this.storageCore = new StorageCore(storageRepository, assetRepository, moveRepository, repository);
}
async getAll(authUser: AuthUserDto, dto: PersonSearchDto): Promise<PeopleResponseDto> {
@ -369,11 +378,7 @@ export class PersonService {
return false;
}
const path = this.storageCore.ensurePath(StorageFolder.THUMBNAILS, person.ownerId, `${id}.jpeg`);
if (person.thumbnailPath && person.thumbnailPath !== path) {
await this.storageRepository.moveFile(person.thumbnailPath, path);
await this.repository.update({ id, thumbnailPath: path });
}
await this.storageCore.movePersonFile(person, PersonPathType.FACE);
return true;
}
@ -411,8 +416,8 @@ export class PersonService {
}
this.logger.verbose(`Cropping face for person: ${personId}`);
const thumbnailPath = this.storageCore.ensurePath(StorageFolder.THUMBNAILS, asset.ownerId, `${personId}.jpeg`);
const thumbnailPath = this.storageCore.getPersonThumbnailPath(person);
this.storageCore.ensureFolders(thumbnailPath);
const halfWidth = (x2 - x1) / 2;
const halfHeight = (y2 - y1) / 2;

View file

@ -0,0 +1,23 @@
export * from './access.repository';
export * from './album.repository';
export * from './api-key.repository';
export * from './asset.repository';
export * from './audit.repository';
export * from './communication.repository';
export * from './crypto.repository';
export * from './job.repository';
export * from './library.repository';
export * from './machine-learning.repository';
export * from './media.repository';
export * from './metadata.repository';
export * from './move.repository';
export * from './partner.repository';
export * from './person.repository';
export * from './search.repository';
export * from './shared-link.repository';
export * from './smart-info.repository';
export * from './storage.repository';
export * from './system-config.repository';
export * from './tag.repository';
export * from './user-token.repository';
export * from './user.repository';

View file

@ -1,4 +1,4 @@
import { JobName, QueueName } from './job.constants';
import { JobName, QueueName } from '../job/job.constants';
import {
IAssetDeletionJob,
@ -9,8 +9,7 @@ import {
IEntityJob,
ILibraryFileJob,
ILibraryRefreshJob,
IOfflineLibraryFileJob,
} from './job.interface';
} from '../job/job.interface';
export interface JobCounts {
active: number;
@ -88,7 +87,6 @@ export type JobItem =
// Library Managment
| { name: JobName.LIBRARY_SCAN_ASSET; data: ILibraryFileJob }
| { name: JobName.LIBRARY_MARK_ASSET_OFFLINE; data: IOfflineLibraryFileJob }
| { name: JobName.LIBRARY_SCAN; data: ILibraryRefreshJob }
| { name: JobName.LIBRARY_REMOVE_OFFLINE; data: IEntityJob }
| { name: JobName.LIBRARY_DELETE; data: IEntityJob }

View file

@ -1,5 +1,5 @@
import { LibraryEntity, LibraryType } from '@app/infra/entities';
import { LibraryStatsResponseDto } from './library.dto';
import { LibraryStatsResponseDto } from '../library/library.dto';
export const ILibraryRepository = 'ILibraryRepository';

View file

@ -1,4 +1,4 @@
import { ClassificationConfig, CLIPConfig, RecognitionConfig } from './dto';
import { ClassificationConfig, CLIPConfig, RecognitionConfig } from '../smart-info/dto';
export const IMachineLearningRepository = 'IMachineLearningRepository';

View file

@ -0,0 +1,12 @@
import { MoveEntity, PathType } from '@app/infra/entities';
export const IMoveRepository = 'IMoveRepository';
export type MoveCreate = Pick<MoveEntity, 'oldPath' | 'newPath' | 'entityId' | 'pathType'> & Partial<MoveEntity>;
export interface IMoveRepository {
create(entity: MoveCreate): Promise<MoveEntity>;
getByEntity(entityId: string, pathType: PathType): Promise<MoveEntity | null>;
update(entity: Partial<MoveEntity>): Promise<MoveEntity>;
delete(move: MoveEntity): Promise<MoveEntity>;
}

View file

@ -22,6 +22,7 @@ export interface IPersonRepository {
getAllForUser(userId: string, options: PersonSearchOptions): Promise<PersonEntity[]>;
getAllWithoutFaces(): Promise<PersonEntity[]>;
getById(personId: string): Promise<PersonEntity | null>;
getByName(userId: string, personName: string): Promise<PersonEntity[]>;
getAssets(personId: string): Promise<AssetEntity[]>;
prepareReassignFaces(data: UpdateFacesData): Promise<string[]>;

View file

@ -18,6 +18,7 @@ export const IUserRepository = 'IUserRepository';
export interface IUserRepository {
get(id: string, withDeleted?: boolean): Promise<UserEntity | null>;
getAdmin(): Promise<UserEntity | null>;
hasAdmin(): Promise<boolean>;
getByEmail(email: string, withPassword?: boolean): Promise<UserEntity | null>;
getByStorageLabel(storageLabel: string): Promise<UserEntity | null>;
getByOAuthId(oauthId: string): Promise<UserEntity | null>;

View file

@ -85,3 +85,9 @@ export class SearchDto {
@Transform(toBoolean)
motion?: boolean;
}
export class SearchPeopleDto {
@IsString()
@IsNotEmpty()
name!: string;
}

View file

@ -1,4 +1,3 @@
export * from './dto';
export * from './response-dto';
export * from './search.repository';
export * from './search.service';

View file

@ -15,16 +15,18 @@ import {
searchStub,
} from '@test';
import { plainToInstance } from 'class-transformer';
import { IAlbumRepository } from '../album/album.repository';
import { mapAsset } from '../asset';
import { IAssetRepository } from '../asset/asset.repository';
import { JobName } from '../job';
import { IJobRepository } from '../job/job.repository';
import { IPersonRepository } from '../person/person.repository';
import { IMachineLearningRepository } from '../smart-info';
import { ISystemConfigRepository } from '../system-config';
import {
IAlbumRepository,
IAssetRepository,
IJobRepository,
IMachineLearningRepository,
IPersonRepository,
ISearchRepository,
ISystemConfigRepository,
} from '../repositories';
import { SearchDto } from './dto';
import { ISearchRepository } from './search.repository';
import { SearchService } from './search.service';
jest.useFakeTimers();

View file

@ -1,25 +1,29 @@
import { AlbumEntity, AssetEntity, AssetFaceEntity } from '@app/infra/entities';
import { Inject, Injectable, Logger } from '@nestjs/common';
import { mapAlbumWithAssets } from '../album';
import { IAlbumRepository } from '../album/album.repository';
import { AssetResponseDto, mapAsset } from '../asset';
import { IAssetRepository } from '../asset/asset.repository';
import { AuthUserDto } from '../auth';
import { usePagination } from '../domain.util';
import { IAssetFaceJob, IBulkEntityJob, IJobRepository, JOBS_ASSET_PAGINATION_SIZE, JobName } from '../job';
import { AssetFaceId, IPersonRepository } from '../person';
import { IMachineLearningRepository } from '../smart-info';
import { FeatureFlag, ISystemConfigRepository, SystemConfigCore } from '../system-config';
import { SearchDto } from './dto';
import { SearchResponseDto } from './response-dto';
import { IAssetFaceJob, IBulkEntityJob, JOBS_ASSET_PAGINATION_SIZE, JobName } from '../job';
import { PersonResponseDto } from '../person/person.dto';
import {
AssetFaceId,
IAlbumRepository,
IAssetRepository,
IJobRepository,
IMachineLearningRepository,
IPersonRepository,
ISearchRepository,
ISystemConfigRepository,
OwnedFaceEntity,
SearchCollection,
SearchExploreItem,
SearchResult,
SearchStrategy,
} from './search.repository';
} from '../repositories';
import { FeatureFlag, SystemConfigCore } from '../system-config';
import { SearchDto, SearchPeopleDto } from './dto';
import { SearchResponseDto } from './response-dto';
interface SyncQueue {
upsert: Set<string>;
@ -155,6 +159,10 @@ export class SearchService {
};
}
async searchPerson(authUser: AuthUserDto, dto: SearchPeopleDto): Promise<PersonResponseDto[]> {
return await this.personRepository.getByName(authUser.id, dto.name);
}
async handleIndexAlbums() {
if (!this.enabled) {
return false;

View file

@ -85,6 +85,7 @@ export class ServerConfigDto {
mapTileUrl!: string;
@ApiProperty({ type: 'integer' })
trashDays!: number;
isInitialized!: boolean;
}
export class ServerFeaturesDto implements FeatureFlags {

View file

@ -1,22 +1,40 @@
import { newStorageRepositoryMock, newSystemConfigRepositoryMock, newUserRepositoryMock } from '@test';
import {
newAssetRepositoryMock,
newMoveRepositoryMock,
newPersonRepositoryMock,
newStorageRepositoryMock,
newSystemConfigRepositoryMock,
newUserRepositoryMock,
} from '@test';
import { serverVersion } from '../domain.constant';
import { ISystemConfigRepository } from '../index';
import { IStorageRepository } from '../storage';
import { IUserRepository } from '../user';
import {
IAssetRepository,
IMoveRepository,
IPersonRepository,
IStorageRepository,
ISystemConfigRepository,
IUserRepository,
} from '../repositories';
import { ServerInfoService } from './server-info.service';
describe(ServerInfoService.name, () => {
let sut: ServerInfoService;
let assetMock: jest.Mocked<IAssetRepository>;
let configMock: jest.Mocked<ISystemConfigRepository>;
let moveMock: jest.Mocked<IMoveRepository>;
let personMock: jest.Mocked<IPersonRepository>;
let storageMock: jest.Mocked<IStorageRepository>;
let userMock: jest.Mocked<IUserRepository>;
beforeEach(() => {
assetMock = newAssetRepositoryMock();
configMock = newSystemConfigRepositoryMock();
moveMock = newMoveRepositoryMock();
personMock = newPersonRepositoryMock();
storageMock = newStorageRepositoryMock();
userMock = newUserRepositoryMock();
sut = new ServerInfoService(configMock, userMock, storageMock);
sut = new ServerInfoService(assetMock, configMock, moveMock, personMock, userMock, storageMock);
});
it('should work', () => {

View file

@ -1,9 +1,17 @@
import { Inject, Injectable } from '@nestjs/common';
import { mimeTypes, serverVersion } from '../domain.constant';
import { asHumanReadable } from '../domain.util';
import { IStorageRepository, StorageCore, StorageFolder } from '../storage';
import { ISystemConfigRepository, SystemConfigCore } from '../system-config';
import { IUserRepository, UserStatsQueryResponse } from '../user';
import {
IAssetRepository,
IMoveRepository,
IPersonRepository,
IStorageRepository,
ISystemConfigRepository,
IUserRepository,
UserStatsQueryResponse,
} from '../repositories';
import { StorageCore, StorageFolder } from '../storage';
import { SystemConfigCore } from '../system-config';
import {
ServerConfigDto,
ServerFeaturesDto,
@ -20,12 +28,15 @@ export class ServerInfoService {
private storageCore: StorageCore;
constructor(
@Inject(IAssetRepository) assetRepository: IAssetRepository,
@Inject(ISystemConfigRepository) configRepository: ISystemConfigRepository,
@Inject(IMoveRepository) moveRepository: IMoveRepository,
@Inject(IPersonRepository) personRepository: IPersonRepository,
@Inject(IUserRepository) private userRepository: IUserRepository,
@Inject(IStorageRepository) private storageRepository: IStorageRepository,
) {
this.configCore = SystemConfigCore.create(configRepository);
this.storageCore = new StorageCore(storageRepository);
this.storageCore = new StorageCore(storageRepository, assetRepository, moveRepository, personRepository);
}
async getInfo(): Promise<ServerInfoResponseDto> {
@ -63,11 +74,14 @@ export class ServerInfoService {
// TODO move to system config
const loginPageMessage = process.env.PUBLIC_LOGIN_PAGE_MESSAGE || '';
const isInitialized = await this.userRepository.hasAdmin();
return {
loginPageMessage,
mapTileUrl: config.map.tileUrl,
trashDays: config.trash.days,
oauthButtonText: config.oauth.buttonText,
isInitialized,
};
}

View file

@ -1,4 +1,3 @@
export * from './shared-link-response.dto';
export * from './shared-link.dto';
export * from './shared-link.repository';
export * from './shared-link.service';

View file

@ -1,3 +1,4 @@
import { SharedLinkType } from '@app/infra/entities';
import { BadRequestException, ForbiddenException } from '@nestjs/common';
import {
IAccessRepositoryMock,
@ -12,9 +13,8 @@ import {
} from '@test';
import { when } from 'jest-when';
import _ from 'lodash';
import { SharedLinkType } from '../../infra/entities/shared-link.entity';
import { AssetIdErrorReason, ICryptoRepository } from '../index';
import { ISharedLinkRepository } from './shared-link.repository';
import { AssetIdErrorReason } from '../asset';
import { ICryptoRepository, ISharedLinkRepository } from '../repositories';
import { SharedLinkService } from './shared-link.service';
describe(SharedLinkService.name, () => {

View file

@ -1,12 +1,11 @@
import { AssetEntity, SharedLinkEntity, SharedLinkType } from '@app/infra/entities';
import { BadRequestException, ForbiddenException, Inject, Injectable } from '@nestjs/common';
import { AccessCore, IAccessRepository, Permission } from '../access';
import { AccessCore, Permission } from '../access';
import { AssetIdErrorReason, AssetIdsDto, AssetIdsResponseDto } from '../asset';
import { AuthUserDto } from '../auth';
import { ICryptoRepository } from '../crypto';
import { IAccessRepository, ICryptoRepository, ISharedLinkRepository } from '../repositories';
import { SharedLinkResponseDto, mapSharedLink, mapSharedLinkWithNoExif } from './shared-link-response.dto';
import { SharedLinkCreateDto, SharedLinkEditDto } from './shared-link.dto';
import { ISharedLinkRepository } from './shared-link.repository';
@Injectable()
export class SharedLinkService {

Some files were not shown because too many files have changed in this diff Show more