Fixes the intermittent EPIPE errors that myself and others are seeing.
By explicitly returning a promise we ensure the caller correctly waits until the `sendFile` is complete before potentially closing or cleaning the socket. This is the most likely bug that would cause EPIPE errors.
Fix was confirmed on a live system -- would benefit from a unit test
though.
* fix(mobile): album thumbnail list tile overflow on large album title
* fix: notify clients about live photo linked event
* refactor: notify clients during meta extraction
---------
Co-authored-by: shalong-tanwen <139912620+shalong-tanwen@users.noreply.github.com>
* chore: text correction
* fix: update activities stat only when the widget is mounted
* feat(mobile): edit date time
* feat(mobile): edit location
* chore(build): update gradle wrapper - 7.6.3
* style: dropdownmenu styling
* style: wrap locationpicker in singlechildscrollview
* test: add unit test for getTZAdjustedTimeAndOffset
* pr changes
---------
Co-authored-by: shalong-tanwen <139912620+shalong-tanwen@users.noreply.github.com>
* feat: unassign person faces
* multiple improvements
* chore: regenerate api
* feat: improve face interactions in photos
* fix: tests
* fix: tests
* optimize
* fix: wrong assignment on complex-multiple re-assignments
* fix: thumbnails with large photos
* fix: complex reassign
* fix: don't send people with faces
* fix: person thumbnail generation
* chore: regenerate api
* add tess
* feat: face box even when zoomed
* fix: change feature photo
* feat: make the blue icon hoverable
* chore: regenerate api
* feat: use websocket
* fix: loading spinner when clicking on the done button
* fix: use the svelte way
* fix: tests
* simplify
* fix: unused vars
* fix: remove unused code
* fix: add migration
* chore: regenerate api
* ci: add unit tests
* chore: regenerate api
* feat: if a new person is created for a face and the server takes more than 15 seconds to generate the person thumbnail, don't wait for it
* reorganize
* chore: regenerate api
* feat: global edit
* pr feedback
* pr feedback
* simplify
* revert test
* fix: face generation
* fix: tests
* fix: face generation
* fix merge
* feat: search names in unmerge face selector modal
* fix: merge face selector
* simplify feature photo generation
* fix: change endpoint
* pr feedback
* chore: fix merge
* chore: fix merge
* fix: tests
* fix: edit & hide buttons
* fix: tests
* feat: show if person is hidden
* feat: rename face to person
* feat: split in new panel
* copy-paste-error
* pr feedback
* fix: feature photo
* do not leak faces
* fix: unmerge modal
* fix: merge modal event
* feat(server): remove duplicates
* fix: title for image thumbnails
* fix: disable side panel when there's no face until next PR
---------
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
Modify Access repository, to evaluate `asset` permissions in bulk.
Queries have been validated to match what they currently generate for single ids.
Queries:
* `asset` album access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "albums" "AlbumEntity"
LEFT JOIN "albums_assets_assets" "AlbumEntity_AlbumEntity__AlbumEntity_assets"
ON "AlbumEntity_AlbumEntity__AlbumEntity_assets"."albumsId"="AlbumEntity"."id"
LEFT JOIN "assets" "AlbumEntity__AlbumEntity_assets"
ON "AlbumEntity__AlbumEntity_assets"."id"="AlbumEntity_AlbumEntity__AlbumEntity_assets"."assetsId"
AND "AlbumEntity__AlbumEntity_assets"."deletedAt" IS NULL
LEFT JOIN "albums_shared_users_users" "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"
ON "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."albumsId"="AlbumEntity"."id"
LEFT JOIN "users" "AlbumEntity__AlbumEntity_sharedUsers"
ON "AlbumEntity__AlbumEntity_sharedUsers"."id"="AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."usersId"
AND "AlbumEntity__AlbumEntity_sharedUsers"."deletedAt" IS NULL
WHERE
(
("AlbumEntity"."ownerId" = $1 AND "AlbumEntity__AlbumEntity_assets"."id" = $2)
OR ("AlbumEntity__AlbumEntity_sharedUsers"."id" = $3 AND "AlbumEntity__AlbumEntity_assets"."id" = $4)
OR ("AlbumEntity"."ownerId" = $5 AND "AlbumEntity__AlbumEntity_assets"."livePhotoVideoId" = $6)
OR ("AlbumEntity__AlbumEntity_sharedUsers"."id" = $7 AND "AlbumEntity__AlbumEntity_assets"."livePhotoVideoId" = $8)
)
AND "AlbumEntity"."deletedAt" IS NULL
)
LIMIT 1
-- After
SELECT
"asset"."id" AS "assetId",
"asset"."livePhotoVideoId" AS "livePhotoVideoId"
FROM "albums" "album"
INNER JOIN "albums_assets_assets" "album_asset"
ON "album_asset"."albumsId"="album"."id"
INNER JOIN "assets" "asset"
ON "asset"."id"="album_asset"."assetsId"
AND "asset"."deletedAt" IS NULL
LEFT JOIN "albums_shared_users_users" "album_sharedUsers"
ON "album_sharedUsers"."albumsId"="album"."id"
LEFT JOIN "users" "sharedUsers"
ON "sharedUsers"."id"="album_sharedUsers"."usersId"
AND "sharedUsers"."deletedAt" IS NULL
WHERE
(
"album"."ownerId" = $1
OR "sharedUsers"."id" = $2
)
AND (
"asset"."id" IN ($3, $4)
OR "asset"."livePhotoVideoId" IN ($5, $6)
)
AND "album"."deletedAt" IS NULL
```
* `asset` owner access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "assets" "AssetEntity"
WHERE
"AssetEntity"."id" = $1
AND "AssetEntity"."ownerId" = $2
)
LIMIT 1
-- After
SELECT
"AssetEntity"."id" AS "AssetEntity_id"
FROM "assets" "AssetEntity"
WHERE
"AssetEntity"."id" IN ($1, $2)
AND "AssetEntity"."ownerId" = $3
```
* `asset` partner access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "partners" "PartnerEntity"
LEFT JOIN "users" "PartnerEntity__PartnerEntity_sharedWith"
ON "PartnerEntity__PartnerEntity_sharedWith"."id"="PartnerEntity"."sharedWithId"
AND "PartnerEntity__PartnerEntity_sharedWith"."deletedAt" IS NULL
LEFT JOIN "users" "PartnerEntity__PartnerEntity_sharedBy"
ON "PartnerEntity__PartnerEntity_sharedBy"."id"="PartnerEntity"."sharedById"
AND "PartnerEntity__PartnerEntity_sharedBy"."deletedAt" IS NULL
LEFT JOIN "assets" "0aabe9f4a62b794e2c24a074297e534f51a4ac6c"
ON "0aabe9f4a62b794e2c24a074297e534f51a4ac6c"."ownerId"="PartnerEntity__PartnerEntity_sharedBy"."id"
AND "0aabe9f4a62b794e2c24a074297e534f51a4ac6c"."deletedAt" IS NULL
LEFT JOIN "users" "PartnerEntity__sharedBy"
ON "PartnerEntity__sharedBy"."id"="PartnerEntity"."sharedById"
AND "PartnerEntity__sharedBy"."deletedAt" IS NULL
LEFT JOIN "users" "PartnerEntity__sharedWith"
ON "PartnerEntity__sharedWith"."id"="PartnerEntity"."sharedWithId"
AND "PartnerEntity__sharedWith"."deletedAt" IS NULL
WHERE
"PartnerEntity__PartnerEntity_sharedWith"."id" = $1
AND "0aabe9f4a62b794e2c24a074297e534f51a4ac6c"."id" = $2
)
LIMIT 1
-- After
SELECT
"asset"."id" AS "assetId"
FROM "partners" "partner"
INNER JOIN "users" "sharedBy"
ON "sharedBy"."id"="partner"."sharedById"
AND "sharedBy"."deletedAt" IS NULL
INNER JOIN "assets" "asset"
ON "asset"."ownerId"="sharedBy"."id"
AND "asset"."deletedAt" IS NULL
WHERE
"partner"."sharedWithId" = $1
AND "asset"."id" IN ($2, $3)
```
* `asset` shared link access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "shared_links" "SharedLinkEntity"
LEFT JOIN "albums" "SharedLinkEntity__SharedLinkEntity_album"
ON "SharedLinkEntity__SharedLinkEntity_album"."id"="SharedLinkEntity"."albumId"
AND "SharedLinkEntity__SharedLinkEntity_album"."deletedAt" IS NULL
LEFT JOIN "albums_assets_assets" "760f12c00d97bdcec1ce224d1e3bf449859942b6"
ON "760f12c00d97bdcec1ce224d1e3bf449859942b6"."albumsId"="SharedLinkEntity__SharedLinkEntity_album"."id"
LEFT JOIN "assets" "4a35f463ae8c5544ede95c4b6d9ce8c686b6bfe6"
ON "4a35f463ae8c5544ede95c4b6d9ce8c686b6bfe6"."id"="760f12c00d97bdcec1ce224d1e3bf449859942b6"."assetsId"
AND "4a35f463ae8c5544ede95c4b6d9ce8c686b6bfe6"."deletedAt" IS NULL
LEFT JOIN "shared_link__asset" "SharedLinkEntity__SharedLinkEntity_assets_SharedLinkEntity"
ON "SharedLinkEntity__SharedLinkEntity_assets_SharedLinkEntity"."sharedLinksId"="SharedLinkEntity"."id"
LEFT JOIN "assets" "SharedLinkEntity__SharedLinkEntity_assets"
ON "SharedLinkEntity__SharedLinkEntity_assets"."id"="SharedLinkEntity__SharedLinkEntity_assets_SharedLinkEntity"."assetsId"
AND "SharedLinkEntity__SharedLinkEntity_assets"."deletedAt" IS NULL
WHERE (
("SharedLinkEntity"."id" = $1 AND "4a35f463ae8c5544ede95c4b6d9ce8c686b6bfe6"."id" = $2)
OR ("SharedLinkEntity"."id" = $3 AND "SharedLinkEntity__SharedLinkEntity_assets"."id" = $4)
OR ("SharedLinkEntity"."id" = $5 AND "4a35f463ae8c5544ede95c4b6d9ce8c686b6bfe6"."livePhotoVideoId" = $6)
OR ("SharedLinkEntity"."id" = $7 AND "SharedLinkEntity__SharedLinkEntity_assets"."livePhotoVideoId" = $8)
)
)
LIMIT 1
-- After
SELECT
"assets"."id" AS "assetId",
"assets"."livePhotoVideoId" AS "assetLivePhotoVideoId",
"albumAssets"."id" AS "albumAssetId",
"albumAssets"."livePhotoVideoId" AS "albumAssetLivePhotoVideoId"
FROM "shared_links" "sharedLink"
LEFT JOIN "albums" "album"
ON "album"."id"="sharedLink"."albumId"
AND "album"."deletedAt" IS NULL
LEFT JOIN "shared_link__asset" "assets_sharedLink"
ON "assets_sharedLink"."sharedLinksId"="sharedLink"."id"
LEFT JOIN "assets" "assets"
ON "assets"."id"="assets_sharedLink"."assetsId"
AND "assets"."deletedAt" IS NULL
LEFT JOIN "albums_assets_assets" "album_albumAssets"
ON "album_albumAssets"."albumsId"="album"."id"
LEFT JOIN "assets" "albumAssets"
ON "albumAssets"."id"="album_albumAssets"."assetsId"
AND "albumAssets"."deletedAt" IS NULL
WHERE
"sharedLink"."id" = $1
AND (
"assets"."id" IN ($2, $3)
OR "albumAssets"."id" IN ($4, $5)
OR "assets"."livePhotoVideoId" IN ($6, $7)
OR "albumAssets"."livePhotoVideoId" IN ($8, $9)
)
```
* fix(web): status box rendering
* Syntax improvement for api import
Co-authored-by: martin <74269598+martabal@users.noreply.github.com>
---------
Co-authored-by: Daniel Taylor <daniel.k.taylor1@gmail.com>
Co-authored-by: martin <74269598+martabal@users.noreply.github.com>
* Allow showing hidden people in image asset details view
This makes it possible to easily find people/faces that have accidentally been hidden.
Unhiding them still requires clicking on the person to go to their page to unhide them.
* Update web/src/lib/components/asset-viewer/detail-panel.svelte
Co-authored-by: martin <74269598+martabal@users.noreply.github.com>
---------
Co-authored-by: brokeh <git@brocky.net>
Co-authored-by: martin <74269598+martabal@users.noreply.github.com>
* fix(web): disable metadata edit if user is not owner
* pr feedback
* pr feedback
* get data from page data
* fix: better representation
* feat: warn user if there's issues with the selected assets
---------
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
The only issue was that line endings of shell scripts may be \r\n when cloned on Windows with autocrlf and shells don't like that.
Co-authored-by: brokeh <git@brocky.net>
* chore: rebase and clean-up
* feat: sync description, add e2e tests
* feat: simplify web code
* chore: unit tests
* fix: linting
* Bug fix with the arrows key
* timezone typeahead filter
timezone typeahead filter
* small stlying
* format fix
* Bug fix in the map selection
Bug fix in the map selection
* Websocket basic
Websocket basic
* Update metadata visualisation through the websocket
* Update timeline
* fix merge
* fix web
* fix web
* maplibre system
* format fix
* format fix
* refactor: clean up
* Fix small bug in the hour/timezone
* Don't diplay modify for readOnly asset
* Add log in case of failure
* Formater + try/catch error
* Remove everything related to websocket
* Revert "Remove everything related to websocket"
This reverts commit 14bcb9e1e4.
* remove notification
* fix test
---------
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
* refactor: scaffoldwhen to log errors during scaffold body render
* refactor: onError and onLoading scaffoldbody
* refactor: more scaffold body to custom extension
* refactor: add skiploadingonrefresh
* Snackbar color
---------
Co-authored-by: shalong-tanwen <139912620+shalong-tanwen@users.noreply.github.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
This fixes issue #4397 and automatically adds the https protocol to the server endpoint url if it is missing
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
* feat(web): Lazy load thumbnails on the people page
Instead of loading all people thumbnails at once, only the first few
should be loaded eagerly.
This reduces the load on client and server side.
* chore: change name
---------
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
Modify Access repository, to evaluate `authDevice`, `library`, `partner`,
`person`, and `timeline` permissions in bulk.
Queries have been validated to match what they currently generate for
single ids.
As an extra performance improvement, we now use a custom QueryBuilder
for the Partners queries, to avoid the eager relationships that add
unneeded `LEFT JOIN` clauses. We only filter based on the ids present in
the `partners` table, so those joins can be avoided.
Queries:
* `library` owner access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "libraries" "LibraryEntity"
WHERE
"LibraryEntity"."id" = $1
AND "LibraryEntity"."ownerId" = $2
AND "LibraryEntity"."deletedAt" IS NULL
)
LIMIT 1
-- After
SELECT "LibraryEntity"."id" AS "LibraryEntity_id"
FROM "libraries" "LibraryEntity"
WHERE
"LibraryEntity"."id" IN ($1, $2)
AND "LibraryEntity"."ownerId" = $3
AND "LibraryEntity"."deletedAt" IS NULL
```
* `library` partner access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "partners" "PartnerEntity"
LEFT JOIN "users" "PartnerEntity__sharedBy"
ON "PartnerEntity__sharedBy"."id"="PartnerEntity"."sharedById"
AND "PartnerEntity__sharedBy"."deletedAt" IS NULL
LEFT JOIN "users" "PartnerEntity__sharedWith"
ON "PartnerEntity__sharedWith"."id"="PartnerEntity"."sharedWithId"
AND "PartnerEntity__sharedWith"."deletedAt" IS NULL
WHERE
"PartnerEntity"."sharedWithId" = $1
AND "PartnerEntity"."sharedById" = $2
)
LIMIT 1
-- After
SELECT
"partner"."sharedById" AS "partner_sharedById",
"partner"."sharedWithId" AS "partner_sharedWithId"
FROM "partners" "partner"
WHERE
"partner"."sharedById" IN ($1, $2)
AND "partner"."sharedWithId" = $3
```
* `authDevice` owner access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "user_token" "UserTokenEntity"
WHERE
"UserTokenEntity"."userId" = $1
AND "UserTokenEntity"."id" = $2
)
LIMIT 1
-- After
SELECT "UserTokenEntity"."id" AS "UserTokenEntity_id"
FROM "user_token" "UserTokenEntity"
WHERE
"UserTokenEntity"."userId" = $1
AND "UserTokenEntity"."id" IN ($2, $3)
```
* `timeline` partner access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "partners" "PartnerEntity"
LEFT JOIN "users" "PartnerEntity__sharedBy"
ON "PartnerEntity__sharedBy"."id"="PartnerEntity"."sharedById"
AND "PartnerEntity__sharedBy"."deletedAt" IS NULL
LEFT JOIN "users" "PartnerEntity__sharedWith"
ON "PartnerEntity__sharedWith"."id"="PartnerEntity"."sharedWithId"
AND "PartnerEntity__sharedWith"."deletedAt" IS NULL
WHERE
"PartnerEntity"."sharedWithId" = $1
AND "PartnerEntity"."sharedById" = $2
)
LIMIT 1
-- After
SELECT
"partner"."sharedById" AS "partner_sharedById",
"partner"."sharedWithId" AS "partner_sharedWithId"
FROM "partners" "partner"
WHERE
"partner"."sharedById" IN ($1, $2)
AND "partner"."sharedWithId" = $3
```
* `person` owner access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "person" "PersonEntity"
WHERE
"PersonEntity"."id" = $1
AND "PersonEntity"."ownerId" = $2
)
LIMIT 1
-- After
SELECT "PersonEntity"."id" AS "PersonEntity_id"
FROM "person" "PersonEntity"
WHERE
"PersonEntity"."id" IN ($1, $2)
AND "PersonEntity"."ownerId" = $3
```
* `partner` update access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "partners" "PartnerEntity"
LEFT JOIN "users" "PartnerEntity__sharedBy"
ON "PartnerEntity__sharedBy"."id"="PartnerEntity"."sharedById"
AND "PartnerEntity__sharedBy"."deletedAt" IS NULL
LEFT JOIN "users" "PartnerEntity__sharedWith"
ON "PartnerEntity__sharedWith"."id"="PartnerEntity"."sharedWithId"
AND "PartnerEntity__sharedWith"."deletedAt" IS NULL
WHERE
"PartnerEntity"."sharedWithId" = $1
AND "PartnerEntity"."sharedById" = $2
)
LIMIT 1
-- After
SELECT
"partner"."sharedById" AS "partner_sharedById",
"partner"."sharedWithId" AS "partner_sharedWithId"
FROM "partners" "partner"
WHERE
"partner"."sharedById" IN ($1, $2)
AND "partner"."sharedWithId" = $3
```
* refactor: mobile - send livephoto as a separate request
* fix: create new request for live asset
---------
Co-authored-by: shalong-tanwen <139912620+shalong-tanwen@users.noreply.github.com>
* chore(server): Check album permissions in bulk
Modify Access repository, to evaluate `album` permissions in bulk.
Queries have been validated to match what they currently generate for
single ids.
Queries:
* Owner access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "albums" "AlbumEntity"
WHERE
"AlbumEntity"."id" = $1
AND "AlbumEntity"."ownerId" = $2
AND "AlbumEntity"."deletedAt" IS NULL
)
LIMIT 1
-- After
SELECT
"AlbumEntity"."id" AS "AlbumEntity_id"
FROM "albums" "AlbumEntity"
WHERE
"AlbumEntity"."id" IN ($1, $2)
AND "AlbumEntity"."ownerId" = $3
AND "AlbumEntity"."deletedAt" IS NULL
```
* Shared link access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "shared_links" "SharedLinkEntity"
WHERE
"SharedLinkEntity"."id" = $1
AND "SharedLinkEntity"."albumId" = $2
)
LIMIT 1
-- After
SELECT
"SharedLinkEntity"."albumId" AS "SharedLinkEntity_albumId",
"SharedLinkEntity"."id" AS "SharedLinkEntity_id"
FROM "shared_links" "SharedLinkEntity"
WHERE
"SharedLinkEntity"."id" = $1
AND "SharedLinkEntity"."albumId" IN ($2, $3)
```
* Shared album access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "albums" "AlbumEntity"
LEFT JOIN "albums_shared_users_users" "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"
ON "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."albumsId"="AlbumEntity"."id"
LEFT JOIN "users" "AlbumEntity__AlbumEntity_sharedUsers"
ON "AlbumEntity__AlbumEntity_sharedUsers"."id"="AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."usersId"
AND "AlbumEntity__AlbumEntity_sharedUsers"."deletedAt" IS NULL
WHERE
"AlbumEntity"."id" = $1
AND "AlbumEntity__AlbumEntity_sharedUsers"."id" = $2
AND "AlbumEntity"."deletedAt" IS NULL
)
LIMIT 1
-- After
SELECT
"AlbumEntity"."id" AS "AlbumEntity_id"
FROM "albums" "AlbumEntity"
LEFT JOIN "albums_shared_users_users" "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"
ON "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."albumsId"="AlbumEntity"."id"
LEFT JOIN "users" "AlbumEntity__AlbumEntity_sharedUsers"
ON "AlbumEntity__AlbumEntity_sharedUsers"."id"="AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."usersId"
AND "AlbumEntity__AlbumEntity_sharedUsers"."deletedAt" IS NULL
WHERE
"AlbumEntity"."id" IN ($1, $2)
AND "AlbumEntity__AlbumEntity_sharedUsers"."id" = $3
AND "AlbumEntity"."deletedAt" IS NULL
```
* chore(server): Add set utils, avoid double queries for same ids
* chore(server): Review feedback
* feat: add system metadata repository for storing key values for internal usage
* feat: add database entities for geodata
* feat: move reverse geocoding from local-reverse-geocoder to postgresql
* infra: disable synchronization for geodata_places table until typeorm supports earth column
* feat: remove cities override config as we will default all instances to cities500 now
* test: e2e tests don't clear geodata tables on reset
The query executed when loading the "People" page joins, among others, over "personId".
The added indices improve the overall performance of those JOIN queries.
Additionally, one ORDER BY clause is dropped since the resulting values
will always be TRUE, and thus, sorting them does not change the result.
* chore(server): Prepare access interfaces for bulk permission checks
This change adds the `AccessCore.getAllowedIds` method, to evaluate
permissions in bulk, along with some other `getAllowedIds*` private
methods.
The added methods still calculate permissions by id, and are not
optimized to reduce the amount of queries and execution time, which will
be implemented in separate pull requests.
Services that were evaluating permissions in a loop have been refactored
to make use of the bulk approach.
* chore(server): Apply review suggestions
* chore(server): Make multiple-permission check more readable
* fix(cli): upload large file with openAPI
* function name
* fix: use boolean false
* chore: version bump
* fix: package-lock version
---------
Co-authored-by: Jonathan Jogenfors <jonathan@jogenfors.se>
* chore(cli): add version option
* chore: update build path
* fix: defer to package details for path to entrypoint
---------
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
* Fix
* open api
* Change to list and delete
* Bug fix
* Change name
* refactor: clean up code and add test
---------
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
* Allow building and installing cli
* feat: add format fix
* docs: remove cli folder
* feat: use immich scoped package
* feat: rewrite cli readme
* docs: add info on running without building
* cleanup
* chore: remove import functionality from cli
* feat: add logout to cli
* docs: add todo for file format from server
* docs: add compilation step to cli
* fix: success message spacing
* feat: can create albums
* fix: add check step to cli
* fix: typos
* feat: pull file formats from server
* chore: use crawl service from server
* chore: fix lint
* docs: add cli documentation
* chore: rename ignore pattern
* chore: add version number to cli
* feat: use sdk
* fix: cleanup
* feat: album name on windows
* chore: remove skipped asset field
* feat: add more info to server-info command
* chore: cleanup
* chore: remove unneeded packages
* chore: fix docs links
* feat: add cli v2 milestone
* fix: set correct cli date
---------
Co-authored-by: Alex <alex.tran1502@gmail.com>
* fix(mobile): use proper context for popping out from share
* mobile: use proper context for popping
---------
Co-authored-by: shalong-tanwen <139912620+shalong-tanwen@users.noreply.github.com>
* fix(mobile): Mark more strings for translation
Moving more strings to the `i18n` JSON file, and also including their
es-US translations.
* Add more translatable strings
* Added extra lower resolution options for thumbnails
Added 200p and 720p as options for small and large thumbnails resolutions respectively.
* Also included 1080p in large thumbnails resolution options
* feat(server): GET /assets endpoint
* chore: open api
* chore: use dumb name
* feat: search by make, model, lens, city, state, country
* chore: open api
* chore: pagination validation and tests
* chore: pr feedback
* feat(web): Add file path info for external assets
Add file path information to the asset details panel for External assets. This will allow a user to easily locate said asset in the filesystem, to perform any desired tasks on that asset. Styling was chosen to be as unobtrusive as possible.
* feat(web): toggleable file path info for external assets
If the user is the owner of the current asset and it's an external asset, then add a button next to the filename to reveal the asset's file path.
* show path on owned asset and styling
---------
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
* fix(mobile): Cannot return to logged in screen due to name changes
* fix(mobile): Cannot return to logged in screen due to name changes
* remove deadcode
* test deprecate
* Add deprecated decorator
* revert api change
* fix: like in global activity
* refactor: rename isGlobal to ReactionLevel.Album
* chore: open api
* chore: e2e test for album vs comment duplicate like checking
---------
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
* Add AssetJobStatus
* fentity
* Add jobStatus field to AssetEntity
* Fix the migration doc paths
* Filter on facesRecognizedAt
* Set facesRecognizedAt field
* Test for facesRecognizedAt
* Done testing
* Adjust FK properties
* Add tests for WithoutProperty.FACES
* chore: non-nullable
---------
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
* refactor: move all extensions to separate package
* refactor(mobile): add BuildContext extension
* refactor(mobile): use theme getters from context
* refactor(mobile): use media query size from context
* refactor(mobile): use auto router methods from context
* refactor(mobile): use navigator methods from context
---------
Co-authored-by: shalong-tanwen <139912620+shalong-tanwen@users.noreply.github.com>
* maplibre on web, custom styles from server
Actually use new vector tile server, custom style.json
support multiple style files, light/dark mode
cleanup, use new map everywhere
send file directly instead of loading first
better light/dark mode switching
remove leaflet
fix mapstyles dto, first draft of map settings
delete and add styles
fix delete default styles
fix tests
only allow one light and one dark style url
revert config core changes
fix server config store
fix tests
move axios fetches to repo
fix package-lock
fix tests
* open api
* add assets to docker container
* web: use mapSettings color for style
* style: add unique ids to map styles
* mobile: use style json for vector / raster
* do not use svelte-material-icons
* add click events to markers, simplify asset detail map
* improve map performance by using asset thumbnails for markers instead of original file
* Remove custom attribution
(by request)
* mobile: update map attribution
* style: map dark mode
* style: map light mode
* zoom level for state
* styling
* overflow gradient
* Limit maxZoom to 14
* mobile: listen for mapStyle changes in MapThumbnail
* mobile: update concurrency
---------
Co-authored-by: shalong-tanwen <139912620+shalong-tanwen@users.noreply.github.com>
Co-authored-by: bo0tzz <git@bo0tzz.me>
Co-authored-by: Alex <alex.tran1502@gmail.com>
* Use base image for server build
* Clean up build scripts
* target tags for base image
* use prod tag instead of runtime
* use runtime stage for dev
---------
Co-authored-by: mertalev <101130780+mertalev@users.noreply.github.com>
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
This change makes albums with same start and end date, to just display a
single date.
Currently, date range is displayed as `Nov 9 - Nov 9 2023`. With this
change, just `Nov 9 2023` is displayed. No changes are made for albums
where start and end dates do not match.
* feat(mobile): shared album activity disable handling
* not show comment/like option on non-shared album, alternative text when activity is disabled
---------
Co-authored-by: shalong-tanwen <139912620+shalong-tanwen@users.noreply.github.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
* fix(server): global activity like duplicate search
* mobile: user_circle_avatar - fallback to text icon if no profile pic available
* mobile: use favourite icon in search "your activity"
* feat(mobile): shared album activities
* mobile: align hearts with user profile icon
* styling
* replace bottom sheet with dismissible
* add auto focus to the input
---------
Co-authored-by: shalong-tanwen <139912620+shalong-tanwen@users.noreply.github.com>
Co-authored-by: Alex <alex.tran1502@gmail.com>
* mobile: tool-tip for server url in app bar dialog
* fix: Add Inkwell around the entire profile image
* mobile: open documentation and github in browser
---------
Co-authored-by: shalong-tanwen <139912620+shalong-tanwen@users.noreply.github.com>
* feat(web): shuffle slideshow order
* Fix play/stop issues
* Enter/exit fullscreen mode
* Prevent navigation to the next asset after exiting slideshow mode
* Fix entering the slideshow mode from an album page
* Simplify markup of the AssetViewer
Group viewer area and navigation (prev/next/slideshow bar) controls together
* Select a random asset from a random bucket
* Preserve assets order in random mode
* Exit fullscreen mode only if it is active
* Extract SlideshowHistory class
* Use traditional functions instead of arrow functions
* Refactor SlideshowHistory class
* Extract SlideshowBar component
* Fix comments
* Hide Say something in slideshow mode
---------
Co-authored-by: brighteyed <sergey.kondrikov@gmail.com>
* add automatic library scan config options
* add validation
* open api
* use CronJob instead of cron-validator
* fix tests
* catch potential error of the library scan initialization
* better description for input field
* move library scan job initialization to server app service
* fix tests
* add comments to all parameters of cronjob contructor
* make scan a child of a more general library object
* open api
* chore: cleanup
* move cronjob handling to job repoistory
* web: select for common cron expressions
* fix open api
* fix tests
* put scanning settings in nested accordion
* fix system config validation
* refactor, tests
---------
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
* fix(server): Correctly set album start and end dates
Currently, the query that retrieves album assets uses
`ORDER BY assets.fileCreatedAt DESC`, which makes the existing logic
return the start/end dates reversed (with `startDate` being taken from
the first asset in the array).
Instead of using the index-based approach, this change iterates through
assets to get the min/max `fileCreatedAt`. This will avoid any future
issues, if the query ordering changes, or becomes customizable (e.g. in
case the user prefers to visualize older assets first).
* fix: Maintain constant cost and only swap variables if needed
* fix(server/oauth): Handle errors from OAuth Discovery.
* fix(server/oauth): Better fix for OAuth discovery error.
* This doesn't break tests.
* Update server/tsconfig.json
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
* Revert back to the mostly original way.
---------
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
* refactor(mobile): add app bar to library and sharing
* mobile: add app bar dialog
* fix(mobile): refetch profile image only when path is changed
* mobile: add server url to dialog
* mobile: move trash to library app bar
* replace discord link with github
* user confirmation before sign out
* edit some styles
---------
Co-authored-by: shalong-tanwen <139912620+shalong-tanwen@users.noreply.github.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
* Fixed semantic problem in navigation-bar.svelte
Closes [4574]
* Reintroduced styling for navigation-bar.svelte
Based on button styling
* Horizontal rule set as decoration in navigation-bar.svelte
* aria-current added as an assistive technology indication for current page
Horizontal rule indicates current page for users that can see the screen and with aria-current screen-reader users get the same information
* Temporary fix for accessibility name of the link when SVG replaces text
This is a temporary fix as we first need to fix the svelte-material-icons to support role="img" on rendered SVGs.
* fix(web): horizontal rule replaced with div
hr is not semantically correct, therefore we tried with role="decoration" (that should be role="presentation") but it's actually better to just use a div as it's best practice to not override semantics when we can avoid that...
Btw. the semantics for active element for assistive technology is added with previous commit setting aria-current on the link...
* chore: format
---------
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
* add initial ui and api definitions for stylesheets
* proper saving
* make custom css work
* add textarea
* rebuild api
* run prettier
* add typecast
* update typings
* move css accordion to be sorted alphabetically
* set content-type properly
* rename stylesheets to theme
* fix server test
* fix(mobile): render error on switching asset while video playing
* fix(mobile): generate proper hero tags for assets from DTOs
---------
Co-authored-by: shalong-tanwen <139912620+shalong-tanwen@users.noreply.github.com>
* Added missing alt text on logo in documentation
* Better to use CSS to do the uppercases
Some assistive technology can spell each letter instead of read the word, depends a lot on internals but it's better to prevent...
* Semantic fix - single paragraph instead of many + uppercase via CSS
Single sentence in single paragraph, using css instead
* React requires className instead of class...
* Unnest Trash link from Archive
Add `AlbumRepository` method to retrieve an album's asset ids, with an
optional parameter to only filter by the provided asset ids. With this,
we can now check asset membership using a single query.
When adding or removing assets to an album, checking whether each asset
is already present in the album now requires a single query, instead of
one query per asset.
Related to #4539 performance improvements.
Before:
```
// Asset membership and permissions check (2 queries per asset)
immich_server | query: SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (SELECT 1 FROM "albums" "AlbumEntity" LEFT JOIN "albums_assets_assets" "AlbumEntity_AlbumEntity__AlbumEntity_assets" ON "AlbumEntity_AlbumEntity__AlbumEntity_assets"."albumsId"="AlbumEntity"."id" LEFT JOIN "assets" "AlbumEntity__AlbumEntity_assets" ON "AlbumEntity__AlbumEntity_assets"."id"="AlbumEntity_AlbumEntity__AlbumEntity_assets"."assetsId" AND ("AlbumEntity__AlbumEntity_assets"."deletedAt" IS NULL) WHERE ( ("AlbumEntity"."id" = $1 AND "AlbumEntity__AlbumEntity_assets"."id" = $2) ) AND ( "AlbumEntity"."deletedAt" IS NULL )) LIMIT 1 -- PARAMETERS: ["3fdf0e58-a1c7-4efe-8288-06e4c3f38df9","b666ae6c-afa8-4d6f-a1ad-7091a0659320"]
immich_server | query: SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (SELECT 1 FROM "assets" "AssetEntity" WHERE ("AssetEntity"."id" = $1 AND "AssetEntity"."ownerId" = $2)) LIMIT 1 -- PARAMETERS: ["b666ae6c-afa8-4d6f-a1ad-7091a0659320","6bc60cf1-bd18-4501-a1c2-120b51276fda"]
immich_server | query: SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (SELECT 1 FROM "albums" "AlbumEntity" LEFT JOIN "albums_assets_assets" "AlbumEntity_AlbumEntity__AlbumEntity_assets" ON "AlbumEntity_AlbumEntity__AlbumEntity_assets"."albumsId"="AlbumEntity"."id" LEFT JOIN "assets" "AlbumEntity__AlbumEntity_assets" ON "AlbumEntity__AlbumEntity_assets"."id"="AlbumEntity_AlbumEntity__AlbumEntity_assets"."assetsId" AND ("AlbumEntity__AlbumEntity_assets"."deletedAt" IS NULL) WHERE ( ("AlbumEntity"."id" = $1 AND "AlbumEntity__AlbumEntity_assets"."id" = $2) ) AND ( "AlbumEntity"."deletedAt" IS NULL )) LIMIT 1 -- PARAMETERS: ["3fdf0e58-a1c7-4efe-8288-06e4c3f38df9","c656ab1c-7775-4ff7-b56f-01308c072a76"]
immich_server | query: SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (SELECT 1 FROM "assets" "AssetEntity" WHERE ("AssetEntity"."id" = $1 AND "AssetEntity"."ownerId" = $2)) LIMIT 1 -- PARAMETERS: ["c656ab1c-7775-4ff7-b56f-01308c072a76","6bc60cf1-bd18-4501-a1c2-120b51276fda"]
immich_server | query: SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (SELECT 1 FROM "albums" "AlbumEntity" LEFT JOIN "albums_assets_assets" "AlbumEntity_AlbumEntity__AlbumEntity_assets" ON "AlbumEntity_AlbumEntity__AlbumEntity_assets"."albumsId"="AlbumEntity"."id" LEFT JOIN "assets" "AlbumEntity__AlbumEntity_assets" ON "AlbumEntity__AlbumEntity_assets"."id"="AlbumEntity_AlbumEntity__AlbumEntity_assets"."assetsId" AND ("AlbumEntity__AlbumEntity_assets"."deletedAt" IS NULL) WHERE ( ("AlbumEntity"."id" = $1 AND "AlbumEntity__AlbumEntity_assets"."id" = $2) ) AND ( "AlbumEntity"."deletedAt" IS NULL )) LIMIT 1 -- PARAMETERS: ["3fdf0e58-a1c7-4efe-8288-06e4c3f38df9","cf82adb2-1fcc-4f9e-9013-8fc03cc8d3a9"]
immich_server | query: SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (SELECT 1 FROM "assets" "AssetEntity" WHERE ("AssetEntity"."id" = $1 AND "AssetEntity"."ownerId" = $2)) LIMIT 1 -- PARAMETERS: ["cf82adb2-1fcc-4f9e-9013-8fc03cc8d3a9","6bc60cf1-bd18-4501-a1c2-120b51276fda"]
```
After:
```
// Asset membership check (1 query for all assets)
immich_server | query: SELECT "albums_assets"."assetsId" AS "assetId" FROM "albums_assets_assets" "albums_assets" WHERE "albums_assets"."albumsId" = $1 AND "albums_assets"."assetsId" IN ($2, $3, $4) -- PARAMETERS: ["ca870d76-6311-4e89-bf9a-f5b51ea2452c","b666ae6c-afa8-4d6f-a1ad-7091a0659320","c656ab1c-7775-4ff7-b56f-01308c072a76","cf82adb2-1fcc-4f9e-9013-8fc03cc8d3a9"]
// Permissions check (1 query per asset)
immich_server | query: SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (SELECT 1 FROM "assets" "AssetEntity" WHERE ("AssetEntity"."id" = $1 AND "AssetEntity"."ownerId" = $2)) LIMIT 1 -- PARAMETERS: ["b666ae6c-afa8-4d6f-a1ad-7091a0659320","6bc60cf1-bd18-4501-a1c2-120b51276fda"]
immich_server | query: SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (SELECT 1 FROM "assets" "AssetEntity" WHERE ("AssetEntity"."id" = $1 AND "AssetEntity"."ownerId" = $2)) LIMIT 1 -- PARAMETERS: ["c656ab1c-7775-4ff7-b56f-01308c072a76","6bc60cf1-bd18-4501-a1c2-120b51276fda"]
immich_server | query: SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (SELECT 1 FROM "assets" "AssetEntity" WHERE ("AssetEntity"."id" = $1 AND "AssetEntity"."ownerId" = $2)) LIMIT 1 -- PARAMETERS: ["cf82adb2-1fcc-4f9e-9013-8fc03cc8d3a9","6bc60cf1-bd18-4501-a1c2-120b51276fda"]
```
* refactor: server info model to use data classes instead of dtos
* mobile: add return types and refactor private variables in map / stack
---------
Co-authored-by: shalong-tanwen <139912620+shalong-tanwen@users.noreply.github.com>
* add shared links page
* feat(mobile): shared link items
* feat(mobile): create / edit shared links page
* server: add changeExpiryTime to SharedLinkEditDto
* fix(mobile): edit expiry to never
* mobile: add icon when shares list is empty
* mobile: create new share from album / timeline
* mobile: add translation texts
* mobile: minor ui fixes
* fix: handle serverURL with /api path
* mobile: show share link on successful creation
* mobile: shared links list - 2 column layout
* mobile: use sharedlink pod class instead of dto
* mobile: show error on link creation
* mobile: show share icon only when remote assets are in selection
* mobile: use server endpoint instead of server url
* styling
* styling
---------
Co-authored-by: shalong-tanwen <139912620+shalong-tanwen@users.noreply.github.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
* Added missing alt text on logo in documentation
* Better to use CSS to do the uppercases
Some assistive technology can spell each letter instead of read the word, depends a lot on internals but it's better to prevent...
* Semantic fix - single paragraph instead of many + uppercase via CSS
Single sentence in single paragraph, using css instead
* React requires className instead of class...
1. In the `docker-compose.dev.yml` file, increased ulimits for the containers that use TS code. This was one of the reasons for failures in my Podman (on macOS/arm64) environment. It wasn't the only failure with Podman, and didn't investigate further (I switched to Docker on Linux/amd64 after), but it can still help others.
2. Added a `make dev-down` to perform a `docker-compose down` on the dev environment
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* Add translations for new album sort options
* Support additional album sort options like on web
* Update generated code
* Fix lint
---------
Co-authored-by: Alex <alex.tran1502@gmail.com>
* fix: don't reveal user count publicly
* fix: mobile and user controller
* fix: update other frontend endpoints
* fix: revert openapi change
* chore: open api
* fix: initialize
* openapi
---------
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
* fix: inline mark asset as offline
* fix: improve log message
* chore: lint
* fix: offline asset algorithm
* fix: use set comparison to check what to import
* fix: only mark new offline files as offline
* fix: compare the correct array
* fix: set default library concurrency to 5
* fix: remove one db call when scanning new files
* chore: remove unused import
---------
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
* feat: improve performances in people page
* feat: add loadingspinner when searching
* fix: reset people on error
* fix: case insensitive
* feat: better sql query
* fix: reset people list before api request
* fix: format
* feat: add hover background to image toolbar icons
* not use background color when not set
---------
Co-authored-by: katsadim <athanasios.katsadimas@refurbed.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
* chore: bump node version to 20.8
* chore(server,web) upgrade vulnerable dependencies
* fix: revert node change
* fix: commander version
* fix: set country to null if undefined
* fix: set docs module resolution
* fix(web): correct return type of interval
* fix(docs): set node in tsconfig
---------
Co-authored-by: Jonathan Jogenfors <jonathan@jogenfors.se>
Co-authored-by: debricked[bot] <47180885+debricked[bot]@users.noreply.github.com>
* send store event to page
* fix format
* add new asset to existing bucket
* format
* debouncing
* format
* load bucket
* feedback
* feat: listen to deletes and auto-subscribe on all asset grid pages
* feat: auto refresh on person thumbnail
* chore: skip upload event for now
* fix: person thumbnail event
* fix merge
* update handleAssetDeletion with websocket communication
* update info box on mount
* fix test
* fix test
* feat: event for trash asset
---------
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
* fix: timezone bucket timezones
* chore: open api
* fix: interpret local time in utc
* fix: tests
* fix: refactor memory lane
* fix(web): use local date in memory viewer
* chore: set localDateTime non-null
* fix: filter out memories from the current year
* wip: move localDateTime to asset
* fix: correct sorting from db
* fix: migration
* fix: web typo
* fix: formatting
* fix: e2e
* chore: localDateTime is non-null
* chore: more non-nulliness
* fix: asset stub
* fix: tests
* fix: use extract and index for day of year
* fix: don't show memories before today
* fix: cleanup
* fix: tests
* fix: only use localtime for tz
* fix: display memories in client timezone
* fix: tests
* fix: svelte tests
* fix: bugs
* chore: open api
---------
Co-authored-by: Jonathan Jogenfors <jonathan@jogenfors.se>
* Add include archive setting to map on web
* open api
* better naming for web isArchived variable
* add withArchived setting to mobile
* (e2e): tests for mapMarker endpoint and isArchived
* isArchived to mobile
* chore: cleanup test
* chore: optimize e2e
---------
Co-authored-by: shalong-tanwen <139912620+shalong-tanwen@users.noreply.github.com>
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
* save thumbnails in subdirectories
* migration job, migrate assets and face thumbnails
* fix tests
* directory depth of two instead of three
* cleanup empty dirs after migration
* clean up empty dirs after migration, migrate people without assetId
* add job card for new migration job
* fix removeEmptyDirs race condition because of missing await
* cleanup empty directories after asset deletion
* move ensurePath to storage core
* rename jobs
* remove unnecessary property of IEntityJob
* use updated person getById, minor refactoring
* ensure that directory cleanup doesn't interfere with migration
* better description for job in ui
* fix remove directories when migration is done
* cleanup empty folders at start of migration
* fix: actually persist concurrency setting
* add comment explaining regex
* chore: cleanup
---------
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
* feat(server): get random assets API
* Fix tests
* Use correct validation annotation
* Fix offset use in query
* Update API specs
* Fix typo
* Random assets e2e tests
* Improve e2e tests
* feat(web): suggest people when entering a name
* fix: border size from 2 to 1 pixel
* pr feedback
* fix: web unit test
* pr feedback
---------
Co-authored-by: Alex <alex.tran1502@gmail.com>
* use access core for all person methods
* minor fixes, feedback
* reorder assignments
* remove unnecessary permission requirement
* unify naming of tests
* reorder variables
* soft delete albums when user gets soft deleted
* fix wrong intl openapi version
* fix tests
* ability to restore albums, automatically restore when user restored
* (e2e) tests for shared albums via link and with user
* (e2e) test deletion of users and linked albums
* (e2e) fix share album with owner test
* fix: deletedAt
* chore: fix restore order
* fix: use timezone date column
* chore: cleanup e2e tests
* (e2e) fix user delete test
---------
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
* feat: add setting for minimum face count for face detection
Adds the minimum face count setting to the web interface to
circumvent detection of strangers and random background people
if desired.
* fix: codestyle, remove max for face count
I've committed to this project, and I will not stop. I will keep updating the docs, adding new features, and fixing bugs. But I can't do it alone. So I need your help to give me additional motivation to keep going.
@ -102,10 +107,16 @@ As our hosts in the [selfhosted.show - In the episode 'The-organization-must-not
If you feel like this is the right cause and the app is something you are seeing yourself using for a long time, please consider supporting the project with the option below.
## Donation
### Donation
- [Monthly donation](https://github.com/sponsors/alextran1502) via GitHub Sponsors
- [One-time donation](https://github.com/sponsors/alextran1502?frequency=one-time&sponsor=alextran1502) via GitHub Sponsors
| Fotos & Videos hochladen und ansehen | Ja | Ja |
| Automatisches Backup wenn die App geöffnet ist | Ja | n. a. |
| Selektive Auswahl von Alben zum Sichern | Ja | n. a. |
| Fotos und Videos auf das Gerät herunterladen | Ja | Ja |
| Unterstützt mehrere Benutzer | Ja | Ja |
| Album und geteilte Alben | Ja | Ja |
| Scrollleiste | Ja | Ja |
| Unterstützt RAW Formate | Ja | Ja |
| Metadaten anzeigen (EXIF, Karte) | Ja | Ja |
| Suchen nach Metadaten, Objekten, Gesichtern und CLIP | Ja | Ja |
| Administrative Funktionen (Benutzerverwaltung) | Nein | Ja |
| Backup im Hintergrund | Ja | n. a. |
| Virtuelles Scrollen | Ja | Ja |
| OAuth Unterstützung | Ja | Ja |
| API-Schlüssel | n. a. | Ja |
| LivePhoto/MotionPhoto Backup und Wiedergabe | Ja | Ja |
| Benutzerdefinierte Speicherstruktur | Ja | Ja |
| Öffentliches Teilen | Nein | Ja |
| Archive und Favoriten | Ja | Ja |
| Globale Karte | Ja | Ja |
| Teilen mit Partner | Ja | Ja |
| Gesichtserkennung und Gruppierung | Ja | Ja |
| Rückblicke (heute vor x Jahren) | Ja | Ja |
| Offline Unterstützung | Ja | Nein |
| Schreibgeschützte Gallerie | Ja | Ja |
| Gestapelte Bilder | Ja | Ja |
## Unterstütze das Projekt
Ich habe mich diesem Projekt verpflichtet und werde nicht aufgeben. Ich werde die Dokumentation weiter aktualisieren, neue Funktionen hinzufügen und Fehler beheben. Allerdings kann ich das nicht alleine schaffen. Daher brauche ich Eure Unterstützung, um mir zusätzliche Motivation zu geben, weiterzumachen.
Wie unsere Gastgeber in der [selfhosted.show - In der Episode 'The-organization-must-not-be-name is a Hostile Actor'](https://selfhosted.show/79?t=1418) gesagt haben, ist dies ein riesiges Unterfangen, welchem das Team und ich uns annehmen. In Zukunft würde ich liebend gerne Vollzeit an dem Projekt arbeiten und bitte daher um Eure Unterstützung.
Wenn Du denkst, dass dies die richtige Sache ist und dich selbst die App für eine längere Zeit nutzen siehst, dann denke bitte darüber nach, das Projekt mit einer der unten aufgelisteten Optionen zu unterstützen.
### Spenden
- [Monatliche Spende](https://github.com/sponsors/immich-app) via GitHub Sponsors
- [Einmalige Spende](https://github.com/sponsors/immich-app?frequency=one-time&sponsor=immich-app) via GitHub Sponsors
| Caricamento e visualizzazione di foto e video | Sì | Sì |
| Backup automatico quando l'app è in esecuzione | Sì | N/A |
| Selezione degli album per backup | Sì | N/A |
| Download foto e video sul dispositivo | Sì | Sì |
| Supporto multi utente | Sì | Sì |
| Album e album condivisi | Sì | Sì |
| Barra di scorrimento con trascinamento | Sì | Sì |
| Supporto formati raw | Sì | Sì |
| Visualizzazione metadata (EXIF, map) | Sì | Sì |
| Ricerca per metadata, oggetti, volti e CLIP | Sì | Sì |
| Funzioni di amministrazione degli utenti | No | Sì |
| Backup in background | Sì | N/A |
| Scroll virtuale | Sì | Sì |
| Supporto OAuth | Sì | Sì |
| API Keys | N/A | Sì |
| Backup e riproduzione di LivePhoto | iOS | Sì |
| Archiviazione impostata dall'utente | Sì | Sì |
| Condivisione pubblica | No | Sì |
| Archivio e Preferiti | Sì | Sì |
| Mappa globale | Sì | Sì |
| Collaborazione con utenti | Sì | Sì |
| Riconoscimento facciale e categorizzazione | Sì | Sì |
| Ricordi (x anni fa) | Sì | Sì |
| Supporto offline | Sì | No |
| Galleria sola lettura | Sì | Sì |
# Supporta il progetto
Mi dedico al progetto e non smetterò di farlo. Manterrò aggiornata la documentazione, aggiungerò nuove funzioni e risolverò i bug, ma non posso farlo da solo. Ho bisogno del tuo aiuto che mi da motivazione per continuare.
Come detto dal nostro host [selfhosted.show - Nell'episodio 'The-organization-must-not-be-name is a Hostile Actor'](https://selfhosted.show/79?t=1418), quello che il team ed io stiamo facendo è un lavoro enorme. Mi piacerebbe dedicarmi al progetto full-time e chiedo il tuo aiuto affinchè sia possibile.
Se pensi che Immich sia una buona causa e che l'app sia qualcosa che useresti nel lungo termine, sappi che puoi supportare il progetto scegliendo tra le opzioni sotto elencate.
## Donazioni
- [Donazione mensile](https://github.com/sponsors/alextran1502) tramite GitHub Sponsors
- [Donazione una tantum](https://github.com/sponsors/alextran1502?frequency=one-time&sponsor=alextran1502) tramite GitHub Sponsors
[selfhosted.show - In the episode 'The-organization-must-いいえt-be-name is a Hostile Actor'](https://selfhosted.show/79?t=1418) のホストが言ったように、これはチームと私がやっていることの大規模な事業だ。そしていつの日か、フルタイムでこの仕事ができるようになりたいと思っています。
저는 이 프로젝트에 전념해왔고, 앞으로도 멈추지 않을 것입니다. 문서를 업데이트하고, 새로운 기능을 추가하고, 버그를 수정하려 합니다. 하지만 혼자서는 할 수 없습니다. 계속해서 나아갈 수 있는 추가적인 동기부여를 위해 당신의 도움이 필요합니다.
[selfhosted.show - In the episode 'The-organization-must-not-be-name is a Hostile Actor'](https://selfhosted.show/79?t=1418) 진행자가 말했듯이, 우리가 하고 있는 것은 대규모 프로젝트입니다. 언젠가는 이 일을 풀타임으로 하는 것을 희망하며, 이를 실현하기 위해 당신의 도움이 필요합니다.
만약 이에 동의하거나 이 앱을 장기간 사용하고자 한다면, 아래의 수단을 통해 이 프로젝트를 지원해 주세요.
### 후원
- GitHub 스폰서를 통한 [정기 후원](https://github.com/sponsors/alextran1502)
- GitHub 스폰서를 통한 [일시 후원](https://github.com/sponsors/alextran1502?frequency=one-time&sponsor=alextran1502)
@ -33,8 +33,6 @@ To be concise, Immich can now read in the gallery files, register the path into
- Only new files that are added to the gallery will be detected.
- Deleted and moved files will not be detected.
You can find more information on how to use the feature by reading the documentation [here](/docs/features/read-only-gallery).
## Memory feature
This is considered a fun feature that the team and I wanted to build for so long, but we had to put it off because of the refactoring of the code base. The code base is now in a good enough form to circle back and add more exciting features.
|  | Asset is only available locally and has not yet been backed up |
|  | Asset was uploaded from this device and is now backed up in the cloud/server and still available in original on the device |
### How can I sync an existing directory with Immich's server?
### Can I add my existing photo library?
Immich doesn't have two-way synchronization ([yet](https://github.com/immich-app/immich/discussions/1006)), but the [command line tool](/docs/features/bulk-upload.md) can bulk upload items from a directory to Immich.
### Why doesn't Immich watch an existing photo gallery directory?
The initial approach of Immich is to become a backup tool, primarily for mobile device usage. Thus, all the assets must be uploaded from the mobile client. The app was architectured to perform that job well.
### Why does my uploaded photo show up with the wrong date or time in Immich?
When a photo is initially uploaded Immich uses the create date of the file to determine where it belongs in the timeline. After that, background jobs will run that extract [exif metadata](https://en.wikipedia.org/wiki/Exif), including the CreateDate, to provide a more accurate date for the photo. If that is not available it will fallback to the modified date. If you want to ensure your photo has the right date, check the exif metadata before uploading.
If the timezone is incorrect in an uploaded photo, check the `DateTimeOriginal` exif field of the uploaded file. Immich uses the very competent library [exiftool-vendored.js](https://github.com/photostructure/exiftool-vendored.js#dates) to handle timezone parsing, but in some cases (like photos taken with DSLR cameras) it has to fallback on the local timezone. If you are using docker, this fallback will be UTC. (Note that even the photo backup app that can't be named [has the same bug!](https://photo.stackexchange.com/a/126978)) In Immich, it is possible to change this assumed fallback timezone system-wide by setting the timezone in the microservices docker container. You might need to run the "Extract Metadata" job after to effect the change.
As an example, the following modification of `docker-compose.yml` will set the timezone of the microservices container to be `Europe/Stockholm`
Yes, with an [external library](/docs/features/libraries.md).
### Why are only photos and not videos being uploaded to Immich?
This often happens when using a reverse proxy or cloudflare tunnel in front of Immich. Make sure to set your reverse proxy to allow large POST requests. In `nginx`, set `client_max_body_size 50000M;` or similar. Cloudflare tunnels are limited to 100 mb file sizes.
This often happens when using a reverse proxy or cloudflare tunnel in front of Immich. Make sure to set your reverse proxy to allow large POST requests. In `nginx`, set `client_max_body_size 50000M;` or similar. Cloudflare tunnels are limited to 100 mb file sizes. Also check the disk space of your reverse proxy, in some cases proxies caches requests to disk before passing them on, and if disk space runs out the request fails.
### Why is Immich slow on low-memory systems like the Raspberry Pi?
docker-compose up -d # Start remainder of Immich apps
dockercompose up -d # Start remainder of Immich apps
```
Note that for the database restore to proceed properly, it requires a completely fresh install (i.e. the Immich server has never run since creating the Docker containers). If the Immich app has run, Postgres conflicts may be encountered upon database restoration (relation already exists, violated foreign key constraints, multiple primary keys, etc.).
When deploying Immich it is important to understand that a reverse proxy is required in front of the server and web container. The reverse proxy acts as an intermediary between the user and container, forwarding requests to the correct container based on the URL path.
## Default Reverse Proxy
Immich provides a default nginx reverse proxy preconfigured to perform the correct routing and set the necessary headers for the server and web container to use. These headers are crucial to redirect to the correct URL and determine the client's IP address.
## Using a Different Reverse Proxy
While the reverse proxy provided by Immich works well for basic deployments, some users may want to use a different reverse proxy. Fortunately, Immich is flexible enough to accommodate different reverse proxies. Users can either:
1. Add another reverse proxy on top of Immich's reverse proxy
2. Completely replace the default reverse proxy
## Adding a Custom Reverse Proxy
Users can deploy a custom reverse proxy that forwards requests to Immich's reverse proxy. This way, the new reverse proxy can handle TLS termination, load balancing, or other advanced features, while still delegating routing decisions to Immich's reverse proxy. All reverse proxies between Immich and the user must forward all headers and set the `Host`, `X-Forwarded-Host`, `X-Forwarded-Proto` and `X-Forwarded-For` headers to their appropriate values. Additionally, your reverse proxy should allow for big enough uploads. By following these practices, you ensure that all custom reverse proxies are fully compatible with Immich.
Users can deploy a custom reverse proxy that forwards requests to Immich. This way, the reverse proxy can handle TLS termination, load balancing, or other advanced features. All reverse proxies between Immich and the user must forward all headers and set the `Host`, `X-Forwarded-Host`, `X-Forwarded-Proto` and `X-Forwarded-For` headers to their appropriate values. Additionally, your reverse proxy should allow for big enough uploads. By following these practices, you ensure that all custom reverse proxies are fully compatible with Immich.
### Nginx example config
Below is an example config for nginx:
Below is an example config for nginx. Make sure to include `client_max_body_size 50000M;` also in a `http` block in `/etc/nginx/nginx.conf`.
```nginx
server {
@ -43,7 +28,3 @@ server {
}
}
```
## Replacing the Default Reverse Proxy
Replacing Immich's default reverse proxy is an advanced deployment and support may be limited. When replacing Immich's default proxy it is important to ensure that requests to `/api/*` are routed to the server container and all other requests to the web container. Additionally, the previously mentioned headers should be configured accordingly. You may find our [nginx configuration file](https://github.com/immich-app/immich/blob/main/nginx/templates/default.conf.template) a helpful reference.
@ -34,7 +34,7 @@ The web app is a [TypeScript](https://www.typescriptlang.org/) project that uses
### CLI
The CLI is a [TypeScript](https://www.typescriptlang.org/) project that parses command line arguments to programmatically upload/import assets to an Immich server. See [Bulk Upload](/docs/features/bulk-upload.md) for more information about its usage.
The Immich CLI is an [npm](https://www.npmjs.com/) package that lets users control their Immich instance from the command line. It uses the API to perform various tasks, especially uploading assets. See the [CLI documentation](/docs/features/command-line-interface.md) for more information.
@ -9,6 +9,6 @@ npm run typeorm:migrations:generate ./src/infra/<migration-name>
```
2. Check if the migration file makes sense.
3. Move the migration file to folder `./src/infra/database/migrations` in your code editor.
3. Move the migration file to folder `./server/src/infra/migrations` in your code editor.
The server will automatically detect `*.ts` file changes and restart. Part of the server start-up process includes running any new migrations, so it will be applied immediately.
@ -4,7 +4,7 @@ Immich uses the [OpenAPI](https://swagger.io/specification/) standard to generat
## Generator
OpenAPI is used to generate the client (Typescript, Dart) SDK. `openapi-generator-cli` can be installed [here](https://openapi-generator.tech/docs/installation/). The generated SDK is based on the `immich-openapi-specs.json` file, which is autogenerated by the server when running in development mode. The `immich-openapi-specs.json` file can be modified with `@nestjs/swagger` decorators used or referenced by controller endpoints. See the [NestJS OpenAPI docs](https://docs.nestjs.com/openapi/types-and-parameters) for more info. When you add a new endpoint or modify an existing one, you must run the command below to update the client SDK.
OpenAPI is used to generate the client (Typescript, Dart) SDK. `openapi-generator-cli` can be installed [here](https://openapi-generator.tech/docs/installation/). The generated SDK is based on the `immich-openapi-specs.json` file, which is autogenerated by the server **when running in development mode**. The `immich-openapi-specs.json` file can be modified with `@nestjs/swagger` decorators used or referenced by controller endpoints. See the [NestJS OpenAPI docs](https://docs.nestjs.com/openapi/types-and-parameters) for more info. When you add a new endpoint or modify an existing one, you must run the server in development mode and run the command below to update the client SDK.
```bash
npm run api:generate # Run from the `server/` directory
@ -52,7 +52,7 @@ If you only want to do web development connected to an existing, remote backend,
3. Start the web development server
```
PUBLIC_IMMICH_SERVER_URL=https://demo.immich.app/api npm run dev
IMMICH_SERVER_URL=https://demo.immich.app/api npm run dev
```
## IDE setup
@ -61,9 +61,15 @@ PUBLIC_IMMICH_SERVER_URL=https://demo.immich.app/api npm run dev
Setting these in the IDE give a better developer experience, auto-formatting code on save, and providing instant feedback on lint issues.
### Dart Code Metris
The mobile app uses DCM (Dart Code Metrics) for linting and metrics calculation. Please refer to the [Getting Started](https://dcm.dev/docs/getting-started/#installation) page for more information on setting up DCM
Note: Activating the license is not required.
### VSCode
Install `Flutter`, `Prettier`, `ESLint` and `Svelte` extensions.
Install `Flutter`,`DCM`,`Prettier`, `ESLint` and `Svelte` extensions.
in User `settings.json` (`cmd + shift + p` and search for `Open User Settings JSON`) add the following:
Unit are run by calling `npm run test` from the `server` directory.
### End to end tests
The backend has an end-to-end test suite that can be called with `npm run test:e2e` from the `server` directory. This will set up a dummy database inside a temporary container and run the tests against it. Setup and teardown is automatically taken care of. That test, however, can not set up all prerequisites to parse file formats, as that is very complex and error-prone. As such, this test excludes some test cases like HEIC file imports. The test suite will also print a friendly warning to remind you that not all tests are being run.
Note that there is a bug in nodejs <20.8thatcausessegmentationfaultswhenrunningthesetests.Ifyourunintosegfaults,ensureyouareusingatleastversion20.8.
To perform a full e2e test, you need to run e2e tests inside docker. The easiest way to do that is to run `make test-e2e` in the root directory. This will build and start a docker-compose consisting of the server, microservices, and a postgres database. It will then perform the tests and exit.
If you manually install the dependencies (see the DOCKERFILE) on your development machine, you can also run the full e2e tests manually by setting the `IMMICH_RUN_ALL_TESTS` environment value to true, i.e. `IMMICH_RUN_ALL_TESTS=true npm run test:e2e`.
A great option to get assistance with troubleshooting is to join our [Discord](https://discord.gg/D8JsnBEuKb) server, where we have a dedicated channel for `#contributing`.
:::
## Known Issues
### Running on Windows
Running Immich on Windows can be frustrating and there are lots of ways it can go wrong. Where possible we recommend using Docker on Linux. However, several people have had success running Immich on Windows using Docker via WSL2.
### NTFS Mounted Volumes
The docker-compose.dev.yml and docker-compose.prod.yml use volume mounts for the postgres database. On start-up, postgres will try to `chown` the data directory, but fail. See [this post](https://forums.docker.com/t/data-directory-var-lib-postgresql-data-pgdata-has-wrong-ownership/17963/24) for more information about this issue and possible solutions.
If you are running the CLI container on the same machine as your Immich server, you may not be able to reach the external address. In that case, try the following steps:
1. Find the internal Docker network used by Immich via `docker network ls`.
2. Adapt the above command to pass the `--network <immich_network>` argument to `docker run`, substituting `<immich_network>` with the result from step 1.
3. Use `--server http://immich-server:3001` for the upload command instead of the external address.
The `immich-server` and `immich-microservices` containers must be able to access the files, or directories at the path referenced in the command. The directories referenced must be set under a user's `External Path` setting. More detailed instructions can be found [here](/docs/features/read-only-gallery).
:::tip Matching volume references
The import command is most easily run on the machine running the immich service, as the path to the files on the machine running the command and the server much match identically.
If you are running immich within docker, the volume pointing to your existing library should be identical with your host machine.
The proper command for above would be as shown below. You should have access to `/path/to/media` exactly on the environment the CLI command is being run on
If you are running the import using the docker command, please note that the volumes should point to the `/path/to/media` exactly on the environment the CLI command is being run on
Immich has a CLI that allows you to perform certain actions from the command line. This CLI replaces the [legacy CLI](https://github.com/immich-app/CLI) that was previously available. The CLI is hosted in the [cli folder of the the main Immich github repository](https://github.com/immich-app/immich/tree/main/cli).
## Features
- Upload photos and videos to Immich
- Check server version
More features are planned for the future.
:::tip Google Photos Takeout
If you are looking to import your Google Photos takeout, we recommed this community maintained tool [immich-go](https://github.com/simulot/immich-go)
:::
## Requirements
- Node.js 20.0 or above
- Npm
## Installation
```bash
npm i -g @immich/cli
```
NOTE: if you previously installed the legacy CLI, you will need to uninstall it first:
```bash
npm uninstall -g immich
```
## Usage
```
immich
```
```
Usage: immich [options] [command]
Immich command line interface
Options:
-h, --help display help for command
Commands:
upload [options] [paths...] Upload assets
server-info Display server information
login-key [instanceUrl] [apiKey] Login using an API key
logout Remove stored credentials
help [command] display help for command
```
## Commands
The upload command supports the following options:
This will store your credentials in a file in your home directory. Please keep the file secure, either by performing the logout command after you are done, or deleting it manually.
Once you are authenticated, you can upload assets to your Immich server.
```bash
immich upload file1.jpg file2.jpg
```
By default, subfolders are not included. To upload a directory including subfolder, use the --recursive option:
```bash
immich upload --recursive directory/
```
If you are unsure what will happen, you can use the `--dry-run` option to see what would happen without actually performing any actions.
```bash
immich upload --dry-run --recursive directory/
```
By default, the upload command will hash the files before uploading them. This is to avoid uploading the same file multiple times. If you are sure that the files are unique, you can skip this step by passing the `--skip-hash` option. Note that Immich always performs its own deduplication through hashing, so this is merely a performance consideration. If you have good bandwidth it might be faster to skip hashing.
```bash
immich upload --skip-hash --recursive directory/
```
You can automatically create albums based on the folder name by passing the `--album` option. This will automatically create albums for each uploaded asset based on the name of the folder they are in.
```bash
immich upload --album --recursive directory/
```
It is possible to skip assets matching a glob pattern by passing the `--ignore` option. See [the library documentation](docs/features/libraries.md) on how to use glob patterns. You can add several exclusion patterns if needed.
Immich supports the creation of libraries which is a top-level asset container. Currently, there are two types of libraries: traditional upload libraries that can sync with a mobile device, and external libraries, that keeps up to date with files on disk. Libraries are different from albums in that an asset can belong to multiple albums but only one library, and deleting a library deletes all assets contained within. As of August 2023, this is a new feature and libraries have a lot of potential for future development beyond what is documented here. This document attempts to describe the current state of libraries.
## The Upload Library
Immich comes preconfigured with an upload library for each user. All assets uploaded to Immich are added to this library. This library can be renamed, but not deleted. The upload library is the only library that can be synced with a mobile device. No items in an upload library is allowed to have the same sha1 hash as another item in the same library in order to prevent duplicates.
## External Libraries
External libraries tracks assets stored outside of immich, i.e. in the file system. Immich will only read data from the files, and will not modify them in any way. Therefore, the delete button is disabled for external assets. When the external library is scanned, immich will read the metadata from the file and create an asset in the library for each image or video file. These items will then be shown in the main timeline, and they will look and behave like any other asset, including viewing on the map, adding to albums, etc.
If a file is modified outside of Immich, the changes will not be reflected in immich until the library is scanned again. There are different ways to scan a library depending on the use case:
- Scan Library Files: This is the default scan method and also the quickest. It will scan all files in the library and add new files to the library. It will notice if any files are missing (see below) but not check existing assets
- Scan All Library Files: Same as above, but will check each existing asset to see if the modification time has changed. If it has, the asset will be updated. Since it has to check each asset, this is slower than Scan Library Files.
- Force Scan All Library Files: Same as above, but will read each asset from disk no matter the modification time. This is useful in some cases where an asset has been modified externally but the modification time has not changed. This is the slowest way to scan because it reads each asset from disk.
:::caution
Due to aggressive caching it can take some time for a refreshed asset to appear correctly in the web view. You need to clear the cache in your browser to see the changes. This is a known issue and will be fixed in a future release. In Chrome, you need to open the developer console with F12, then reload the page with F5, and finally right click on the reload button and select "Empty Cache and Hard Reload".
:::
In external libraries, the file path is used for duplicate detection. This means that if a file is moved to a different location, it will be added as a new asset. If the file is moved back to its original location, it will be added as a new asset. In contrast to upload libraries, two identical files can be uploaded if they are in different locations. This is a deliberate design choice to make Immich reflect the file system as closely as possible. Remember that duplication detection is only done within the same library, so if you have multiple external libraries, the same file can be added to multiple libraries.
:::caution
If you add assets from an external library to an album and then move the asset to another location within the library, the asset will be removed from the album upon rescan. This is because the asset is considered a new asset after the move. This is a known issue and will be fixed in a future release.
:::
### Deleted External Assets
In all above scan methods, Immich will check if any files are missing. This can happen if files are deleted, or if they are on a storage location that is currently unavailable, like a network drive that is not mounted, or a USB drive that has been unplugged. In order to prevent accidental deletion of assets, Immich will not immediately delete an asset from the library if the file is missing. Instead, the asset will be internally marked as offline and will still be visible in the main timeline. If the file is moved back to its original location and the library is scanned again, the asset will be restored.
Finally, files can be deleted from Immich via the `Remove Offline Files` job. Any assets marked as offline will then be removed from Immich. Run this job whenever files have been deleted from the file system and you want to remove them from Immich. Note that a library scan must be performed first to mark the assets as offline.
### Import Paths
External libraries use import paths to determine which files to scan. Each library can have multiple import paths so that files from different locations can be added to the same library. Import paths are scanned recursively, and if a file is in multiple import paths, it will only be added once. If the import paths are edited in a way that an external file is no longer in any import path, it will be removed from the library in the same way a deleted file would. If the file is moved back to an import path, it will be added again as if it was a new file.
### Troubleshooting
Sometimes, an external library will not scan correctly. This can happen if the immich_server or immich_microservices can't access the files. Here are some things to check:
- Is the external path set correctly?
- In the docker-compose file, are the volumes mounted correctly?
- Are the volumes identical between the `server` and `microservices` container?
- Are the import paths set correctly, and do they match the path set in docker-compose file?
- Are the permissions set correctly?
If all else fails, you can always start a shell inside the container and check if the path is accessible. For example, `docker exec -it immich_microservices /bin/bash` will start a bash shell. If your import path, for instance, is `/data/import/photos`, you can check if the files are accessible by running `ls /data/import/photos`. Also check the `immich_server` container in the same way.
### Security Considerations
:::caution
Please read and understand this section before setting external paths, as there are important security considerations.
:::
For security purposes, each Immich user is disallowed to add external files by default. This is to prevent devastating [path traversal attacks](https://owasp.org/www-community/attacks/Path_Traversal). An admin can allow individual users to use external path feature via the `external path` setting found in the admin panel. Without the external path restriction, a user can add any image or video file on the Immich host filesystem to be imported into Immich, potentially allowing sensitive data to be accessed. If you are running Immich as root in your Docker setup (which is the default), all external file reads are done with root privileges. This is particularly dangerous if the Immich host is a shared server.
With the `external path` set, a user is restricted to accessing external files to files or directories within that path. The Immich admin should still be careful not set the external path too generously. For example, `user1` wants to read their photos in to `/home/user1`. A lazy admin sets that user's external path to `/home/` since it "gets the job done". However, that user will then be able to read all photos in `/home/user2/private-photos`, too! Please set the external path as specific as possible. If multiple folders must be added, do this using the docker volume mount feature described below.
### Exclusion Patterns and Scan Settings
By default, all files in the import paths will be added to the library. If there are files that should not be added, exclusion patterns can be used to exclude them. Exclusion patterns are glob patterns are matched against the full file path. If a file matches an exclusion pattern, it will not be added to the library. Exclusion patterns can be added in the Scan Settings page for each library. Under the hood, Immich uses the [glob](https://www.npmjs.com/package/glob) package to match patterns, so please refer to [their documentation](https://github.com/isaacs/node-glob#glob-primer) to see what patterns are supported.
Some basic examples:
- `*.tif` will exclude all files with the extension `.tif`
- `hidden.jpg` will exclude all files named `hidden.jpg`
- `**/Raw/**` will exclude all files in any directory named `Raw`
- `*.{tif,jpg}` will exclude all files with the extension `.tif` or `.jpg`
### Nightly job
There is an automatic job that's run once a day and refreshes all modified files in all libraries as well as cleans up any libraries stuck in deletion.
## Usage
Let's show a concrete example where we add an existing gallery to Immich. Here, we have the following folders we want to add:
- `/home/user/old-pics`: a folder containing childhood photos.
- `/mnt/nas/christmas-trip`: photos from a christmas trip. The subfolder `/mnt/nas/christmas-trip/Raw` contains the raw files directly from the DSLR. We don't want to import the raw files to Immich
- `/mnt/media/videos`: Videos from the same christmas trip.
First, we need to plan how we want to organize the libraries. The christmas trip photos should belong to its own library since we want to exclude the raw files. The videos and old photos can be in the same library since we want to import all files. We could also add all three folders to the same library if there are no files matching the Raw exclusion pattern in the other folders.
### Mount Docker Volumes
`immich-server` and `immich-microservices` containers will need access to the gallery. Modify your docker compose file as follows
The `ro` flag at the end only gives read-only access to the volumes. While Immich does not modify files, it's a good practice to mount read-only.
:::
_Remember to bring the container down/up to register the changes. Make sure you can see the mounted path in the container._
### Set External Path
Only an admin can do this.
- Navigate to `Administration > Users` page on the web.
- Click on the user edit button.
- Set `/mnt/media` to be the external path. This folder will only contain the three folders that we want to import, so nothing else can be accessed.
### Create External Libraries
- Click on your user name in the top right corner -> Account Settings
- Click on Libraries
- Click on Create External Library
- Click the drop-down menu on the newly created library
- Click on Rename Library and rename it to "Christmas Trip"
- Click Edit Import Paths
- Click on Add Path
- Enter `/mnt/media/christmas-trip` then click Add
NOTE: We have to use the `/mnt/media/christmas-trip` path and not the `/mnt/nas/christmas-trip` path since all paths have to be what the Docker containers see.
Next, we'll add an exclusion pattern to filter out raw files.
- Click the drop-down menu on the newly christmas library
- Click on Manage
- Click on Scan Settings
- Click on Add Exclusion Pattern
- Enter `**/Raw/**` and click save.
- Click save
- Click the drop-down menu on the newly created library
- Click on Scan Library Files
The christmas trip library will now be scanned in the background. In the meantime, let's add the videos and old photos to another library.
- Click on Create External Library.
:::info Note
If you get an error here, please rename the other external library to something else. This is a bug that will be fixed in a future release.
:::
- Click the drop-down menu on the newly created library
- Click Edit Import Paths
- Click on Add Path
- Enter `/mnt/media/old-pics` then click Add
- Click on Add Path
- Enter `/mnt/media/videos` then click Add
- Click Save
- Click on Scan Library Files
Within seconds, the assets from the old-pics and videos folders should show up in the main timeline.
Keep in mind that mucking around in the database might set the moon on fire. Avoid modifying the database directly when possible, and always have current backups.
:::
:::tip
Run `docker exec -it immich_postgres psql immich <DB_USERNAME>` to connect to the database via the container directly.
(Replace `<DB_USERNAME>` wit the value from your [`.env` file](/docs/install/environment-variables#database)).
:::
## Assets
:::note
The `"originalFileName"` column is the name of the uploaded file _without_ the extension.
:::
```sql title="Find by original filename"
SELECT * FROM "assets" WHERE "originalFileName" = 'PXL_20230903_232542848';
SELECT * FROM "assets" WHERE "originalFileName" LIKE 'PXL_%'; -- all files starting with PXL_
SELECT * FROM "assets" WHERE "originalFileName" LIKE '%_2023_%'; -- all files with _2023_ in the middle
```
```sql title="Find by path"
SELECT * FROM "assets" WHERE "originalPath" = 'upload/library/admin/2023/2023-09-03/PXL_20230903_232542848.jpg';
SELECT * FROM "assets" WHERE "originalPath" LIKE 'upload/library/admin/2023/%';
```
```sql title="Find by checksum" (sha1)
SELECT encode("checksum", 'hex') FROM "assets";
SELECT * FROM "assets" WHERE "checksum" = decode('69de19c87658c4c15d9cacb9967b8e033bf74dd1', 'hex');
```
```sql title="Live photos"
SELECT * FROM "assets" where "livePhotoVideoId" IS NOT NULL;
```
```sql title="Without metadata"
SELECT "assets".* FROM "exif" LEFT JOIN "assets" ON "assets"."id" = "exif"."assetId" WHERE "exif"."assetId" IS NULL;
```
```sql title="Without thumbnails"
SELECT * FROM "assets" WHERE "assets"."resizePath" IS NULL OR "assets"."webpPath" IS NULL;
```
```sql title="By type"
SELECT * FROM "assets" WHERE "assets"."type" = 'VIDEO';
SELECT * FROM "assets" WHERE "assets"."type" = 'IMAGE';
```
```sql title="Count by type"
SELECT "assets"."type", count(*) FROM "assets" GROUP BY "assets"."type";
```
```sql title="Count by type (per user)"
SELECT
"users"."email", "assets"."type", COUNT(*)
FROM
"assets"
JOIN
"users" ON "assets"."ownerId" = "users"."id"
GROUP BY
"assets"."type", "users"."email"
ORDER BY
"users"."email";
```
```sql title="Failed file movements"
SELECT * FROM "move_history";
```
## Users
```sql title="List"
SELECT * FROM "users";
```
## System Config
```sql title="Custom settings"
SELECT "key", "value" FROM "system_config";
```
(Only used when not using the [config file](/docs/install/config-file))
# Hosting the machine-learning service on another system
# Remote Machine Learning
To alleviate [performance issues on low-memory systems](/docs/FAQ.md#why-is-immich-slow-on-low-memory-systems-like-the-raspberry-pi) like the Raspberry Pi, you may also host Immich's machine-learning container on a more powerful system (e.g. your laptop or desktop computer):
- Set `IMMICH_MACHINE_LEARNING_URL` to point to the designated ML system, e.g. `http://workstation:3003`.
- Set the URL in Machine Learning Settings on the Admin Settings page to point to the designated ML system, e.g. `http://workstation:3003`.
- Copy the following `docker-compose.yml` to your ML system.
- Start the container by running `docker-compose up -d` or `docker compose up -d` (depending on your Docker version).
This page gives a few pointers on how to access your Immich instance from outside your LAN.
:::danger
Never forward port 2283 directly to the internet without additional configuration. This will expose the web interface via http to the internet, making you succeptible to [man in the middle](https://en.wikipedia.org/wiki/Man-in-the-middle_attack) attacks.
:::
## Option 1: VPN to home network
You may use a VPN service to open an encrypted connection to your Immich instance. OpenVPN and Wireguard are two popular VPN solutions. Here is a guide on setting up VPN access to your server - [Pihole documentation](https://docs.pi-hole.net/guides/vpn/wireguard/overview/)
### Pros:
- Simple to set up and very secure.
- Single point of potential failure, i.e., the VPN software itself. Even if there is a zero-day vulnerability on Immich, you will not be at risk.
- Both Wireguard and OpenVPN are independently security-audited, so the risk of serious zero-day exploits are minimal.
### Cons:
- If you don't have a static IP address, you would need to set up a [Dynamic DNS](https://www.cloudflare.com/learning/dns/glossary/dynamic-dns/). [DuckDNS](https://www.duckdns.org/) is a free DDNS provider.
- VPN software needs to be installed and active on both server-side and client-side.
- Requires you to open a port on your router to your server.
## Option 2: Tailscale
If you are unable to open a port on your router for Wireguard or OpenVPN to your server, [Tailscale](https://tailscale.com/) is a good option. Tailscale mediates a peer-to-peer wireguard tunnel between your server and remote device, even if one or both of them are behind a [NAT firewall](https://en.wikipedia.org/wiki/Network_address_translation).
### Pros
- Minimal configuration needed on server and client sides.
- You are protected against zero-day vulnerabilities on Immich.
### Cons
- The Tailscale client usually needs to run as root on your devices and it increases the attack surface slightly compared to a minimal Wireguard server. e.g., an [RCE vulnerability](https://github.com/tailscale/tailscale/security/advisories/GHSA-vqp6-rc3h-83cp) was discovered in the Windows Tailscale client in November 2022.
- Tailscale is a paid service. However, there is a generous [free tier](https://tailscale.com/pricing/) that permits up to 3 users and up to 100 devices.
- Tailscale needs to be installed and running on both server-side and client-side.
## Option 3: Reverse Proxy
A reverse proxy is a service that sits between web servers and clients. A reverse proxy can either be hosted on the server itself or remotely. Clients can connect to the reverse proxy via https, and the proxy relays data to Immich. This setup makes most sense if you have your own domain and want to access your Immich instance just like any other website, from outside your LAN. You can also use a DDNS provider like DuckDNS or no-ip if you don't have a domain. This configuration allows the Immich Android and iphone apps to connect to your server without a VPN or tailscale app on the client side.
If you're hosting your own reverse proxy, [Nginx](https://docs.nginx.com/nginx/admin-guide/web-server/reverse-proxy/) is a great option. An example configuration for Nginx is provided [here](https://immich.app/docs/administration/reverse-proxy).
You'll also need your own certificate to authenticate https connections. If you're making Immich publicly accesible, [Let's Encrypt](https://letsencrypt.org/) can provide a free certificate for your domain and is the recommended option. Alternatively, a [self-signed certificate](https://en.wikipedia.org/wiki/Self-signed_certificate) allows you to encrypt your connection to Immich, but it raises a security warning on the client's browser.
A remote reverse proxy like [Cloudflare](https://www.cloudflare.com/learning/cdn/glossary/reverse-proxy/) increases security by hiding the server IP address, which makes targeted attacks like [DDoS](https://www.cloudflare.com/learning/ddos/what-is-a-ddos-attack/) harder.
### Pros
- No additional software needs to be installed client-side
- If you only need access to the web interface remotely, it is possible to set up access controls that shield you from zero-day vulnerabilities on Immich. [Cloudflare Access](https://www.cloudflare.com/zero-trust/products/access/) has a generous free tier.
### Cons
- Complex configuration
- Depending on your configuration, both the Immich web interface and API may be exposed to the internet. Immich is under very active developement and the existence of severe security vulnerabilities cannot be ruled out.
# This is an advanced feature for users who may be running their immich services on different hosts.
# It will not change which address or port that services bind to within their containers, but it will change where other services look for their peers.
# Note: immich-microservices is bound to 3002, but no references are made
From the directory you created in Step 1, (which should now contain your customized `docker-compose.yml` and `.env` files) run `docker-compose up -d`.
```bash title="Start the containers using docker compose command"
docker-compose up -d # or `docker compose up -d` based on your docker-compose version
docker compose up -d
```
:::tip
@ -184,7 +162,7 @@ If `IMMICH_VERSION` is set, it will need to be updated to the latest or desired
When a new version of Immich is [released](https://github.com/immich-app/immich/releases), the application can be upgraded with the following commands, run in the directory with the `docker-compose.yml` file:
```bash title="Upgrade Immich"
docker-compose pull && docker-compose up -d # Or `docker compose up -d`
| `MACHINE_LEARNING_MODEL_TTL`<sup>\*1</sup> | Inactivity time (s) before a model is unloaded (disabled if <= 0) | `0` | machine learning |
| `MACHINE_LEARNING_MODEL_TTL` | Inactivity time (s) before a model is unloaded (disabled if <= 0) | `300` | machine learning |
| `MACHINE_LEARNING_MODEL_TTL_POLL_S` | Interval (s) between checks for the model TTL (disabled if <= 0) | `10` | machine learning |
| `MACHINE_LEARNING_CACHE_FOLDER` | Directory where models are downloaded | `/cache` | machine learning |
| `MACHINE_LEARNING_REQUEST_THREADS`<sup>\*2</sup> | Thread count of the request thread pool (disabled if <= 0) | number of CPU cores | machine learning |
| `MACHINE_LEARNING_REQUEST_THREADS`<sup>\*1</sup> | Thread count of the request thread pool (disabled if <= 0) | number of CPU cores | machine learning |
| `MACHINE_LEARNING_MODEL_INTER_OP_THREADS` | Number of parallel model operations | `1` | machine learning |
| `MACHINE_LEARNING_MODEL_INTRA_OP_THREADS` | Number of threads for each model operation | `2` | machine learning |
| `MACHINE_LEARNING_WORKERS`<sup>\*3</sup> | Number of worker processes to spawn | `1` | machine learning |
| `MACHINE_LEARNING_WORKERS`<sup>\*2</sup> | Number of worker processes to spawn | `1` | machine learning |
| `MACHINE_LEARNING_WORKER_TIMEOUT` | Maximum time (s) of unresponsiveness before a worker is killed | `120` | machine learning |
\*1: This is an experimental feature. It may result in increased memory use over time when loading models repeatedly.
\*1: It is recommended to begin with this parameter when changing the concurrency levels of the machine learning service and then tune the other ones.
\*2: It is recommended to begin with this parameter when changing the concurrency levels of the machine learning service and then tune the other ones.
\*3: Since each process duplicates models in memory, changing this is not recommended unless you have abundant memory to go around.
\*2: Since each process duplicates models in memory, changing this is not recommended unless you have abundant memory to go around.
