* just check file extension for XMP instead of mimetype
* use path to get extension instead of regex
* single quotes
* remove unused import
---------
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
* Added read-only flag for assets, endpoint to trigger file import vs upload
* updated fixtures with new property
* if upload is 'read-only', ensure there is no existing asset at the designated originalPath
* added test for file import as well as detecting existing image at read-only destination location
* Added storage service test for a case where it should not move read-only assets
* upload doesn't need the read-only flag available, just importing
* default isReadOnly on import endpoint to true
* formatting fixes
* create-asset dto needs isReadOnly, so set it to false by default on create, updated api generation
* updated code to reflect changes in MR
* fixed read stream promise return type
* new index for originalPath, check for existing path on import, reglardless of user, to prevent duplicates
* refactor: import asset
* chore: open api
* chore: tests
* Added externalPath support for individual users, updated UI to allow this to be set by admin
* added missing var for externalPath in ui
* chore: open api
* fix: compilation issues
* fix: server test
* built api, fixed user-response dto to include externalPath
* reverted accidental commit
* bad commit of duplicate externalPath in user response dto
* fixed tests to include externalPath on expected result
* fix: unit tests
* centralized supported filetypes, perform file type checking of asset and sidecar during file import process
* centralized supported filetype check method to keep regex DRY
* fixed typo
* combined migrations into one
* update api
* Removed externalPath from shared-link code, added column to admin user page whether external paths / import is enabled or not
* update mimetype
* Fixed detect correct mimetype
* revert asset-upload config
* reverted domain.constant
* refactor
* fix mime-type issue
* fix format
---------
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
The NGINX gzip module does not compress responses with a status of 201, which is
a major issue specifically for the /api/asset/time-bucket endpoint where
responses can be upwards of 5Mi. The size of the response is dramatically
reduced with gzip to 500Ki in some cases.
https://trac.nginx.org/nginx/ticket/471https://trac.nginx.org/nginx/ticket/394
The signature of these endpoints should be GET rather than POST anyway, but that
is a bigger discussion.
* Allow upload of AVIF and x-canon-cr2 mime types
* Allow generic RAW file mime type image/x-dcraw
* Another place to uploading avif and cr2
* Determine mime type for .avif and .cr2 files correctly
* Update asset-upload.config.spec.ts for CR2 and AVIF files
* More changes for AVIF & CR2 files
Found some other places where avif and cr2 should be mentioned.
* Merge in upstream changes
* Allow uploading and using most of the formats that libraw supports
* Add raw files to allowable mobile uploads
* Update asset-upload.config.spec.ts
Fix errant commas.
* Update asset-utils.ts
Remove duplicate entry in hash table.
* Fix missing k25 mime type in server upload check.
Fix prettier formatting message in web file-uploader.
* fix test
---------
Co-authored-by: Elliot Lee <sopwith@gmail.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
* using pydantic BaseSetting
* ML API takes image file as input
* keeping image in memory
* reducing duplicate code
* using bytes instead of UploadFile & other small code improvements
* removed form-multipart, using HTTP body
* format code
---------
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
* add thumbhash: server generation and web impl
* move logic to infra & use byta in db
* remove unnecesary logs
* update generated API and simplify thumbhash gen
* fix check errors
* removed unnecessary library and css tag
* style edits
* syntax mistake
* update server test, change thumbhash job name
* fix tests
* Update server/src/domain/asset/response-dto/asset-response.dto.ts
Co-authored-by: Thomas <9749173+uhthomas@users.noreply.github.com>
* add unit test, change migration date
* change to official thumbhash impl
* update call method to not use eval
* "generate missing" looks for thumbhash
* improve queue & improve syntax
* update syntax again
* update tests
* fix thumbhash generation
* consolidate queueing to avoid duplication
* cover all types of incorrect thumbnail cases
* split out jest tasks
* put back thumbnail duration loading for images without thumbhash
* Remove stray package.json
---------
Co-authored-by: Luke McCarthy <mail@lukehmcc.com>
Co-authored-by: Thomas <9749173+uhthomas@users.noreply.github.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
Given #2668 introduced support for imagemagick and libraw, this should no
longer be necessary which allow for reduced code footprint and complexity.
Fixes: #2744
* use imagemagick and libraw for raw image support
imagemagick and libraw have generally good support for raw images, including
Sony's ARW format. These tools should also allow Immich to support many more
image formats in future without any major code changes.
https://www.libraw.org/supported-cameras
I've tested and verified this change with .ARW files and other standard formats.
Fixes: #2156
* Add additional type for awr
* pr feedback
---------
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
* Add on this day
* add query for x year
* dev: add query
* dev: front end
* dev: styling
* styling
* more styling
* add new page
* navigating
* navigate back and forth
* styling
* show gallery
* fix test
* fix test
* show previous and next title
* fix test
* show up down scrolling button
* more styling
* styling
* fix app bar
* fix height of next/previous
* autoplay
* auto play
* refactor
* refactor
* refactor
* show date
* Navigate
* finish
* pr feedback
It's perfectly valid to have an email address without a TLD, for instance:
- test@localhost
- test@svc-in-same-k8s-namespace
- test@internal-corp
Fixes#2667
This is required to support raw images as Alpine 3.18 included fixes to imagemagick.
Related: #2156
In addtion, the images have stricter tags and are pinned with a digest. The
manifest list digest can be found using:
```sh
❯ docker buildx imagetools inspect node:18.16.0-alpine3.18
```
* Implement justified layout for timeline
* Add withoutThumbs field to GetTimelineLayotDto
* Back to rough estimation of initial buckets height
* Remove getTimelineLayout endpoint
* Estimate rough viewport height better
* Fix shift/jump issues while scrolling up
---------
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
* updated dockerfile, added typing, packaging
apply env change
* added arm64 support
* added ml version pump, second try for arm64
* added linting config to pyproject.toml
* renamed ml input field
* fixed linter config
* fixed dev docker compose
* feat(server): dynamic job concurrency
* styling and add setting info to top of the job list
* regenerate api
* remove DETECT_OBJECT job
---------
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
* refactor: tags
* chore: open api
* chore: unused import
* feat: add/remove/get tag assets
* chore: open api
* chore: finish tag tests for add/remove assets
* feat: return asset checksum
* chore: generate open api
* chore: coverage
* feat(server): support base64 hashes in bulk upload check:
* chore: generate open api
* feat(mobile): partner sharing
* getAllAssets for other users
* i18n
* fix tests
* try to fix web tests
* shared with/by confusion
* error logging
* guard against outdated server version
* initial commit for XMP sidecar support
* Added support for 'missing' metadata files to include those without sidecar files, now detects sidecar files in the filesystem for media already ingested but the sidecar was created afterwards
* didn't mean to commit default log level during testing
* new sidecar logic for video metadata as well
* Added xml mimetype for sidecars only
* don't need capture group for this regex
* wrong default value reverted
* simplified the move here - keep it in the same try catch since the outcome is to move the media back anyway
* simplified setter logic
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
* simplified logic per suggestions
* sidecar is now its own queue with a discover and sync, updated UI for the new job queueing
* queue a sidecar job for every asset based on discovery or sync, though the logic is almost identical aside from linking the sidecar
* now queue sidecar jobs for each assset, though logic is mostly the same between discovery and sync
* simplified logic of filename extraction and asset instantiation
* not sure how that got deleted..
* updated code per suggestions and comments in the PR
* stat was not being used, removed the variable set
* better type checking, using in-scope variables for exif getter instead of passing in every time
* removed commented out test
* ran and resolved all lints, formats, checks, and tests
* resolved suggested change in PR
* made getExifProperty more dynamic with multiple possible args for fallbacks, fixed typo, used generic in function for better type checking
* better error handling and moving files back to positions on move or save failure
* regenerated api
* format fixes
* Added XMP documentation
* documentation typo
* Merged in main
* missed merge conflict
* more changes due to a merge
* Resolving conflicts
* added icon for sidecar jobs
---------
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
* Modify controller DTOs
* Can check duplicates on server side
* Remove deviceassetid and deviceid
* Remove device ids from file uploader
* Add db migration for removed device ids
* Don't sanitize checksum
* Convert asset checksum to string
* Make checksum not optional for asset
* Use enums when rejecting duplicates
* Cleanup
* Return of the device id, but optional
* Don't use deviceId for upload folder
* Use checksum in thumb path
* Only use asset id in thumb path
* Openapi generation
* Put deviceAssetId back in asset response dto
* Add missing checksum in test fixture
* Add another missing checksum in test fixture
* Cleanup asset repository
* Add back previous /exists endpoint
* Require checksum to not be null
* Correctly set deviceId in db
* Remove index
* Fix compilation errors
* Make device id nullabel in asset response dto
* Reduce PR scope
* Revert asset service
* Reorder imports
* Reorder imports
* Reduce PR scope
* Reduce PR scope
* Reduce PR scope
* Reduce PR scope
* Reduce PR scope
* Update openapi
* Reduce PR scope
* refactor: asset bulk upload check
* chore: regenreate open-api
* chore: fix tests
* chore: tests
* update migrations and regenerate api
* Feat: use checksum in web file uploader
* Change to wasm-crypto
* Use crypto api for checksumming in web uploader
* Minor cleanup of file upload
* feat(web): pause and resume jobs
* Make device asset id not nullable again
* Cleanup
* Device id not nullable in response dto
* Update API specs
* Bump api specs
* Remove old TODO comment
* Remove NOT NULL constraint on checksum index
* Fix requested pubspec changes
* Remove unneeded import
* Update server/apps/immich/src/api-v1/asset/asset.service.ts
Co-authored-by: Michel Heusschen <59014050+michelheusschen@users.noreply.github.com>
* Update server/apps/immich/src/api-v1/asset/asset-repository.ts
Co-authored-by: Michel Heusschen <59014050+michelheusschen@users.noreply.github.com>
* Remove unneeded check
* Update server/apps/immich/src/api-v1/asset/asset-repository.ts
Co-authored-by: Michel Heusschen <59014050+michelheusschen@users.noreply.github.com>
* Remove hashing in the web uploader
* Cleanup file uploader
* Remove varchar from asset entity fields
* Return 200 from bulk upload check
* Put device asset id back into asset repository
* Merge migrations
* Revert pubspec lock
* Update openapi specs
* Merge upstream changes
* Fix failing asset service tests
* Fix formatting issue
* Cleanup migrations
* Remove newline from pubspec
* Revert newline
* Checkout main version
* Revert again
* Only return AssetCheck
---------
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: Michel Heusschen <59014050+michelheusschen@users.noreply.github.com>
A default entrypoint and command make it just a bit easier to use the images as
there is no longer a need for an explicit entrypoint. The exception is the
server image, which still requires the shell script to be specified.
* feat(server): pagination for asset queries in jobs
* default mock value for getAll
* remove live photo name correction
* order paginated results by createdAt
* change log level
* move usePagination to domain
* chore: update to node 18 and alpine 3.17
Signed-off-by: martin <martin.labat92@gmail.com>
* chore: fix sharp version
Signed-off-by: martin <martin.labat92@gmail.com>
* chore(server): use vips-dev
Signed-off-by: martin <martin.labat92@gmail.com>
* update checkDiskUsage
Signed-off-by: martin <martin.labat92@gmail.com>
* fix: use vips-heif instead of libheif
Signed-off-by: martin <martin.labat92@gmail.com>
* fix: use vips instead of vips-cpp
Signed-off-by: martin <martin.labat92@gmail.com>
* fix: ensure vips installation
Signed-off-by: martin <martin.labat92@gmail.com>
---------
Signed-off-by: martin <martin.labat92@gmail.com>
* First crude implementation of the global asset map in web
* Use single DOM element for all markers
* Minor layout changes
* Refactor
* Add asset viewer
* Add API endpoint that returns only assets with location information (Thanks @EPP100)
* Remove sidebar icon flip
* Add dark theme support
* Center map to most recent asset
* Allow cluster viewing
* Fix linter errors
* Add newlines
* Fix ts errors
* Fix eslint error
* Run prettier
* Server code style
* Fix openapi mobile code generation issues
* Map markers test
* fix: Support video thumbnails
* Update API
* Review suggestions
* Review suggestions
* Linter error
* Chage mapMarker endpoint to map-marker
* Clean up leaflet imports
* chore(server): remove unused device info code
* chore: generate open api
* remove any DeviceTypeEnum usage from mobile
* chore: coverage
* fix: drop device info table
---------
Co-authored-by: Fynn Petersen-Frey <zody22@gmail.com>
* feat: manage authorized devices
* chore: open api
* get header from mobile app
* write header from mobile app
* styling
* fix unit test
* feat: use relative time
* feat: update access time
* fix: tests
* chore: confirm wording
* chore: bump test coverage thresholds
* feat: add some icons
* chore: icon tweaks
---------
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
* update asset to include isArchive property
* Not display archived assets on timeline
* replace share button to archive button
* Added archive page
* Add bottom nav bar
* clean up homepage
* remove deadcode
* improve on sync is archive
* show archive asset correctly
* better merge condition
* Added back renderList to re-rendering don't jump around
* Better way to handle showing archive assets
* complete ArchiveSelectionNotifier
* toggle archive
* remove deadcode
* fix unit tests
* update assets in DB when changing assets
* update asset state to reflect archived status
* allow to archive assets via multi-select from timeline
* fixed logic
* Add options to bulk unarchive
* regenerate api
* Change position of toast message
---------
Co-authored-by: Fynn Petersen-Frey <zoodyy@users.noreply.github.com>
* Added dto, logic to insert description and web implementation
* create text field and update on remote database
* Update description and save changes
* styling
* fix web test
* fix server test
* preserve description on metadata extraction job run
* handle exif info is null situation
* pr feedback
* format openapi spec
* update createAssetDto
* refactor logic to service
* move files
* only owner can update description
* Render description correctly in shared album
* Render description correctly in shared link
* disable description edit for not owner of asset on mobile
* localization and clean up
* fix test
* Uses providers for description text (#2244)
* uses providers for description text
* comments
* fixes initial data setting
* fixes notifier
---------
Co-authored-by: martyfuhry <martyfuhry@gmail.com>
* fix(server): generate thumbnail job' using stale path
* add query for webp generation
* revert query for webp because it happens after files are moved
* Add log info
* chore(server): commented out debug log line for codecs
* chore(server): removed debug log line for codecs as it's not needed
* Prettier run
* Make the log more useful and move it to verbose level
* test: rename some fixtures and add text for vertical video conversion
* feat: transcode video asset when audio or container don't match target
* chore: add niceness to the ffmpeg command to allow other processes to be prioritised
* chore: change video conversion queue to one concurrency
* feat: add transcode disabled preset to completely turn off transcoding
* linter
* Change log level and remove unused await
* opps forgot to save
* better logging
---------
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
* feat: change target scaling to resolution in ffmpeg config
* feat(microservices): scale vertical video correctly, only scale if video is larger than target
* default NODE_ENV to production for server image
* update node image to use 3.17 alpine in server
* update web docker image to use alpine 3.17
* remove NODE_ENV from production docker-compose
* NODE_ENV is also needed default in machine-learning
* feat(server/web): jobs clear button + queue status
* adjust design and colors
* Adjust some styling
* show status next to buttons instead of on top
* Update rounded corner for badge
---------
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
* fix(server): user update
* update dto
* generate api
* improve validation
* add e2e tests for updating user
---------
Co-authored-by: Michel Heusschen <59014050+michelheusschen@users.noreply.github.com>
* feat(server): require auth for more endpoints
* dev: add authorization header to profile image on mobile
---------
Co-authored-by: Alex <alex.tran1502@gmail.com>
