beenull/immich
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity
2674 commits 74 branches 159 tags 243 MiB
Commit graph

26 commits

Author SHA1 Message Date
Mert
ddf04a7eb4
chore(ml): increase spool threshold to 64MiB (#5176) 2023-11-20 09:05:35 -06:00
Mert
a6af4892e3
fix(ml): better model unloading (#3340) 
* restart process on inactivity

* formatting

* always update `last_called`

* load models sequentially

* renamed variable, updated docs

* formatting

* made poll env name consistent with model ttl env
 2023-11-16 20:42:44 -06:00
Mert
291159e7fc
fixed tests (#5017) 2023-11-13 19:37:39 +00:00
Mert
935f471ccb
chore(ml): use strict mypy (#5001) 
* improved typing

* improved export typing

* strict mypy & check export folder

* formatting

* add formatting checks for export folder

* re-added init call
 2023-11-13 10:18:46 -06:00
Mert
328a58ac0d
feat(ml): add face models (#4952) 
added models to config dropdown

fixed downloading

updated tests

use hf for face models

formatting
 2023-11-11 19:04:49 -06:00
Mert
87a0ba3db3
feat(ml): export clip models to ONNX and host models on Hugging Face (#4700) 
* export clip models

* export to hf

refactored export code

* export mclip, general refactoring

cleanup

* updated conda deps

* do transforms with pillow and numpy, add tokenization config to export, general refactoring

* moved conda dockerfile, re-added poetry

* minor fixes

* updated link

* updated tests

* removed `requirements.txt` from workflow

* fixed mimalloc path

* removed torchvision

* cleaner np typing

* review suggestions

* update default model name

* update test
 2023-10-31 05:02:04 -05:00
Mert
d8ecefaea5
chore(ml): removed vit-b check and st warning (#4422) 2023-10-10 12:26:30 -05:00
Mert
258b98c262
fix(ml): load models in separate threads (#4034) 
* load models in thread

* set clip mode logs to debug level

* updated tests

* made fixtures slightly less ugly

* moved responses to json file

* formatting
 2023-09-09 16:02:44 +07:00
Mert
3cf0f5f11b
fix(ml): model downloading improvements (#3988) 
* handle ort `NoSuchFile` error, stricter file check

* keep exception order
 2023-09-06 08:48:40 +07:00
Daniele Ricci
c7d53a5006
docs: remove obsolete environment variables (#3926) 
* Obsolete environment variables

* Review fixes

* Review fixes

* Review fixes
 2023-09-01 18:05:45 +07:00
Mert
41461e0d5d
chore(ml): memory optimisations (#3934) 2023-08-31 18:30:53 -05:00
Mert
54b2779b79
chore(ml): improved logging (#3918) 
* fixed `minScore` not being set correctly

* apply to init

* don't send `enabled`

* fix eslint warning

* added logger

* added logging

* refinements

* enable access log for info level

* formatting

* merged strings

---------

Co-authored-by: Alex <alex.tran1502@gmail.com>
 2023-08-30 08:22:01 +00:00
Mert
df26e12db6
fix(ml): minScore not being set correctly (#3916) 
* fixed `minScore` not being set correctly

* apply to init

* don't send `enabled`

* fix eslint warning

* better error message
 2023-08-30 03:16:00 -05:00
Mert
bcc36d14a1
feat(ml)!: customizable ML settings (#3891) 
* consolidated endpoints, added live configuration

* added ml settings to server

* added settings dashboard

* updated deps, fixed typos

* simplified modelconfig

updated tests

* Added ml setting accordion for admin page

updated tests

* merge `clipText` and `clipVision`

* added face distance setting

clarified setting

* add clip mode in request, dropdown for face models

* polished ml settings

updated descriptions

* update clip field on error

* removed unused import

* add description for image classification threshold

* pin safetensors for arm wheel

updated poetry lock

* moved dto

* set model type only in ml repository

* revert form-data package install

use fetch instead of axios

* added slotted description with link

updated facial recognition description

clarified effect of disabling tasks

* validation before model load

* removed unnecessary getconfig call

* added migration

* updated api

updated api

updated api

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
 2023-08-29 08:58:00 -05:00
Mert
165b91b068
feat(ml)!: switch image classification and CLIP models to ONNX (#3809) 2023-08-25 06:28:51 +02:00
Mert
60729a091a
make lazy loading default (#3797) 2023-08-20 18:24:14 -05:00
Mert
fa57853bd2
fix(ml): clear cache if bad zip file (#3600) 2023-08-08 12:33:13 -05:00
Mert
c73832bd9c
refactor(ml): model downloading (#3545) 
* download facial recognition models

* download hf models

* simplified logic

* updated `predict` for facial recognition

* ensure download method is called

* fixed repo_id for clip

* fixed download destination

* use st's own `snapshot_download`

* conditional download

* fixed predict method

* check if loaded

* minor fixes

* updated mypy overrides

* added pytest-mock

* updated tests

* updated lock
 2023-08-05 21:45:13 -05:00
Mert
848ba685eb
fix(ml): race condition when loading models (#3207) 
* sync model loading, disabled model ttl by default

* disable revalidation if model unloading disabled

* moved lock
 2023-07-11 12:01:21 -05:00
Mert
df1e8679d9
chore(ml): added testing and github workflow (#2969) 
* added testing

* github action for python, made mypy happy

* formatted with black

* minor fixes and styling

* test model cache

* cache test dependencies

* narrowed model cache tests

* moved endpoint tests to their own class

* cleaned up fixtures

* formatting

* removed unused dep
 2023-06-27 18:21:33 -05:00
Mert
47982641b2
fix(ml): clear model cache on load error (#2951) 
* clear model cache on load error

* updated caught exceptions
 2023-06-27 16:01:24 -05:00
Mert
4d3ce0a65e
fixed setting different clip, removed unused stubs (#2987) 2023-06-27 12:21:50 -05:00
Mert
a2f5674bbb
refactor(ml): modularization and styling (#2835) 
* basic refactor and styling

* removed batching

* module entrypoint

* removed unused imports

* model superclass,  model cache now in app state

* fixed cache dir and enforced abstract method

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
 2023-06-24 22:18:09 -05:00
Zeeshan Khan
34201be74c
feat(ml) backend takes image over HTTP (#2783) 
* using pydantic BaseSetting

* ML API takes image file as input

* keeping image in memory

* reducing duplicate code

* using bytes instead of UploadFile & other small code improvements

* removed form-multipart, using HTTP body

* format code

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
 2023-06-17 22:49:19 -05:00
Mert
d0cc231782
feat(ml): model unloading (#2661) 
* model cache

* fixed revalidation when using cache namespace

* fixed ttl not being set, added lock
 2023-06-06 20:48:51 -05:00
Mert
1e748864c5
chore(ml): updated dockerfile, added typing, packaging (#2642) 
* updated dockerfile, added typing, packaging

apply env change

* added arm64 support

* added ml version pump, second try for arm64

* added linting config to pyproject.toml

* renamed ml input field

* fixed linter config

* fixed dev docker compose
 2023-06-05 09:40:48 -05:00