Commit graph

230 commits

Author SHA1 Message Date
vishnukvmd
db81f52cf1 Update mail when subscription is on-hold 2024-03-20 14:31:01 +05:30
vishnukvmd
628565cfaf Merge branch 'main' into sepa-fix 2024-03-20 14:08:36 +05:30
Vishnu Mohandas
8101fe1495
Patch few Stripe related issues (#1156) 2024-03-20 14:07:26 +05:30
vishnukvmd
61d5f3189b Update cancellation status on receiving webhook from Stripe 2024-03-20 14:05:09 +05:30
Abhinav
6b749294ce fix sepa issue 2024-03-19 18:40:32 +05:30
Neeraj Gupta
449284a6a1
[server] API to fetch ML embeddings for given fileIDs (#1144)
## Description
- Also started storing the embedding size and version in the table. This
will come handy while comparing overall size when different compression
and serialization formats are used..
- Also, it can be used to smartly auto-download the embeddings or inform
the user about approximate size when they decide to enable indexing or
search on a particular client.

## Tests

 Verified that embedding fetch and store is working fine.
 Verified that embeddings/diff API is working fine.
2024-03-19 12:23:24 +05:30
Neeraj Gupta
685e1b82c9 [server] Fix query for diff API 2024-03-19 12:09:39 +05:30
Neeraj Gupta
2b3494e61c [server] Return embedding version in API response 2024-03-19 11:25:00 +05:30
Neeraj Gupta
e8756a8cf7 [server] Remove unused code 2024-03-19 11:06:36 +05:30
Neeraj Gupta
3df49beb07 [server] Remove unused model name 2024-03-19 11:04:25 +05:30
Neeraj Gupta
05b0f1649f [server] store embedding objSize & version 2024-03-19 11:02:22 +05:30
Neeraj Gupta
7634b2c153 [server] DB migration scripts for emedding table changes 2024-03-19 10:56:25 +05:30
Manav Rathi
303b12c709
Fix the dependency on nginx 2024-03-19 09:31:51 +05:30
Neeraj Gupta
f0236acf8f [server] Minor bug fixes in embedding/files API 2024-03-19 06:15:55 +05:30
Manav Rathi
19724367df
Support nginx configuration reloads 2024-03-18 21:47:53 +05:30
Manav Rathi
fb0e2d2604
Don't restart museum on restarting nginx 2024-03-18 15:54:25 +05:30
Manav Rathi
c177efe79c
Add documentation comments 2024-03-18 15:44:00 +05:30
Manav Rathi
27b13765d7
Fix the environment var 2024-03-18 15:23:23 +05:30
Manav Rathi
f494832bd7
Limit max connections at the nginx level 2024-03-18 15:21:47 +05:30
Neeraj Gupta
c564e50d59 [server] change method type to post 2024-03-18 14:38:11 +05:30
Manav Rathi
53cf5ffc45
Fix some paths in README 2024-03-18 14:35:42 +05:30
Manav Rathi
a341f81932
[museum] Support running behind Nginx (#1130)
- Move deployment related stuff into a separate folder
- Document the overall approach better
- Add an Nginx specific environment and service definition
- Remove the restart limiter
2024-03-18 13:56:01 +05:30
Neeraj Gupta
b9ec72bbb4 Merge branch 'main' into file_embedding 2024-03-18 13:12:14 +05:30
Manav Rathi
7ded8ad4fe
Fix key name 2024-03-18 13:07:36 +05:30
Manav Rathi
30cf709eb2
Wrap 2024-03-18 12:39:16 +05:30
Manav Rathi
d8347c3976
Remove extra environment 2024-03-18 12:36:29 +05:30
Manav Rathi
5ee878c30e
Require nginx before starting museum.nginx 2024-03-18 12:33:27 +05:30
Manav Rathi
f695c5e0d3
Remove extra newline 2024-03-18 11:06:54 +05:30
Manav Rathi
303b26b95f
Tweak the nginx setup 2024-03-18 11:04:31 +05:30
Manav Rathi
9ba9b8a984
Add initial nginx conf 2024-03-18 10:39:39 +05:30
Manav Rathi
d893efe5d1
Remove the restart limiter
During recent DDoS attacks, these restart limits had to be manually
disabled. For now, disabling them in our reference copy too, will bring them
back if they are needed later.
2024-03-18 10:27:42 +05:30
Manav Rathi
a0599f7b33
Outline the nginx approach 2024-03-18 10:25:11 +05:30
Manav Rathi
9beedcf4e0
Move into separate folder 2024-03-18 10:16:55 +05:30
Neeraj Gupta
47747f1d56 Update config 2024-03-18 10:12:02 +05:30
Neeraj Gupta
7de60f1ced [server] Reduce discord message frequency 2024-03-18 10:02:13 +05:30
Neeraj Gupta
4acc8395ba [server] Prefix discord msg with hostname 2024-03-18 09:59:35 +05:30
Neeraj Gupta
12b9ac4db6
[server] Rate limit (#1121)
## Description

## Tests
2024-03-17 09:59:59 +05:30
Neeraj Gupta
e927064476 [server] Add endpoint to get embedding for givenFilesIds 2024-03-17 09:37:02 +05:30
Neeraj Gupta
36982c5332 [server] Add util to findMissingElements while comparing two arrays 2024-03-16 02:30:25 +05:30
Manav Rathi
92ae1be40a
Add a new flag to support S3 path style URLs without disabling SSL
The use case for this came up in Discord - someone is running a custom self
hosted instance where they're using MinIO as their "production S3", but it is
not local (so they don't want the clients to connect without SSL). So we need a
way to get MinIO to work, which needs path style URLs, without also disabling
the SSL.
2024-03-15 17:18:48 +05:30
Manav Rathi
87d39ae2dd
Disable the example.org domains in the default self-hosted configuration
People won't be able to get emails, but they can still create accounts on that
domain in the default setup. So just comment it out as a saner default.
2024-03-15 17:09:40 +05:30
Manav Rathi
966cc697e7
[server] Move the server related gitattributes within that folder
I mistakenly assumed that the gitattributes file can only be at the top
level. However, this doesn't seem to be the case - I noticed this in a recent PR
- https://github.com/ente-io/ente/pull/1104/files.

And it is indeed documented:

> When deciding what attributes are assigned to a path, Git .... .gitattributes
> file in the same directory as the path in question. ..
>
> - https://git-scm.com/docs/gitattributes

Ref: https://stackoverflow.com/questions/71784062/can-we-have-multiple-gitattributes-files-in-different-repo-subfolders
2024-03-14 16:35:36 +05:30
Manav Rathi
2905315e00
[server] Improve the architecture diagram
Add a dotted arrow between museum and the hot object storage to indicate that
clients talk directly to the primary S3 storage.

This has confused some users who were trying to setup self-hosting, them not
realizing that the minio buckets need to be accessible outside the container.

The actual flow is - for both GET and PUT, museum conjures up pre-signed URLs by
talking to the S3 bucket, and vends these pre-signed URLs back to the
client. Thereafter though (unless routed via a Cloudflare worker, which is not
the case when self-hosting), the clients directly connect to this presigned URL.

Maybe too much information to convey with a single dotted arrow, but at least
it'll give a hint that there is a direct connection there somewhere.
2024-03-14 16:23:53 +05:30
Manav Rathi
c71d06d405
Mention the self-hosting docs in the server README 2024-03-14 11:12:27 +05:30
vishnukvmd
77276d8d6c [server] Add admin API to change email address 2024-03-13 17:31:38 +05:30
Neeraj Gupta
c0a2347b80 [server] Refactor /user/plans 2024-03-12 12:17:04 +05:30
Neeraj Gupta
e843ea6669 [server] Change param type from uuid to string 2024-03-11 22:45:53 +05:30
Neeraj Gupta
bb68b22adb [server] mark queue as deprecated 2024-03-11 15:41:18 +05:30
Neeraj Gupta
5accf4c6a8 [server] Remove dead code related to collecton trashV2 2024-03-11 15:41:18 +05:30
Neeraj Gupta
28335700e3 [server] Remove deprecated API 2024-03-11 15:41:18 +05:30
Neeraj Gupta
9711e0e29e Improve log 2024-03-11 12:34:03 +05:30
Neeraj Gupta
d7292dc670 [server] Release locks before initiating any other controller 2024-03-11 12:34:03 +05:30
Neeraj Gupta
6930aaf220 [server]Update freq for metadatacron 2024-03-11 12:34:03 +05:30
Neeraj Gupta
12903a3748 [server] Clean up collecitonDiff logs 2024-03-11 12:34:03 +05:30
Neeraj Gupta
4af3030c81 [server] Speed up RemoveComplianceHolds 2024-03-11 11:43:02 +05:30
Neeraj Gupta
c32f0a28f1 [server] Release previous locks taken by host on statup 2024-03-11 11:43:02 +05:30
Neeraj Gupta
89d761a096 Reduce lock duration 2024-03-11 10:39:46 +05:30
Neeraj Gupta
e667eef951 Tweak speed for crons 2024-03-11 10:39:46 +05:30
Neeraj Gupta
6e160dca43 Fix minor bugs 2024-03-08 15:15:00 +05:30
Neeraj Gupta
1f7d9dbb86 Rename passKey to passkey 2024-03-08 15:15:00 +05:30
Neeraj Gupta
a780598607 Fix sql query 2024-03-08 15:15:00 +05:30
Neeraj Gupta
7f66714d96 Refactor + bug fixes 2024-03-08 15:15:00 +05:30
Neeraj Gupta
980ab6c49c Refactor: extend totp recovery API to recover passkey 2024-03-08 15:15:00 +05:30
Neeraj Gupta
fe181fecbe Rename 2024-03-08 15:15:00 +05:30
Neeraj Gupta
50c3a7a8e5 Store resetSecret in encrypted form 2024-03-08 15:15:00 +05:30
Neeraj Gupta
f766484b2e Rename account_recovery -> two_factor_recovery 2024-03-08 15:15:00 +05:30
Neeraj Gupta
42e4364fda Add APIs to allow user to skip passkey based two-fa 2024-03-08 15:15:00 +05:30
Neeraj Gupta
09a7d557d2 Add API to get account two recovery status 2024-03-08 15:15:00 +05:30
Neeraj Gupta
13bae268ec Add models passkey reset via recoveryKey 2024-03-08 15:15:00 +05:30
Neeraj Gupta
23fcce245d Add table for account recovery 2024-03-08 15:15:00 +05:30
Neeraj Gupta
8186246690
[server] Update freq for removing compliance hold (#731)
## Description
- Taking lock at host level to avoid redundant downstream request when
multiple host pick same items.

## Tests
Tested locally.
2024-03-07 13:08:04 +05:30
Manav Rathi
795c2baf0f Fix lint 2024-03-05 16:53:15 +05:30
Manav Rathi
3cad79dfa9 Add museum release action 2024-03-05 16:39:11 +05:30
Manav Rathi
0b37f74636 Add museum lint action 2024-03-05 16:33:20 +05:30
Neeraj Gupta
34743171f9
[server] Add API to re-queue item for processing (#679)
## Description

## Tests
2024-03-05 15:15:23 +05:30
Neeraj Gupta
3513d7a297
[server] Remove entry from object_keys before marking del as completed (#677)
## Description

## Tests
2024-03-05 14:56:02 +05:30
Manav Rathi
44eb8353a2 Update server/ to account for new monorepo structure 2024-03-02 18:04:59 +05:30
Neeraj Gupta
f796a7f63a
[server] Remove custom metrics for collection diff (#520) 2024-03-02 17:14:35 +05:30
Julien
bb32d7684a
docs: Fixed url to RUNNING.md in README 2024-03-02 10:39:26 +01:00
Neeraj Gupta
531bb344fe Import museum 2024-03-01 13:37:01 +05:30