Commit graph

233 commits

Author SHA1 Message Date
Vishal
92715b658c Change API parameter 2024-04-03 19:24:12 +05:30
Vishal
2ddf4c897c Rectify if else 2024-04-03 18:25:41 +05:30
Vishal
ffefae89a6 Redefine struct 2024-04-03 17:50:53 +05:30
Vishal
39ec761949 fix warnings 2024-04-03 12:57:37 +05:30
Vishal
01f842c445 Rearrange methods 2024-04-03 12:41:18 +05:30
Neeraj Gupta
2fe703df92
[server] Increase embedding fetch limit (#1300)
## Description

Also use different semaphore than existing diff API

## Tests
2024-04-03 12:38:34 +05:30
Vishal
18c48c7e0a Fix typo in comment 2024-04-03 11:14:55 +05:30
Manav Rathi
885308471f
[server] Add a notice that minio is only for getting started
From our Discord, someone commented

> if minio's cautions about Single-Node, Single-Drive are to be taken seriously:
>
> "SNSD deployments use a zero-parity erasure coded backend that provides no
  added reliability or availability beyond what the underlying storage volume
  implements. These deployments are best suited for local testing and
  evaluation, or for small-scale data workloads that do not have availability or
  performance requirements."

MinIO was never meant as a production replacement, it was only to make it easy
for people to get started. So add a notice in the docs re this.
2024-04-03 10:01:48 +05:30
Neeraj Gupta
85778bcdaa [server] Fix handling of redundant auth update 2024-04-03 07:05:03 +05:30
Vishal
b8100b1273 rename functions 2024-04-02 17:45:30 +05:30
Vishal
a5fcbbf901 add listmonk mailing list control (subscribe/usubscribe) 2024-04-02 17:36:53 +05:30
Neeraj Gupta
25cedd5e2f
[server] Gracefully handle stale collection entry (#1260)
## Description

## Tests
2024-03-30 15:24:52 +05:30
Neeraj Gupta
f6dca2dfc9
[server] Drop locationTag table and related code (#1245)
## Description

## Tests
2024-03-29 16:25:39 +05:30
Neeraj Gupta
ef250acad9
[server] Add person EntityType (#1246)
## Description

## Tests
2024-03-29 16:12:59 +05:30
Neeraj Gupta
459c4515a0 [server] Drop locationTag table and related code 2024-03-29 16:08:28 +05:30
Manav Rathi
54d93647ef
Add SMTP for sending emails (#1228)
## Tests
Tested for templated mails in user.go, userauth.go, stripe.go,
public_collection.go, playstore.go, admin.go, email_notification.go

Screenshots for all the templates -
[https://drive.google.com/drive/folders/1f7BjM5MHicRUbr5jlCe2-4MGz7s1moLw?usp=sharing](url)
2024-03-29 09:22:40 +05:30
Vishal
d5f0334a34 Remove isEnabled from config 2024-03-28 17:44:08 +05:30
Vishal
1f75633c5c Private method to bottom of the file 2024-03-28 15:03:27 +05:30
Manav Rathi
2b379640ca
Allow billing plans to be mounted into the Docker container if needed 2024-03-28 13:07:43 +05:30
Vishal
cc3376154f Change name 2024-03-27 22:39:12 +05:30
Vishal
a7692b7d2e Update if else 2024-03-27 21:40:40 +05:30
Vishal
9893e3f305 Add Amazon SES for sending mails 2024-03-27 19:58:27 +05:30
Vishnu Mohandas
eef33e9c0c
[mobile][web] Redirect to payment portal if subscription is past due (#1222)
## Description

When a customer whose Stripe subscription is past due (within the 30 day
window after expiry time and has not been cancelled) clicks on the
subscription modal, take them to the payment portal to complete the
subscription.

## Tests

- [x] Tested web
- [x] Tested mobile
2024-03-27 16:29:58 +05:30
vishnukvmd
44403c94c2 Cancel SEPA subscriptions when payments fail 2024-03-27 16:20:58 +05:30
Vishnu Mohandas
0c400db649
[server] Fix Stripe related issues (#1152)
## Description

### Stripe
- Settings have been modified to "Cancel" the subscription if all
attempts to charge an invoice fails. The previous configuration was to
set the subscription to "Past Due".

### Museum
- Updated our Stripe workflow to use `PaymentBehaviorAllowIncomplete` in
case of SEPA payments
- Fixed the issue caused by us spawning unnecessary subscriptions when a
customer tried to renew their expired subscription
- Removed the email that was being sent out when a Stripe subscription
expired. We will have to run our own cron to send a notification and
perform necessary clean up. When payments fail, we will still send a
notification to intimate the customer that their account is on hold.

### Mobile
- Fixed the dialogs and error messages on the payment page

## Tests
The following cases were tested for SEPA, Credit Cards, and Credit Cards
with 3DS.
- [x] Purchase
  - [x] Success
  - [x] Failure
- [x] Renewal
  - [x] Success
  - [x] Failure
- [x] Upgrades
  - [x] Success
  - [x] Failure
- [x] Cancellation
- [x] Resumption post recovery

## Note

These changes are likely to break in the event of some corner cases we
haven't thought of. But this will land us in a state that is better than
where we are right now in terms of collecting payments for expired
subscriptions and those via SEPA.
2024-03-27 14:43:36 +05:30
vishnukvmd
25a054287a Prevent duplicate emails for SEPA 2024-03-27 14:22:30 +05:30
Manav Rathi
e803b7f053
Document in main README 2024-03-27 13:44:24 +05:30
Manav Rathi
4a4a6ba44a
Give an example of the pull command 2024-03-27 13:04:28 +05:30
Manav Rathi
cb45a070be
Tweaks 2024-03-27 12:58:17 +05:30
Manav Rathi
4ba76e4aa2
Tweaks 2024-03-27 12:46:08 +05:30
Manav Rathi
7c0f41166c
Add a first cut of the publish script 2024-03-27 12:42:29 +05:30
Manav Rathi
b31332ff30
Outline the plan 2024-03-27 12:42:26 +05:30
vishnukvmd
c9618d8038 Update copy 2024-03-26 21:31:29 +05:30
vishnukvmd
888aca7e6c Extract common code 2024-03-26 21:11:57 +05:30
vishnukvmd
31d3319d20 Revert "Depend on subscription-past-due events to trigger the account-on-hold email"
This reverts commit a131c16f4e.
2024-03-26 20:59:02 +05:30
vishnukvmd
a131c16f4e Depend on subscription-past-due events to trigger the account-on-hold email 2024-03-26 20:29:55 +05:30
vishnukvmd
f2aac262b1 Remove unused flag 2024-03-25 23:27:39 +05:30
vishnukvmd
4afc056a9f Stop listening to subscription deleted events 2024-03-25 23:26:48 +05:30
vishnukvmd
2dcf5daf54 Handle cases where default payment method might not be specified 2024-03-25 23:16:20 +05:30
vishnukvmd
29245b73b9 Cancel subscriptions on SEPA failures immediately 2024-03-25 23:15:43 +05:30
vishnukvmd
d003542e42 Simplify how last payment method is fetched 2024-03-25 23:13:56 +05:30
Manav Rathi
a4ecc2c344
Update ente.io/faq => help.ente.io where possible
On desktop, now there Help menu will have an "Ente Help" option instead of "FAQ"
(Unrelated: I noticed that user facing strings in the desktop app are not
localized).

The only place remaining after this is the sidebar on mobile
(`support_section_widget.dart`), but I wasn't sure of what string to use (and
how to add it to the localized strings for mobile) so I've left it unchanged.
2024-03-25 19:26:10 +05:30
vishnukvmd
a275ab7740 Set subscription to expire 7 days after payment failure 2024-03-23 14:37:29 +05:30
vishnukvmd
e634310cb1 Remove unused method 2024-03-21 14:49:37 +05:30
vishnukvmd
b95d7c241d Create new subs only if the existing subscription is in a cancelled state 2024-03-21 14:48:29 +05:30
vishnukvmd
3cf680eec1 [WiP] Prevent subscription cancellation 2024-03-21 14:42:38 +05:30
vishnukvmd
1f17f2442a Handle SEPA failures 2024-03-21 14:33:10 +05:30
vishnukvmd
859af52c64 Remove commented code 2024-03-21 13:55:42 +05:30
vishnukvmd
78e5e3bdb1 Minor refactor 2024-03-21 13:31:46 +05:30
vishnukvmd
5635e63cb7 Update handling for subscription updated events 2024-03-21 13:29:42 +05:30
Abhinav
520f7ab64c only use PaymentBehaviorAllowIncomplete for SEPA payment method 2024-03-21 07:18:28 +05:30
Abhinav
4e074770d2 invoice client specific invoice api 2024-03-21 06:48:49 +05:30
vishnukvmd
49893d9732 Update on-hold email subject 2024-03-20 14:31:33 +05:30
vishnukvmd
db81f52cf1 Update mail when subscription is on-hold 2024-03-20 14:31:01 +05:30
vishnukvmd
628565cfaf Merge branch 'main' into sepa-fix 2024-03-20 14:08:36 +05:30
Vishnu Mohandas
8101fe1495
Patch few Stripe related issues (#1156) 2024-03-20 14:07:26 +05:30
vishnukvmd
61d5f3189b Update cancellation status on receiving webhook from Stripe 2024-03-20 14:05:09 +05:30
Abhinav
6b749294ce fix sepa issue 2024-03-19 18:40:32 +05:30
Neeraj Gupta
449284a6a1
[server] API to fetch ML embeddings for given fileIDs (#1144)
## Description
- Also started storing the embedding size and version in the table. This
will come handy while comparing overall size when different compression
and serialization formats are used..
- Also, it can be used to smartly auto-download the embeddings or inform
the user about approximate size when they decide to enable indexing or
search on a particular client.

## Tests

 Verified that embedding fetch and store is working fine.
 Verified that embeddings/diff API is working fine.
2024-03-19 12:23:24 +05:30
Neeraj Gupta
685e1b82c9 [server] Fix query for diff API 2024-03-19 12:09:39 +05:30
Neeraj Gupta
2b3494e61c [server] Return embedding version in API response 2024-03-19 11:25:00 +05:30
Neeraj Gupta
e8756a8cf7 [server] Remove unused code 2024-03-19 11:06:36 +05:30
Neeraj Gupta
3df49beb07 [server] Remove unused model name 2024-03-19 11:04:25 +05:30
Neeraj Gupta
05b0f1649f [server] store embedding objSize & version 2024-03-19 11:02:22 +05:30
Neeraj Gupta
7634b2c153 [server] DB migration scripts for emedding table changes 2024-03-19 10:56:25 +05:30
Manav Rathi
303b12c709
Fix the dependency on nginx 2024-03-19 09:31:51 +05:30
Neeraj Gupta
f0236acf8f [server] Minor bug fixes in embedding/files API 2024-03-19 06:15:55 +05:30
Manav Rathi
19724367df
Support nginx configuration reloads 2024-03-18 21:47:53 +05:30
Manav Rathi
fb0e2d2604
Don't restart museum on restarting nginx 2024-03-18 15:54:25 +05:30
Manav Rathi
c177efe79c
Add documentation comments 2024-03-18 15:44:00 +05:30
Manav Rathi
27b13765d7
Fix the environment var 2024-03-18 15:23:23 +05:30
Manav Rathi
f494832bd7
Limit max connections at the nginx level 2024-03-18 15:21:47 +05:30
Neeraj Gupta
c564e50d59 [server] change method type to post 2024-03-18 14:38:11 +05:30
Manav Rathi
53cf5ffc45
Fix some paths in README 2024-03-18 14:35:42 +05:30
Manav Rathi
a341f81932
[museum] Support running behind Nginx (#1130)
- Move deployment related stuff into a separate folder
- Document the overall approach better
- Add an Nginx specific environment and service definition
- Remove the restart limiter
2024-03-18 13:56:01 +05:30
Neeraj Gupta
b9ec72bbb4 Merge branch 'main' into file_embedding 2024-03-18 13:12:14 +05:30
Manav Rathi
7ded8ad4fe
Fix key name 2024-03-18 13:07:36 +05:30
Manav Rathi
30cf709eb2
Wrap 2024-03-18 12:39:16 +05:30
Manav Rathi
d8347c3976
Remove extra environment 2024-03-18 12:36:29 +05:30
Manav Rathi
5ee878c30e
Require nginx before starting museum.nginx 2024-03-18 12:33:27 +05:30
Manav Rathi
f695c5e0d3
Remove extra newline 2024-03-18 11:06:54 +05:30
Manav Rathi
303b26b95f
Tweak the nginx setup 2024-03-18 11:04:31 +05:30
Manav Rathi
9ba9b8a984
Add initial nginx conf 2024-03-18 10:39:39 +05:30
Manav Rathi
d893efe5d1
Remove the restart limiter
During recent DDoS attacks, these restart limits had to be manually
disabled. For now, disabling them in our reference copy too, will bring them
back if they are needed later.
2024-03-18 10:27:42 +05:30
Manav Rathi
a0599f7b33
Outline the nginx approach 2024-03-18 10:25:11 +05:30
Manav Rathi
9beedcf4e0
Move into separate folder 2024-03-18 10:16:55 +05:30
Neeraj Gupta
47747f1d56 Update config 2024-03-18 10:12:02 +05:30
Neeraj Gupta
7de60f1ced [server] Reduce discord message frequency 2024-03-18 10:02:13 +05:30
Neeraj Gupta
4acc8395ba [server] Prefix discord msg with hostname 2024-03-18 09:59:35 +05:30
Neeraj Gupta
12b9ac4db6
[server] Rate limit (#1121)
## Description

## Tests
2024-03-17 09:59:59 +05:30
Neeraj Gupta
e927064476 [server] Add endpoint to get embedding for givenFilesIds 2024-03-17 09:37:02 +05:30
Neeraj Gupta
36982c5332 [server] Add util to findMissingElements while comparing two arrays 2024-03-16 02:30:25 +05:30
Manav Rathi
92ae1be40a
Add a new flag to support S3 path style URLs without disabling SSL
The use case for this came up in Discord - someone is running a custom self
hosted instance where they're using MinIO as their "production S3", but it is
not local (so they don't want the clients to connect without SSL). So we need a
way to get MinIO to work, which needs path style URLs, without also disabling
the SSL.
2024-03-15 17:18:48 +05:30
Manav Rathi
87d39ae2dd
Disable the example.org domains in the default self-hosted configuration
People won't be able to get emails, but they can still create accounts on that
domain in the default setup. So just comment it out as a saner default.
2024-03-15 17:09:40 +05:30
Manav Rathi
966cc697e7
[server] Move the server related gitattributes within that folder
I mistakenly assumed that the gitattributes file can only be at the top
level. However, this doesn't seem to be the case - I noticed this in a recent PR
- https://github.com/ente-io/ente/pull/1104/files.

And it is indeed documented:

> When deciding what attributes are assigned to a path, Git .... .gitattributes
> file in the same directory as the path in question. ..
>
> - https://git-scm.com/docs/gitattributes

Ref: https://stackoverflow.com/questions/71784062/can-we-have-multiple-gitattributes-files-in-different-repo-subfolders
2024-03-14 16:35:36 +05:30
Manav Rathi
2905315e00
[server] Improve the architecture diagram
Add a dotted arrow between museum and the hot object storage to indicate that
clients talk directly to the primary S3 storage.

This has confused some users who were trying to setup self-hosting, them not
realizing that the minio buckets need to be accessible outside the container.

The actual flow is - for both GET and PUT, museum conjures up pre-signed URLs by
talking to the S3 bucket, and vends these pre-signed URLs back to the
client. Thereafter though (unless routed via a Cloudflare worker, which is not
the case when self-hosting), the clients directly connect to this presigned URL.

Maybe too much information to convey with a single dotted arrow, but at least
it'll give a hint that there is a direct connection there somewhere.
2024-03-14 16:23:53 +05:30
Manav Rathi
c71d06d405
Mention the self-hosting docs in the server README 2024-03-14 11:12:27 +05:30
vishnukvmd
77276d8d6c [server] Add admin API to change email address 2024-03-13 17:31:38 +05:30
Neeraj Gupta
c0a2347b80 [server] Refactor /user/plans 2024-03-12 12:17:04 +05:30
Neeraj Gupta
e843ea6669 [server] Change param type from uuid to string 2024-03-11 22:45:53 +05:30
Neeraj Gupta
bb68b22adb [server] mark queue as deprecated 2024-03-11 15:41:18 +05:30
Neeraj Gupta
5accf4c6a8 [server] Remove dead code related to collecton trashV2 2024-03-11 15:41:18 +05:30
Neeraj Gupta
28335700e3 [server] Remove deprecated API 2024-03-11 15:41:18 +05:30
Neeraj Gupta
9711e0e29e Improve log 2024-03-11 12:34:03 +05:30
Neeraj Gupta
d7292dc670 [server] Release locks before initiating any other controller 2024-03-11 12:34:03 +05:30
Neeraj Gupta
6930aaf220 [server]Update freq for metadatacron 2024-03-11 12:34:03 +05:30
Neeraj Gupta
12903a3748 [server] Clean up collecitonDiff logs 2024-03-11 12:34:03 +05:30
Neeraj Gupta
4af3030c81 [server] Speed up RemoveComplianceHolds 2024-03-11 11:43:02 +05:30
Neeraj Gupta
c32f0a28f1 [server] Release previous locks taken by host on statup 2024-03-11 11:43:02 +05:30
Neeraj Gupta
89d761a096 Reduce lock duration 2024-03-11 10:39:46 +05:30
Neeraj Gupta
e667eef951 Tweak speed for crons 2024-03-11 10:39:46 +05:30
Neeraj Gupta
6e160dca43 Fix minor bugs 2024-03-08 15:15:00 +05:30
Neeraj Gupta
1f7d9dbb86 Rename passKey to passkey 2024-03-08 15:15:00 +05:30
Neeraj Gupta
a780598607 Fix sql query 2024-03-08 15:15:00 +05:30
Neeraj Gupta
7f66714d96 Refactor + bug fixes 2024-03-08 15:15:00 +05:30
Neeraj Gupta
980ab6c49c Refactor: extend totp recovery API to recover passkey 2024-03-08 15:15:00 +05:30
Neeraj Gupta
fe181fecbe Rename 2024-03-08 15:15:00 +05:30
Neeraj Gupta
50c3a7a8e5 Store resetSecret in encrypted form 2024-03-08 15:15:00 +05:30
Neeraj Gupta
f766484b2e Rename account_recovery -> two_factor_recovery 2024-03-08 15:15:00 +05:30
Neeraj Gupta
42e4364fda Add APIs to allow user to skip passkey based two-fa 2024-03-08 15:15:00 +05:30
Neeraj Gupta
09a7d557d2 Add API to get account two recovery status 2024-03-08 15:15:00 +05:30
Neeraj Gupta
13bae268ec Add models passkey reset via recoveryKey 2024-03-08 15:15:00 +05:30
Neeraj Gupta
23fcce245d Add table for account recovery 2024-03-08 15:15:00 +05:30
Neeraj Gupta
8186246690
[server] Update freq for removing compliance hold (#731)
## Description
- Taking lock at host level to avoid redundant downstream request when
multiple host pick same items.

## Tests
Tested locally.
2024-03-07 13:08:04 +05:30
Manav Rathi
795c2baf0f Fix lint 2024-03-05 16:53:15 +05:30
Manav Rathi
3cad79dfa9 Add museum release action 2024-03-05 16:39:11 +05:30
Manav Rathi
0b37f74636 Add museum lint action 2024-03-05 16:33:20 +05:30
Neeraj Gupta
34743171f9
[server] Add API to re-queue item for processing (#679)
## Description

## Tests
2024-03-05 15:15:23 +05:30
Neeraj Gupta
3513d7a297
[server] Remove entry from object_keys before marking del as completed (#677)
## Description

## Tests
2024-03-05 14:56:02 +05:30
Manav Rathi
44eb8353a2 Update server/ to account for new monorepo structure 2024-03-02 18:04:59 +05:30
Neeraj Gupta
f796a7f63a
[server] Remove custom metrics for collection diff (#520) 2024-03-02 17:14:35 +05:30
Julien
bb32d7684a
docs: Fixed url to RUNNING.md in README 2024-03-02 10:39:26 +01:00
Neeraj Gupta
531bb344fe Import museum 2024-03-01 13:37:01 +05:30