Commit graph

80 commits

Author SHA1 Message Date
Kailash Nadh
8733b205a0 Refactor SQL schema and queries for performance improvements.
- Add indexes.
- Refactor dashboard charts and view/click count queries.
  (~10x speed bump on a setup of 7mn subscribers and 80mn views)
- Refactor get subscriber queries.
  (~10x speed bump on 7mn subscribers)
- Make subscriber UI issue an equality query for email seach strings.
2021-09-18 17:25:08 +05:30
Kailash Nadh
9302dfbd56 Add missing id (pkey) to analytics tables for faster queries 2021-09-17 20:11:45 +05:30
Kailash Nadh
3d0031b207 Add campaign analytics APIs and UI 2021-09-17 18:45:35 +05:30
Kailash Nadh
158ea9fad2 Fix bounce action only triggering on n+1st bounce. 2021-08-22 15:42:54 +05:30
Kailash Nadh
1ae98699e7 Add support for bounce processing.
- Blocklist or unsubscribe subscribers based on a bounce threshold
- Add /bounces UI for viewing bounces and in the subscriber view
- Add settings UI for managing bounce settings
- Add support for scanning POP3 bounce mailboxes
- Add a generic webhook for posting custom bounces at /webhooks/bounce
- Add SES bounce webhook support at /webhooks/services/ses
- Add Sendgrid bounce webhook support at /webhooks/services/sendgrid
2021-08-14 15:35:29 +05:30
Kailash Nadh
e6566189ed Add preconfirm_subscriptions to subscriber update. Closes #426. 2021-08-02 19:23:46 +05:30
Kailash Nadh
fb48477aa7 Fix SQL expressions breaking subscriber export. Closes #408 2021-07-25 22:42:54 +05:30
Kailash Nadh
868fae6ac2 Refactor subsbscription status option on the import page.
- Refactor subimporter New*() funcs to take opt structs.
- Refactor and simplify Vue code.
- Remove redundant i18n entries and use existing ones.
- Remove redundant subimporter constants and use existing ones.

- Consider 'overwrite' option for subscription status as well.
- Write Cypress integration tests for the new feature.
2021-06-06 17:33:23 +05:30
Russ Smith
7ca08f0a36 Adding a subscription status option to the import.
Ref #168
2021-06-06 17:33:23 +05:30
Kailash Nadh
3d26366620 Fix pagination query.
- Fix '?per_page=all' not working inconditional LIMIT queries.
- Fetch all lists on the UI for list dropdowns everywhere.
2021-06-04 19:47:55 +05:30
Kailash Nadh
ad0a0e0841 Add preconfirm_subscriptions=true/falsenew subs API.
Sending th optional flag as `trunue` in the POST /api/subscrirs
body will skip sending opt-iconfirmation e-mails to subscribers
and mark list subscriptions in the request a`confirmed`.
2021-04-17 13:34:37 +05:30
Kailash Nadh
708d0e0b00 Fix re-submission of public form e-mails not registering 2021-04-15 21:53:36 +05:30
Kailash Nadh
e8ad7a9adc Fix subscriber attribs update API.
Change the behaviour where not passing attribs to the update API
overwrites the attribs with empty values. This commit changes the
behaviour so that in the absence of the attribs field in the
subscriber API, the existing value in the DB is retained.
2021-03-10 21:20:26 +05:30
Kailash Nadh
2f6bd05ca0 Fix the optin-in, form re-subscribe behaviour.
If a user is already subscribed to an optin list but hasn't
confirmed, subscribing using the same e-mail id from the public
form now re-sends the optin e-mail while also showing an
appropriate message on the frontend rather than just saying
"subscribed successfully".

https://github.com/knadh/listmonk/issues/266
https://github.com/knadh/listmonk/issues/264
2021-02-13 17:55:10 +05:30
Kailash Nadh
2235d30063 Add a new public page for end users to subscribe to public lists.
In addition to generating HTML forms for selected public lists,
the form page now shows a URL (/subscription/form) that can be
publicly shared to solicit subscriptions. The page lists all
public lists in the database. This page can be disabled on the
Settings UI.
2021-01-31 16:19:39 +05:30
Kailash Nadh
68afd61024 Add support for alternate plaintext body for e-mails.
This commit removes the Go html2text lib that would automatically
convert all HTML messages to plaintext and add them as the alt
text body to outgoing e-mails. This lib also had memory leak
issues with certain kinds of HTML templates.

A new UI field for optionally adding an alt plaintext body to
a campaign is added. On enabling, it converts the HTML message in
the campaign editor into plaintext (using the textversionjs lib).

This introduces breaking changes in the campaigns table schema,
model, and template compilation.
2021-01-30 18:49:47 +05:30
Kailash Nadh
ec1c4f30ed Add subscriber export feature 2021-01-23 18:23:29 +05:30
Kailash Nadh
62182ab979 Refactor get-lists query for speed 2020-11-21 17:29:24 +05:30
Kailash Nadh
684c64ced1 Fix SQL ORDER BY compatibility issue in Postgres 12/13.
- Fix for `pq: SELECT DISTINCT ON expressions must match
  initial ORDER BY expressions` in Postgres 13.
2020-11-08 16:02:11 +05:30
Kailash Nadh
1aecd6f2e1 Add serverside sort to tables.
Lists, campaigns, and subscribers tables now support server-side
sorting from the UI. This significantly changes the internal
queries from prepared to string interpolated to support dynamic
sort params.
2020-10-24 20:00:29 +05:30
Kailash Nadh
a1aeba22bb Fix invalid link click registrations
The link_clicks.link_id table was NULLable incorrectly. Links that
do not exist should not register a tracking entry. Fix the query
and also update the schema + migration (breaking table change).
2020-10-24 14:03:37 +05:30
Kailash Nadh
1b279478fb Make individual subscriber tracking optional.
A new toggle switch in Settings -> Privacy, which is off by
default, allows campaign views (pixel) and link clicks to function
without registering the subscriber ID against view and click
events, anonymising tracking. When off, the subscriber UUIDs in
view and link tracking URLs are removed, anonymising subscriber
information from HTTP logs as well.
2020-10-18 17:49:46 +05:30
Kailash Nadh
f81d75a787 Allow deleting of all campaigns irrespective of status 2020-10-10 22:08:13 +05:30
Kailash Nadh
6cf43ea674 Add generic HTTP postback Messenger support.
This is a major feature that builds upon the `Messenger` interface
that has been in listmonk since its inception (with SMTP as the only
messenger). This commit introduces a new Messenger implementation, an
HTTP "postback", that can post campaign messages as a standard JSON
payload to arbitrary HTTP servers. These servers can in turn push them
to FCM, SMS, or any or any such upstream, enabling listmonk to be a
generic campaign messenger for any type of communication, not just
e-mails.

Postback HTTP endpoints can be defined in settings and they can be
selected on campaigns.
2020-10-10 18:52:08 +05:30
Kailash Nadh
d219f6549e Fix missing status column in import that affects auto-migrated
installations.

In v0.7.0's automatic DB migration, the default value 'enabled'
of subscribers.status was missed incorrectly, causing imports to
break on auto-upgraded versions. This is now fixed by explictly
inserting the status field in CSV imports.
2020-09-02 11:40:34 +05:30
Kailash Nadh
7ed07550ff Fix minor inconsistencies in settings.
- Add missing `app.root_url` key in migration.
- Register `/settings` handler in the backend.
- Add dummy dots in secret fields on the UI for visibility.
2020-08-07 19:28:23 +05:30
Kailash Nadh
378e8145fe Add campaign search UI. 2020-08-01 19:46:47 +05:30
Kailash Nadh
8c0804ba9f Refactor blacklist to blocklist 2020-08-01 16:45:29 +05:30
Kailash Nadh
9c8b7ad7f8 Fix broken tag clearing on lists and campaign updation 2020-08-01 13:09:52 +05:30
Kailash Nadh
942eb7c3d8 Add settings UI and "hot reload" support to the app.
This is a major breaking change that moves away from having the
entire app configuration in external TOML files to settings being
in the database with a UI to update them dynamically.

The app loads all config into memory (app settings, SMTP conf)
on boot. "Hot" replacing them is complex and it's a fair tradeoff
to instead just restart the application as it is practically
instant.

A new `settings` table stores arbitrary string keys with a JSONB
value field which happens to support arbitrary types. After every
settings update, the app gracefully releases all resources
(HTTP server, DB pool, SMTP pool etc.) and restarts itself,
occupying the same PID. If there are any running campaigns, the
auto-restart doesn't happen and the user is prompted to invoke
it manually with a one-click button once all running campaigns
have been paused.
2020-07-21 00:23:57 +05:30
Kailash Nadh
61f8fae50d Add 'overwrite?' option to bulk import.
- Fix minor UI inconsitency on import states.
- Minor refactor to importer initialisation.
2020-07-05 21:35:17 +05:30
Kailash Nadh
e7da8fa668 Fix ordering of records on the subscribers page 2020-07-05 19:23:45 +05:30
Kailash Nadh
db032d3001 Change campaign status to draft on removing schedule 2020-07-05 18:42:38 +05:30
Kailash Nadh
24192a327f Refactor and fix media uploads.
- Fix path related issues in filesystem and S3.
- Add checks for S3 "/" path prefix.
- Add support for custom S3 domain names.
- Remove obsolete `width` and `height` columns from media table (breaking)
- Add `provider` field to media table (breaking)
2020-07-05 17:35:05 +05:30
Kailash Nadh
feb5ba09be Add new dashboard (with new metrics) 2020-07-04 22:25:02 +05:30
Kailash Nadh
97583fe4b4 Rewrite frontend with Vue+Buevy and ditch React+Ant Design.
- antd+react was resulting in extremely clunky and unreadable
  spaghetti frontend code (primarily due to how antd is).
- Buefy is lighter by an order of magnitude, has excellent
  responsive views (especially tables) and usability.
- Vue's templating produces far more readable template code.
2020-07-04 00:12:14 +05:30
Kailash Nadh
3a9a2ef4ec Refactor get-campaign to accept uuid + get body 2020-04-26 15:51:26 +05:30
thejaswinipathi
c93ba78938
Changed on PR comment 2020-04-22 14:25:42 +05:30
Kailash Nadh
07856d34a2 Fix bug in e-mail scheduling 2020-03-08 15:29:53 +05:30
Kailash Nadh
442dec9341 Fix broken subscriber data export 2020-03-08 14:36:50 +05:30
Kailash Nadh
c08ca14a5b Add subscription forms 2020-03-07 20:22:01 +05:30
Kailash Nadh
6be3352f52 Refactor/merge optin-list fetch queries 2020-02-09 16:25:19 +05:30
Kailash Nadh
022b35c4a7 Add support for sending 'opt-in' campaigns.
- Campaigns now have a `type` property (regular, opt-in)
- Opt-in campaigns work for double opt-in lists and e-mail
  subscribers who haven't confirmed their subscriptions.
- Lists UI shows a 'Send opt-in campaign' optin that
  automatically creates an opt-in campaign for the list
  with a default message body that can be tweaked before
  sending the campaign.
- Primary usecase is to send opt-in campaigns to subscribers
  who are added via bulk import.

This is a breaking change. Adds a new Postgres enum type
`campaign_type` and a new column `type` to the campaigns table.
2020-02-09 11:36:15 +05:30
Kailash Nadh
a2d21a8bf0 Refactor opt-in subscriber selection queries 2020-02-09 11:36:15 +05:30
Kailash Nadh
ea302d11b7 Use default template if there isn't one in campaign creation 2020-02-09 11:36:15 +05:30
Kailash Nadh
871893a9d2 Add double opt-in support.
- Lists can now be marked as single | double optin.
- Insert subscribers to double opt-in lists send out a
  confirmation e-mail to the subscriber with a confirmation link.
- Add `{{ OptinURL }}` to template functions.

This is a breaking change. Adds a new field 'optin' to the lists
table and changes how campaigns behave. Campaigns on double opt-in
lists exclude subscribers who haven't explicitly confirmed subscriptions.

Changes the structure and behaviour of how notification e-mail routines,
including notif email template compilation,  notification callbacks for
campaign and bulk import completions.
2020-02-09 11:36:15 +05:30
Kailash Nadh
6681f189fc Fix 'send_at' option on the UI and bug in starting scheduled campaigns 2020-01-19 20:46:25 +05:30
Kailash Nadh
649d1b11f0 Fix 'send later' switch + date not updating on campaign edit 2019-08-26 23:45:18 +05:30
Kailash Nadh
31b9690d74 Remove orphan func 2019-07-21 21:22:26 +05:30
Kailash Nadh
3b7902802e Add data privacy export / wipe features (aimed at GDPR compliance).
- Toggle options to enable self-service data export and wipe
  options on the public unsubscription page. Subscribers can get
  a copy of all data on them e-mailed to them as JSON, or
  instantly wipe all their data.
- Refactor "unsubscribe" pages and URIs to "subscription".
- Add export icon to subscriber admin view.
2019-07-21 19:21:23 +05:30