Commit graph

50 commits

Author SHA1 Message Date
Kailash Nadh
8f2a08b8db Fix invalid suffix 'd' in timestring string in s3 expiry config. 2023-09-19 14:45:51 +05:30
Kailash Nadh
2b95c88188
Add Postmark bounce webhook support (refactor #1385) (#1485)
Co-authored-by: Thomas Siebers <tom@tsiebers.de>
2023-08-31 21:27:34 +05:30
Kailash Nadh
ad80c716f9 Add new privacy option 'Record opt-in IP' to record IP address of optin confirmation.
- Add new 'Subscriptions' table on the subscriber list form that shows subs,
  IP, and other data.
- Add new `meta` JSONB field to `subscriber_lsts` table.

Closes #1329.
2023-07-26 23:00:32 +05:30
Kailash Nadh
d359ad27aa Refactor media management.
- Change tiled UI to table UI.
- Add support for search and pagination.
- Important: This breaks the `GET /api/media` API to introduce pagination
  fields. Media items are now moved into `{ data: results[] }`.
2023-05-21 15:19:12 +05:30
Kailash Nadh
3b9a0f782e
Add support for file attachments on campaigns (#1341)
- Adds support for arbitrary file uploads with an admin setting to select allowed file extensions.
- Adds support for attaching media (files) to campaigns.
2023-05-18 16:55:59 +05:30
Kailash Nadh
5fc28a733c Add support for variable bounce processing actions.
- Add support for `complaint` to the SES bounce processor.
- Add support for `hard/soft` to Sendgrid bounce processor.
- Add new bounce actions `None` and `Unsubscribe`.
- Add per type (`soft/hard/complaint`) bounce rule configuration to
  admin settings UI.
- Refactor Cypress bounce tests.
2023-04-11 11:33:40 +05:30
Justin Beaty
476d5bebf2
Add support for publishing full content in public archive RSS feed body (#1262)
- Introduces a new option on the settings UI to optionally publish the full campaign body in
  public archive RSS feeds.

Closes #1033 

Co-authored-by: Kailash Nadh <kailash@nadh.in>
2023-04-08 09:39:10 +05:30
Kailash Nadh
8985e5c24a
Add hCaptcha.com support to public subscription form. (#1152)
Bots easily bypass the simple `nonce` hack. This commit adds support
for the hcaptcha.com widget.

- New `Security` tab in the admin settings UI.
- Enable/disable CAPTCHA.
- Render CAPTCHA on the public subscription form.

Closes #1116.
2023-01-23 21:50:10 +05:30
Kailash Nadh
818f2c9d8e Add public archive on/off toggle to settings. 2022-11-10 23:30:53 +05:30
Kailash Nadh
438568eeb0 Add global site name setting to render name on public pages. 2022-11-10 23:30:53 +05:30
Kailash Nadh
9add728b08 WIP: Add support for publishing campaigns to publish archives. 2022-11-10 23:30:11 +05:30
Kailash Nadh
ef1f84ee7c Add new description field to lists. Closes #925. 2022-11-01 21:04:35 +05:30
Kailash Nadh
3b0083190e Add ability for subscribers to manage preferences on the unsub form.
- Ability to change name.
- Ability to unsubscribe from individual lists.
- Toggle option to enable this in Admin Settings -> Privacy.

Closes #455.
2022-10-29 15:23:28 +05:30
Kailash Nadh
c38100427d Add arbitrary meta field to media. Closes #938.
- Add new `meta` JSONB field to `media` table.
- Start storing image width and height as meta with media uploads.
2022-10-02 23:04:51 +05:30
Kailash Nadh
463e92d1e1 Add transactional (tx) messaging capability.
This commit adds a new API `POST /api/tx` that sends an ad-hoc message
to a subscriber based on a pre-defined transactional template. This is
a large commit that adds the following:

- New campaign / tx template types on the UI. tx templates have an
  additional subject field.
- New fields `type` and `subject` to the templates table.
- Refactor template CRUD operations and models.
- Refactor template func assignment in manager.
- Add pre-compiled template caching to manager runtime.
- Pre-compile all tx templates into memory on program boot to avoid
  expensive template compilation on ad-hoc tx messages.
2022-07-09 10:36:12 +05:30
Manu
9ed0ae747b Use empty logo_url as default 2022-02-16 12:09:38 +04:00
Kailash Nadh
64d2c5aeb9 Add support for custom public S3 URLs. Closes #505. 2022-01-15 21:20:32 +05:30
Kailash Nadh
583dab4bc6 Add support for per-campaign custom headers.
- Add new `headers[]` column to the campain table.
- Add new headers box to the campaign UI that takes a JSON array of
  custom headers like the headers on the SMTP settings UI.
- Headers are added to e-mails and messenger postback webhooks.
- Add cypress tests.

Closes #514.
2022-01-04 22:27:40 +05:30
Kailash Nadh
dd061f56d4 Add support for direct SSL/TLS (non-STARTTLS) SMTP connections.
- Add support for TLS in `smtppool` (v0.4.0) and upgrade the lib.
- Change `tls_enabled: bool` in the settings table to string
  `tls_type: STARTTLS|TLS|none` and on the settings UI.
- Add DB migrations and schema changes to apply the field change.

Closes #504.
2022-01-03 19:28:36 +05:30
Kailash Nadh
fabe06e339 Add support for custom CSS/JS in settings for admin and public pages.
This feature was originally authored by @sweetppro in PR #438.
However, since the PR ended up in an unclean state with
multiple master merges (instead of rebase) from the upstream, there are
several commits that are out of order and can can no longer be be
squashed for a clean feature merge.

This commit aggregates the changes from the original PR and applies the
following fixes on top of it.

- Add custom admin JS box to appearance UI.
- Refactor i18n language strings.
- Add handlers and migrations for the new `appearance.admin.custom_js`
  field.
- Fix migration version to `v2.1.0`
- Load custom appearance CSS/JS bytes into global constants during boot
  instead of making a DB call on every request.
- Fix and canonicalize URIs from `/api/custom*` to `/public/*.css`
  and `/admin/*.css`. Add proxy paths to yarn proxy config.
- Remove redundant HTTP handlers for different custom appearance files
  and refactor into a single handler `serveCustomApperance()`
- Fix content-type and UTF8 encoding headers for different file types.
- Fix incorrect registration of public facing custom CSS/JS handlers
  in the authenticated admin URI group.
- Fix merge conflicts in `Settings.vue`.
- Minor HTML and style fixes.
- Remove the `AppearanceEditor` component and use the existing
  `HTMLEditor` component instead.
- Add `language` prop to the `HTMLEditor` component.

Co-authored-by: SweetPPro <sweetppro@users.noreply.github.com>
2021-12-18 15:38:42 +05:30
Kailash Nadh
7aee36eab1 Add support for blocklisting e-mail domains.
E-mails in the domain blocklist are disallowed on the admin UI, public
subscription forms, API, and in the bulk importer.

- Add blocklist setting that takes a list of multi-line domains on the
  Settings -> Privacy UI.
- Refactor e-mail validation in subimporter to add blocklist checking
  centrally.
- Add Cypress testr testing domain blocklist behaviour on admin
  and non-admin views.

Closes #336.
2021-09-25 15:39:09 +05:30
Kailash Nadh
e71115db26 Add option to toggle sending opt-in confirmation. Closes #363. 2021-09-25 10:38:13 +05:30
Kailash Nadh
8733b205a0 Refactor SQL schema and queries for performance improvements.
- Add indexes.
- Refactor dashboard charts and view/click count queries.
  (~10x speed bump on a setup of 7mn subscribers and 80mn views)
- Refactor get subscriber queries.
  (~10x speed bump on 7mn subscribers)
- Make subscriber UI issue an equality query for email seach strings.
2021-09-18 17:25:08 +05:30
Kailash Nadh
fd8f5a96c9 Add missing bounce_type to v2 migration. 2021-09-17 20:18:53 +05:30
Kailash Nadh
9302dfbd56 Add missing id (pkey) to analytics tables for faster queries 2021-09-17 20:11:45 +05:30
Kailash Nadh
d6d1883587 Add custom S3 backend support (eg: Minio) to media uploads
- Introduce a new S3 backend URL on the settings UI
- Add DB migration to populate S3 URL for existing S3 settings
- Refactor and fix URL formatting

Closes #139
2021-08-15 16:09:00 +05:30
Kailash Nadh
923b882f05 Add migration to remove obsolete subscribers.campaigns field 2021-08-14 17:23:05 +05:30
Kailash Nadh
1ae98699e7 Add support for bounce processing.
- Blocklist or unsubscribe subscribers based on a bounce threshold
- Add /bounces UI for viewing bounces and in the subscriber view
- Add settings UI for managing bounce settings
- Add support for scanning POP3 bounce mailboxes
- Add a generic webhook for posting custom bounces at /webhooks/bounce
- Add SES bounce webhook support at /webhooks/services/ses
- Add Sendgrid bounce webhook support at /webhooks/services/sendgrid
2021-08-14 15:35:29 +05:30
Kailash Nadh
95a81d17ce Add option on UI to toggle update checks.
Closes #326
2021-05-16 16:54:55 +05:30
Kailash Nadh
1e59d53135 Add markdown support to campaign content. 2021-04-14 12:26:09 +05:30
Kailash Nadh
2235d30063 Add a new public page for end users to subscribe to public lists.
In addition to generating HTML forms for selected public lists,
the form page now shows a URL (/subscription/form) that can be
publicly shared to solicit subscriptions. The page lists all
public lists in the database. This page can be disabled on the
Settings UI.
2021-01-31 16:19:39 +05:30
Kailash Nadh
68afd61024 Add support for alternate plaintext body for e-mails.
This commit removes the Go html2text lib that would automatically
convert all HTML messages to plaintext and add them as the alt
text body to outgoing e-mails. This lib also had memory leak
issues with certain kinds of HTML templates.

A new UI field for optionally adding an alt plaintext body to
a campaign is added. On enabling, it converts the HTML message in
the campaign editor into plaintext (using the textversionjs lib).

This introduces breaking changes in the campaigns table schema,
model, and template compilation.
2021-01-30 18:49:47 +05:30
Kailash Nadh
027261793f Add support for rate limiting messages with a sliding window.
Certain SMTP hosts limit the total number of messages that can be
sent within a window, for instance, X / 24 hours. The concurrency
and message rate controls can only limit that to a max of
1 messages / second, without a global cap.

This commit introduces a simple sliding window rate limit feature
that counts the number of messages sent in a specific window, and
upon reaching that limit, waits for the window to reset before
any more messages are pushed out globally across any number of
campaigns.

Context: https://github.com/knadh/listmonk/issues/119
2021-01-24 12:19:26 +05:30
Kailash Nadh
3498a727f5 WIP: Add i18n support 2021-01-17 20:59:01 +05:30
Kailash Nadh
a1aeba22bb Fix invalid link click registrations
The link_clicks.link_id table was NULLable incorrectly. Links that
do not exist should not register a tracking entry. Fix the query
and also update the schema + migration (breaking table change).
2020-10-24 14:03:37 +05:30
Kailash Nadh
1b279478fb Make individual subscriber tracking optional.
A new toggle switch in Settings -> Privacy, which is off by
default, allows campaign views (pixel) and link clicks to function
without registering the subscriber ID against view and click
events, anonymising tracking. When off, the subscriber UUIDs in
view and link tracking URLs are removed, anonymising subscriber
information from HTTP logs as well.
2020-10-18 17:49:46 +05:30
Kailash Nadh
5fb7c6cfb0 Add support for automatic, idempotent DB migrations
- On boot, the app now checks if the DB version matches its
  expected version and refuses to start if there are pending
  migrations to be run.
- The new `--upgrade` flag runs data migrations from the last
  recorded migration (in the settings table) to the latest one
  in the binary.
- Migrations are DB/arbitrary logic functions in .go files in
  internal/migrations.
- All migration functions are idempotent.
2020-08-03 19:02:23 +05:30
Kailash Nadh
ec097909db Add support for List-Unsubscribe header.
- Added as a setting in the settings UI.
- Refactor Messenger.Push() method to accept messenger.Message{}
  instead of a growing number of positional arguments.
2020-08-01 17:54:51 +05:30
Kailash Nadh
e4f233e2e5 Add missing root_url to settings UI 2020-08-01 16:53:29 +05:30
Kailash Nadh
8c0804ba9f Refactor blacklist to blocklist 2020-08-01 16:45:29 +05:30
Kailash Nadh
942eb7c3d8 Add settings UI and "hot reload" support to the app.
This is a major breaking change that moves away from having the
entire app configuration in external TOML files to settings being
in the database with a UI to update them dynamically.

The app loads all config into memory (app settings, SMTP conf)
on boot. "Hot" replacing them is complex and it's a fair tradeoff
to instead just restart the application as it is practically
instant.

A new `settings` table stores arbitrary string keys with a JSONB
value field which happens to support arbitrary types. After every
settings update, the app gracefully releases all resources
(HTTP server, DB pool, SMTP pool etc.) and restarts itself,
occupying the same PID. If there are any running campaigns, the
auto-restart doesn't happen and the user is prompted to invoke
it manually with a one-click button once all running campaigns
have been paused.
2020-07-21 00:23:57 +05:30
Kailash Nadh
24192a327f Refactor and fix media uploads.
- Fix path related issues in filesystem and S3.
- Add checks for S3 "/" path prefix.
- Add support for custom S3 domain names.
- Remove obsolete `width` and `height` columns from media table (breaking)
- Add `provider` field to media table (breaking)
2020-07-05 17:35:05 +05:30
Kailash Nadh
022b35c4a7 Add support for sending 'opt-in' campaigns.
- Campaigns now have a `type` property (regular, opt-in)
- Opt-in campaigns work for double opt-in lists and e-mail
  subscribers who haven't confirmed their subscriptions.
- Lists UI shows a 'Send opt-in campaign' optin that
  automatically creates an opt-in campaign for the list
  with a default message body that can be tweaked before
  sending the campaign.
- Primary usecase is to send opt-in campaigns to subscribers
  who are added via bulk import.

This is a breaking change. Adds a new Postgres enum type
`campaign_type` and a new column `type` to the campaigns table.
2020-02-09 11:36:15 +05:30
Kailash Nadh
871893a9d2 Add double opt-in support.
- Lists can now be marked as single | double optin.
- Insert subscribers to double opt-in lists send out a
  confirmation e-mail to the subscriber with a confirmation link.
- Add `{{ OptinURL }}` to template functions.

This is a breaking change. Adds a new field 'optin' to the lists
table and changes how campaigns behave. Campaigns on double opt-in
lists exclude subscribers who haven't explicitly confirmed subscriptions.

Changes the structure and behaviour of how notification e-mail routines,
including notif email template compilation,  notification callbacks for
campaign and bulk import completions.
2020-02-09 11:36:15 +05:30
Kailash Nadh
fe9102120c Remove stub user features and handlers. 2019-06-26 16:22:47 +05:30
Kailash Nadh
08bc6bc67c Refactor SQL schema and add missing indexes 2018-12-26 15:31:30 +05:30
Kailash Nadh
f2c09e716c Refactor the import process
- Add 'Subscribe' and 'Blacklist' modes to the importer
- Removed 'override status' and the support for the 'status' field in import files
2018-11-05 16:59:09 +05:30
Kailash Nadh
6c5cf0da7a Add support for campaign view tracking with {{ TrackView }} pixel tag 2018-11-02 13:20:32 +05:30
Kailash Nadh
81953d68d0 - Refactor and move template compilation from runner to models.Campaign to support adhoc template funcs
- Add support for {{ Track "https://url.com" }} in templates to register and track links
2018-10-31 18:24:21 +05:30
Kailash Nadh
3ab21383b1 Fresh start 2018-10-25 19:21:47 +05:30