Commit graph

57 commits

Author SHA1 Message Date
Kailash Nadh
8f2a08b8db Fix invalid suffix 'd' in timestring string in s3 expiry config. 2023-09-19 14:45:51 +05:30
Kailash Nadh
717a6362d2 Add Sprig and other tpl functions to static templates as well. Closes #1527. 2023-09-19 11:19:34 +05:30
Kailash Nadh
2b95c88188
Add Postmark bounce webhook support (refactor #1385) (#1485)
Co-authored-by: Thomas Siebers <tom@tsiebers.de>
2023-08-31 21:27:34 +05:30
Kailash Nadh
104c4fc993 Remove non-crossplatform syscalls from /api/about.
The only way to get precise OS information cross platform is to have
a separate file for each platform with a build tag that returns the
info, which seems excessive for this usecase.
2023-08-06 10:49:20 +05:30
Kailash Nadh
ad80c716f9 Add new privacy option 'Record opt-in IP' to record IP address of optin confirmation.
- Add new 'Subscriptions' table on the subscriber list form that shows subs,
  IP, and other data.
- Add new `meta` JSONB field to `subscriber_lsts` table.

Closes #1329.
2023-07-26 23:00:32 +05:30
Kailash Nadh
c581fe2f3a Add GET /api/about that returns useful system info. Closes #1354. 2023-06-24 13:07:13 +05:30
Kailash Nadh
3b9a0f782e
Add support for file attachments on campaigns (#1341)
- Adds support for arbitrary file uploads with an admin setting to select allowed file extensions.
- Adds support for attaching media (files) to campaigns.
2023-05-18 16:55:59 +05:30
Kailash Nadh
917696a543 Refactor Messenger interface.
Remove `messenger.go` and move the interface definition to `manager`
and the `Message` struct to the `models` package, removing superflous
and redundant message structs used in multiple places.
2023-05-08 22:43:25 +05:30
Kailash Nadh
13ad9adb8b Upgrade koanf to v2. 2023-04-10 12:45:25 +05:30
Kailash Nadh
98729f63df Refactor email domain blocklist config field in importer package. 2023-04-09 14:04:02 +05:30
Justin Beaty
476d5bebf2
Add support for publishing full content in public archive RSS feed body (#1262)
- Introduces a new option on the settings UI to optionally publish the full campaign body in
  public archive RSS feeds.

Closes #1033 

Co-authored-by: Kailash Nadh <kailash@nadh.in>
2023-04-08 09:39:10 +05:30
Justin Beaty
a95510260d
Add functions to notification templates (#1263) 2023-03-26 12:00:35 +05:30
Kailash Nadh
8985e5c24a
Add hCaptcha.com support to public subscription form. (#1152)
Bots easily bypass the simple `nonce` hack. This commit adds support
for the hcaptcha.com widget.

- New `Security` tab in the admin settings UI.
- Enable/disable CAPTCHA.
- Render CAPTCHA on the public subscription form.

Closes #1116.
2023-01-23 21:50:10 +05:30
Kailash Nadh
5d4f1ea0ad Add optional params in DB config to accept arbitrary Postgres params. Closes #1016. 2022-12-25 16:28:19 +05:30
Kailash Nadh
818f2c9d8e Add public archive on/off toggle to settings. 2022-11-10 23:30:53 +05:30
Kailash Nadh
438568eeb0 Add global site name setting to render name on public pages. 2022-11-10 23:30:53 +05:30
Kailash Nadh
23fb178ec4 Add subscription/archive links to public pages. 2022-11-10 23:30:53 +05:30
Kailash Nadh
9add728b08 WIP: Add support for publishing campaigns to publish archives. 2022-11-10 23:30:11 +05:30
Kailash Nadh
3b0083190e Add ability for subscribers to manage preferences on the unsub form.
- Ability to change name.
- Ability to unsubscribe from individual lists.
- Toggle option to enable this in Admin Settings -> Privacy.

Closes #455.
2022-10-29 15:23:28 +05:30
Kailash Nadh
281c47198c Fix go-for-loop reference bug in template caching. 2022-10-21 17:59:56 +05:30
Kailash Nadh
edb4c9168d Improve HTML check in notif template loading. Closes #903. 2022-09-10 12:21:53 +05:30
Kailash Nadh
f266f93cc8 Add Safe() template function to notif templates. 2022-09-10 12:20:27 +05:30
Kailash Nadh
3550d5453d Fix incorrect analytics count. Closes #712. 2022-07-30 19:01:20 +05:30
Kailash Nadh
4a6e041ca8 Don't break boot on tx template compilation errors. 2022-07-09 10:36:12 +05:30
Kailash Nadh
463e92d1e1 Add transactional (tx) messaging capability.
This commit adds a new API `POST /api/tx` that sends an ad-hoc message
to a subscriber based on a pre-defined transactional template. This is
a large commit that adds the following:

- New campaign / tx template types on the UI. tx templates have an
  additional subject field.
- New fields `type` and `subject` to the templates table.
- Refactor template CRUD operations and models.
- Refactor template func assignment in manager.
- Add pre-compiled template caching to manager runtime.
- Pre-compile all tx templates into memory on program boot to avoid
  expensive template compilation on ad-hoc tx messages.
2022-07-09 10:36:12 +05:30
Kailash Nadh
aa19771307 Refactor bounces package to remove db/queries dependency.
Instead of passing a DB/SQL statement references, instead pass a
callback that inserts a bounce into the DB via the `core` package.
2022-05-03 10:50:33 +05:30
Kailash Nadh
b5cd9498b1 Refactore all CRUD functions to a new core package.
This is a long pending refactor. All the DB, query, CRUD, and related
logic scattered across HTTP handlers are now moved into a central
`core` package with clean, abstracted methods, decoupling HTTP
handlers from executing direct DB queries and other business logic.

eg: `core.CreateList()`, `core.GetLists()` etc.

- Remove obsolete subscriber methods.
- Move optin hook queries to core.
- Move campaign methods to `core`.
- Move all campaign methods to `core`.
- Move public page functions to `core`.
- Move all template functions to `core`.
- Move media and settings function to `core`.
- Move handler middleware functions to `core`.
- Move all bounce functions to `core`.
- Move all dashboard functions to `core`.
- Fix GetLists() not honouring type
- Fix unwrapped JSON responses.
- Clean up obsolete pre-core util function.
- Replace SQL array null check with cardinality check.
- Fix missing validations in `core` queries.
- Remove superfluous deps on internal `subimporter`.
- Add dashboard functions to `core`.
- Fix broken domain ban check.
- Fix broken subscriber check middleware.
- Remove redundant error handling.
- Remove obsolete functions.
- Remove obsolete structs.
- Remove obsolete queries and DB functions.
- Document the `core` package.
2022-05-03 10:50:29 +05:30
Yatish Mehta
6c903239dd Fix typos 2022-02-13 08:54:39 -08:00
Kailash Nadh
2614b072f2 Refactor campaign analytics to show unique / non-unique data.
The analytics page showed non-unique counts for views and clicks which
was misleading and source of confusion: #522, #561, #571, #676, #680
This commit changes this behaviour to pull unique views and clicks when
individual subscriber tracking is turned on in settings, and non-unique
counts when it is turned off (as `subscriber_id` in `campaign_views`
and `link_clicks` will be NULL, rendering unique queries dysfunctional).

This commit changes the stats SQL queries to use string interpolation
to either to SELECT `*` or `DISTINCT subscriber_id` on app boot based
on the setting in the DB. This involves significant changes to how
queries are read and prepared on init.

- Refactor `initQueries()` to `readQueries()` and `prepareQueries()`.
- Read queries first before preparing.
- Load settings from the DB using the read settings query.
- Prepare queries next. Use the privacy setting from the DB to apply
  string interpolation to the analytics queries to pull
  unique/non-unique before preparing the queries.

On the UI:
- Show a note on the analytics page about unique/non-unique counts.
- Hide the % donut charts on the analytics page in non-unique mode.

Closes #676, closes #680
2022-02-01 23:40:03 +05:30
Kailash Nadh
fabe06e339 Add support for custom CSS/JS in settings for admin and public pages.
This feature was originally authored by @sweetppro in PR #438.
However, since the PR ended up in an unclean state with
multiple master merges (instead of rebase) from the upstream, there are
several commits that are out of order and can can no longer be be
squashed for a clean feature merge.

This commit aggregates the changes from the original PR and applies the
following fixes on top of it.

- Add custom admin JS box to appearance UI.
- Refactor i18n language strings.
- Add handlers and migrations for the new `appearance.admin.custom_js`
  field.
- Fix migration version to `v2.1.0`
- Load custom appearance CSS/JS bytes into global constants during boot
  instead of making a DB call on every request.
- Fix and canonicalize URIs from `/api/custom*` to `/public/*.css`
  and `/admin/*.css`. Add proxy paths to yarn proxy config.
- Remove redundant HTTP handlers for different custom appearance files
  and refactor into a single handler `serveCustomApperance()`
- Fix content-type and UTF8 encoding headers for different file types.
- Fix incorrect registration of public facing custom CSS/JS handlers
  in the authenticated admin URI group.
- Fix merge conflicts in `Settings.vue`.
- Minor HTML and style fixes.
- Remove the `AppearanceEditor` component and use the existing
  `HTMLEditor` component instead.
- Add `language` prop to the `HTMLEditor` component.

Co-authored-by: SweetPPro <sweetppro@users.noreply.github.com>
2021-12-18 15:38:42 +05:30
Kailash Nadh
e9709e54ee Upgrade labstack/echo webserver to major version v4.
- echo is now on v4 with major changes including a few breaking changes
- bind() behaviour is now strict. JSON / form etc. unmarshalling of
  request data need appropriate `json`, `form` tags. Missing tags for
  the public subscription page is added in this commit.
- This also closes #602.
2021-12-09 20:51:07 +05:30
Kailash Nadh
4cb5eb782f Fix settings form input validation.
- Fix settings UI form submit button.
- Validate upload URI. Closes #621.
2021-12-09 19:30:15 +05:30
Kailash Nadh
11010393d8 Add "passive" mode with --passive flag.
Every listmonk instance scans the DB periodically to look for
running campaigns to process. This made running multiple instances of
listmonk impractical as they would all pick up the same running
campaign and process them, resulting in duplicate e-mails.

This commit adds a `--passive` flag to the binary that runs listmonk
in a "passive" mode where campaign processing is disabled. This allows
multiple instances of listmonk to be run to handle different kinds of
requests if there is a requirement (scale/traffic?). It is important
to note that there should only be one non-passive instance running at
any given time. If distributed campaign processing is ever considered,
this will change.
2021-10-29 14:40:22 +05:30
Kailash Nadh
b290d271c0 Add support for plaintext system e-mail templates.
If `<!doctype html>` is not found in static/email-templates/base.html,
all system e-mail templates are assumed to be plaintext and go out
as content-type: plaintext e-mails. With this, all HTML tags can
be stripped out of the system e-mail templates (while maintaining
Go template tags and logic) to have plaintext system e-mail templates.

Closes #546
2021-10-28 20:09:06 +05:30
Kailash Nadh
7aee36eab1 Add support for blocklisting e-mail domains.
E-mails in the domain blocklist are disallowed on the admin UI, public
subscription forms, API, and in the bulk importer.

- Add blocklist setting that takes a list of multi-line domains on the
  Settings -> Privacy UI.
- Refactor e-mail validation in subimporter to add blocklist checking
  centrally.
- Add Cypress testr testing domain blocklist behaviour on admin
  and non-admin views.

Closes #336.
2021-09-25 15:39:09 +05:30
Kailash Nadh
e71115db26 Add option to toggle sending opt-in confirmation. Closes #363. 2021-09-25 10:38:13 +05:30
Kailash Nadh
bb340b8785 Refactor frontend build and name space all admin URIs behind /admin/.
- Namespace all admin UI URLs behind `/admin/*`.
  This breaks the current admin UI URLs.
- Make Vue output build assets to `frontend/dist/*` instead of
  `frontend/dist/frontend`.
- Namespace Vue static assets to `/admin/static/*`.

This commit reduces the cofusing and convoluted Vue+WebPack build URI
and static path schemes. In addition, it removes ambiguity in URLs
where non-UI URLs like `/public`, `/api`, `/webhooks` etc. were in the
same name space as UI URLs like `/campaigns`, `/lists` etc. Now all UI
URLs are behind `/admin/`, also simplifying security rules for proxies.
2021-09-23 19:21:35 +05:30
Kailash Nadh
3847c67087 Add --idempotent to make --install idempotent 2021-08-22 20:09:39 +05:30
Kailash Nadh
d6d1883587 Add custom S3 backend support (eg: Minio) to media uploads
- Introduce a new S3 backend URL on the settings UI
- Add DB migration to populate S3 URL for existing S3 settings
- Refactor and fix URL formatting

Closes #139
2021-08-15 16:09:00 +05:30
Kailash Nadh
1ae98699e7 Add support for bounce processing.
- Blocklist or unsubscribe subscribers based on a bounce threshold
- Add /bounces UI for viewing bounces and in the subscriber view
- Add settings UI for managing bounce settings
- Add support for scanning POP3 bounce mailboxes
- Add a generic webhook for posting custom bounces at /webhooks/bounce
- Add SES bounce webhook support at /webhooks/services/ses
- Add Sendgrid bounce webhook support at /webhooks/services/sendgrid
2021-08-14 15:35:29 +05:30
Kailash Nadh
82735bba69 Refactor behaviour of loading static files from disk vs. embedding.
Ref: https://github.com/knadh/listmonk/issues/409

- Introduce `main.appDir` and `main.fronendDir` Go compile-time flags
  to hardcode custom paths for loading frontend assets
  (frontend/dist/frontend in the repo after build) and app assets
  (queries.sql, schema.sql, config.toml.sample) in environments where
  embedding files in the binary is not feasible.
  These default to CWD unless explicitly set during compilation.

- Fix the Vue favicon path oddity by copying the icon into the built
  frontend dir in the `make-frontend` step.
2021-07-11 10:46:45 +05:30
Kailash Nadh
95a81d17ce Add option on UI to toggle update checks.
Closes #326
2021-05-16 16:54:55 +05:30
Kailash Nadh
d695bb34cc Prioritise --static-dir on init when no assets are embedded.
When no static assets are found on init, i.e., when a binary without
stuffbin assets are loaded, the app looks for all necessary static
files in the working dir, including the `./static/*` path which renders
the `--static-dir` flag irrelevant.

This patch gives `--static-dir`, if set, precedence over `./static/*`
when loading assets from the working dir when a binary is not stuffed
with static files.

Closes #340.
2021-05-16 15:53:30 +05:30
Kailash Nadh
c479a90c42 Add support for loading external i18n language files.
The new `--i18n-dir` directory allows the loading of an external
directory of i18n JSON files, milar to have `--static-dir`
works. New languages can be added and existing language files
can be customized this way.

This commit changes file loading behaviour so that invalid or
non-existent don't halt the execution of the app completely but
merely throw a warning and continue with the default (en) lang.
2021-04-17 14:26:56 +05:30
Kailash Nadh
570a81f966 WIP: Add tests 2021-04-10 12:26:33 +05:30
Kailash Nadh
b950d2f4ff Refactor fetching of server config and settings.
The earlier approach of loading `/api/config.js` as a script on
initial page load with the necessary variables to init the UI is
ditched. Instead, it's now `/api/config` and `/api/settings` like
all other API calls. On load of the frontend, these two resources
are fetched and the frontend is initialised.
2021-02-13 12:34:36 +05:30
Kailash Nadh
2235d30063 Add a new public page for end users to subscribe to public lists.
In addition to generating HTML forms for selected public lists,
the form page now shows a URL (/subscription/form) that can be
publicly shared to solicit subscriptions. The page lists all
public lists in the database. This page can be disabled on the
Settings UI.
2021-01-31 16:19:39 +05:30
Kailash Nadh
4bfdda9b91 Load a default (en) language underneath the selected language.
The default language (en) is loaded first and the selected
language is loaded on top of it so that missing translation keys
in the selected language will have the original English strings
available on the UI.
2021-01-26 21:59:27 +05:30
Kailash Nadh
027261793f Add support for rate limiting messages with a sliding window.
Certain SMTP hosts limit the total number of messages that can be
sent within a window, for instance, X / 24 hours. The concurrency
and message rate controls can only limit that to a max of
1 messages / second, without a global cap.

This commit introduces a simple sliding window rate limit feature
that counts the number of messages sent in a specific window, and
upon reaching that limit, waits for the window to reset before
any more messages are pushed out globally across any number of
campaigns.

Context: https://github.com/knadh/listmonk/issues/119
2021-01-24 12:19:26 +05:30
Kailash Nadh
ee4fb7182f Refactor i18n name and fix the L() function in public HTML templates 2021-01-23 19:34:30 +05:30