Setting the aforementioned env var skips creation of the app router,
which is useful for running update.exs when the main app is already
running (otherwise there's a port conflict).
Whoogle is now eligible for Farside links. I haven't decided how I'd
like this implemented yet though. My idea is actually to refactor part
of Whoogle to work a bit better with this new "instance hopping"
paradigm -- namely, I'd like the config settings to be easily turned
into query params and appended to a farside link when a user's preferred
instance is blocked.
Query params ("/watch?v=dQw4w9WgXcQ" for instance) would previously be
lost in Farside redirects. This now includes them if they were included
in the original request.
This adds a couple of new scripts: install-crontab.sh and update.sh. The
install-crontab script does exactly that -- it installs a new crontab
that runs the update script every 5 minutes. An update script was added
to simplify the crontab a bit and to ensure the Elixir update script is
executed from the correct directory.
This introduces a way of throttling requests in a way that makes sense
for the purpose of the app. The app only supports redirecting to one
particular service when browsing, which would seldom be required more
than once per second for normal "human" browsing.
Without this, the service could easily be used to DOS multiple instances
at once. That being said, anyone concerned about someone DOS-ing
multiple instances at once should be aware that this would be trivial to
do with a simple bash script. This is simply a preventative measure to
hopefully deter people from trying to attack all public instances of
private frontends using farside.link.
Note that this throttling applies to all routes in the app, including
the homepage. This could be updated to exclude the homepage I guess,
but I'm not really sure what the use case would be for that.
Introduces a new db key "<service>-previous" to track which instance was
last selected for a particular service. This allows for filtering the
list of available instances to exclude the instance that was last
picked, to ensure a (slightly) more even distribution of traffic.
There's still the possiblity of the following scenario, however:
:service instances > 2
/:service request #1 -> instance #1
/:service request #2 -> instance #2
/:service request #3 -> instance #1
/:service request #4 -> instance #2
where there are many ignored instances for a particular service. One
possible solution would be to implement the "<service>-previous" value
to be a list, rather than a single value, and push to that list until
only one element is left in the original "instance" array after
filtering, and then delete the "<service>-previous" key.
The process of selecting a random (working) instance for a specified
service has been moved out of the router and into lib/farside.ex. Moving
forward, the router itself should have very simple and easy to follow
logic for all paths.
Service struct now defined in lib/service.ex
This makes a bit more sense now that its a shared resource, rather than
just defining it only in the update.exs script.
Now allows setting FARSIDE_TEST to skip individually fetching each
instance, and instead just adds all of them to redis instantly. This
allows for an easier time in CI builds, for both the sake of speed and
to prevent a scenario where many simultaneous builds have a noticeable
impact on actual instances.
This introduces a number of new changes:
- Services are now inserted into redis with a prefix prepended to the
key name. This allows for easier filtering to get only live instances.
- The home page now uses an eex template for displaying all live
instances for every service, determined by the last update
- A "last_updated" field was added
- farside.ex was added to contain all functionality related to querying
for instances (WIP)
- Other improvements
The name of the project is being refactored from Privacy Revolver to
Farside. The reasoning behind this is:
1. A shorter name is easier to remember
2. It can stand for "FOSS alternative redirecting service" (which I know
doesn't encapsulate all letters from "farside", but it's close enough).
This commit also includes improvements to the update script for
determining how far along the script is.
Not sure if this is the Elixir-y way to do this, but seems more logical
than hardcoding values such as redis connection.
Also went through and improved how string formatting was performed
throughout the app. Rather than "combining" <> "strings" this way, I'm
now just doing "#${variable}#{formatting}", which looks a lot cleaner.
Router has been updated with a basic /:service/*glob endpoint, which
retrieves a list of instances for the requested service and picks a
random one to forward the user to (or falls back to a default instance
if none are found).
Should probably add a check to make sure the requested service exists
first.
Rather than use a full blown framework*, adding basic routing with
Plug.Router seems to make more sense, since I'm not planning on hosting
any content through this app. The app itself will just be endpoints for
all available services that redirect the user to an available instance
for the requested service.
Note that I might change my mind about this, but that's unlikely. At
most there would just be a home page with info about available
instances, but even then that seems kinda pointless. Trying to keep this
as absolutely simple as possible.
*like Phoenix
The update script now writes the available instances to a
.update-results* file (where previous runs have "-prev" appended to the
file name). This helps to see how instance availability changes between
runs of the script when debugging overall functionality of the app.
Once a list of available URLs has been determined for a particular
service, the list is written as "service -> [list of instances]" to a
local redis connection. These can then be used in the greater routing
logic to pick a random instance from the list, or use a fallback
instance if none are determined to be available.
My initial thought for this: create a simple redis db for storing key
value pairs of instance -> list of live instances for each privacy front
end (libreddit, bibliogram, etc). A script executed on a certain
schedule would (in the background) check each instance to make sure it
isn't down or unreasonably slow. If the instance is available, add it to
a list of available instances in the db.
When a user navigates to the revolver url (something like
<url>/<service>/<...>), the app would pick a random value from the list
returned by redis.get('<service>') and forward the user to that
instance.
As a side note, this could instead load the instances json from a remote
source (like github or something) so that changes to instances don't
need to involve a redeploy of the entire app.
