Instance URLs and API URLs are not always the same, and require
different testing strategies. This allows the two types of URLs for an
instance, as well as the testing path, to be separated by a "|" character.
When Farside runs the instance checker, it will split instance strings
by that character and perform separate tests. Instances without a
separate API URL will only be tested once against their primary instance
URL.
Currently the only service using this paradigm is Piped, but allows for
others using the same approach to be added/tested in Farside in the
future.
Closes#115
This introduces a simple way of compiling Farside to a somewhat portable,
standalone binary. The resulting binary isn't completely portable since it
depends on the C runtime of the host system. As a result, it's advised to use
systems with older library versions when compiling for true portability.
Closes#50
Co-authored-by: Jason Clark <mithereal@gmail.com>
This removes the dependency on Redis for core app functionality. Rather
than using the key/value store provided by Redis, Farside now uses a
key/val store provided by [cubdb](https://github.com/lucaong/cubdb) for
identical functionality but without reliance on a non-Elixir service.
This solution was chosen instead of ets, because storing instance data
in memory leads to a period of broken functionality whenever the app
restarts and hasn't re-populated instance data yet. It was also chosen
instead of dets, because the documentation for dets was pretty hard to
understand at first glance.
Tests and the CI build were updated to reflect the removed dependency on
Redis.
New environment variable `FARSIDE_DATA_DIR` can be used to point to a
directory where the instance data can be stored by cubdb.
Co-authored-by: Jason Clark <mithereal@gmail.com>
Rather than requiring a traditional crontab install, the app now
leverages quantum-core (link below) to schedule the instance update/sync
task every 5 minutes. Some updates as a result:
- The new job is scheduled at runtime in server.ex.
- The update.exs script was refactored to be compiled along with the
rest of the app as instances.ex.
- Scheduler and Server modules were added for creating and executing
the new update task
- All shell scripts were removed, as they are no longer needed
https://github.com/quantum-elixir/quantum-core
This introduces a way of throttling requests in a way that makes sense
for the purpose of the app. The app only supports redirecting to one
particular service when browsing, which would seldom be required more
than once per second for normal "human" browsing.
Without this, the service could easily be used to DOS multiple instances
at once. That being said, anyone concerned about someone DOS-ing
multiple instances at once should be aware that this would be trivial to
do with a simple bash script. This is simply a preventative measure to
hopefully deter people from trying to attack all public instances of
private frontends using farside.link.
Note that this throttling applies to all routes in the app, including
the homepage. This could be updated to exclude the homepage I guess,
but I'm not really sure what the use case would be for that.
This introduces a number of new changes:
- Services are now inserted into redis with a prefix prepended to the
key name. This allows for easier filtering to get only live instances.
- The home page now uses an eex template for displaying all live
instances for every service, determined by the last update
- A "last_updated" field was added
- farside.ex was added to contain all functionality related to querying
for instances (WIP)
- Other improvements
Rather than use a full blown framework*, adding basic routing with
Plug.Router seems to make more sense, since I'm not planning on hosting
any content through this app. The app itself will just be endpoints for
all available services that redirect the user to an available instance
for the requested service.
Note that I might change my mind about this, but that's unlikely. At
most there would just be a home page with info about available
instances, but even then that seems kinda pointless. Trying to keep this
as absolutely simple as possible.
*like Phoenix
Once a list of available URLs has been determined for a particular
service, the list is written as "service -> [list of instances]" to a
local redis connection. These can then be used in the greater routing
logic to pick a random instance from the list, or use a fallback
instance if none are determined to be available.
My initial thought for this: create a simple redis db for storing key
value pairs of instance -> list of live instances for each privacy front
end (libreddit, bibliogram, etc). A script executed on a certain
schedule would (in the background) check each instance to make sure it
isn't down or unreasonably slow. If the instance is available, add it to
a list of available instances in the db.
When a user navigates to the revolver url (something like
<url>/<service>/<...>), the app would pick a random value from the list
returned by redis.get('<service>') and forward the user to that
instance.
As a side note, this could instead load the instances json from a remote
source (like github or something) so that changes to instances don't
need to involve a redeploy of the entire app.
