Custom and decentralized self-hostable search engine https://infinitydecentralized.com
Find a file
2022-01-29 16:08:01 -06:00
helpers Added everything to remote 2021-02-05 13:53:28 -06:00
search_engines Added everything to remote 2021-02-05 13:53:28 -06:00
static Added Wolfram to external search options 2022-01-29 15:49:35 -06:00
templates Fixed result link spacing 2022-01-29 16:08:01 -06:00
website_blueprints Added an easy web monetization integration 2021-02-19 18:00:41 -06:00
__init__.py Added everything to remote 2021-02-05 13:53:28 -06:00
admin_operations.py Added everything to remote 2021-02-05 13:53:28 -06:00
crawler.py Fixed possible bug for sites_to_crawl.txt files that ended with a newline 2021-03-14 14:25:05 -05:00
crawler_pid.txt Added everything to remote 2021-02-05 13:53:28 -06:00
Endpoints.md Update Endpoints.md 2021-04-20 22:58:43 +00:00
flask_admin_server.py Added everything to remote 2021-02-05 13:53:28 -06:00
flask_app_server.py Removed the auto redirect to https 2021-03-14 18:44:29 -05:00
LICENSE Added everything to remote 2021-02-05 13:53:28 -06:00
README.md Small wording change in README 2021-03-22 12:15:53 -05:00
requirements.txt Added more details to the README and edited the requirements.txt and requirements_strict.txt files 2021-03-22 12:13:23 -05:00
requirements_strict.txt Added more details to the README and edited the requirements.txt and requirements_strict.txt files 2021-03-22 12:13:23 -05:00
robots_parser.py Created a robot parser that supports * and $ 2021-02-11 11:22:51 -06:00
session_cookie.txt Added everything to remote 2021-02-05 13:53:28 -06:00
sites_to_crawl.txt Added everything to remote 2021-02-05 13:53:28 -06:00
url_analysis.py Error catching 2021-02-19 18:21:32 -06:00
wsgi.py Added everything to remote 2021-02-05 13:53:28 -06:00

Infinity Decentralized

banner

Infinity Decentralized is a decentralized search engine and crawler that can get results from its own instance and from any other Infinity instance on the internet.

How To Find Other Infinity Instances

On our GitLab repository at Endpoints.md, we try to keep it as updated as possible with instances that are currently running. If you know of more, please let us know and we'll add them to this list!

Prerequisites

A computer/server that can run Python3, ElasticSearch, and MongoDB. If you want to use Elastic Search and MongoDB from a cloud location, then all that you would need is Python3.

Setup

Install:

Then when you are in this directory, run the following command in your terminal:

pip3 install -r requirements.txt

Usage

The User Interface

To start the user interface, run

python3 -m wsgi

and you should be able to access the interface at http://localhost:5000.

Note: If you are getting package errors when you run this, you may need to run this command first and then try running the program again:

pip3 install -r requirements_strict.txt

The crawler

There are two ways to manage the crawler and how it works.

Option 1

For an easy to use web interface, just run

python3 -m flask_admin_server

and you should be able to access an admin interface at http://localhost:5001 where you can control the crawler and update some of its settings.

Option 2

If you want to control the crawler directly from the command line, then all you will need to do is manually edit the site_to_crawl.txt file with a new url on each line and then run crawler.py:

# To just run it in your terminal
python3 -m crawler
# If you want to run it in the background, use this one below instead
nohup python3 -u -m crawler > /dev/null &

You would only really need to run the "python3 - m crawler" command to make sure that the crawler is actually working. Then, the second one will run in the background without any output. when you want to stop the crawler, you'll need to run the "stop_crawler" function from the admin_operations.py file.

The admin_operations.py file has many other functions that you can use to handle the crawler and index like adding and removing urls, getting the crawler status, and resetting the index.

Deployment

Since this is only a Flask app, you would want to deploy it with a web server like Nginx or Apache. The Flask documentation and tutorials about this can be found here.

If you are using Ubuntu, this Digital Ocean Article has start to finish documentation for deploying a Flask app with Nginx. It also includes (optional) instructions on how to secure your server with HTTPS.

Monetization

Infinity Decentralized has default support for monetizing through Coil with the Web Monetization Standard.

Once you have a web monetization payment pointer ready, the only thing you need to change in this repository is the web_monetization_payment_pointer variable at the top of the results_api.py file and then you'll be all set to receive micropayments!

When your Infinity Decentralized instance is live on the internet, Coil members will be able to stream micropayments to your payment pointer whenever they visit your site.