Add files via upload

This commit is contained in:
wibyweb 2023-05-23 10:11:09 -04:00 committed by GitHub
parent fbc9c70cb2
commit 713e55b166
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23

View file

@ -273,7 +273,7 @@ You may want to run this on startup, easiest way to set that is with a cron job
<br>
<br>
<h3>Start the Crawler</h3>
It is best to run the crawler in a screen session so that you can monitor its output. You can have more than one crawler running as long as you keep them in separate directories, include a symlink to the same robots folder, and also set the <b>correct parameters</b> on each.
It is best to run the crawler in a screen session so that you can monitor its output. You can have more than one crawler running as long as you keep them in separate directories, include symlinks to the same robots folder and 'shards' file, and also set the <b>correct parameters</b> on each.
To view the parameters, type './cr -h'. Without any parameters set, you can only run one crawler (which might be all you need anyway). If necessary, you can change the database connection from 'localhost' to a different IP from inside cr.c, then rebuild.
<br>
<br>