|
|
@@ -273,7 +273,7 @@ You may want to run this on startup, easiest way to set that is with a cron job
|
|
|
<br>
|
|
|
<br>
|
|
|
<h3>Start the Crawler</h3>
|
|
|
-It is best to run the crawler in a screen session so that you can monitor its output. You can have more than one crawler running as long as you keep them in separate directories, include a symlink to the same robots folder, and also set the <b>correct parameters</b> on each.
|
|
|
+It is best to run the crawler in a screen session so that you can monitor its output. You can have more than one crawler running as long as you keep them in separate directories, include symlinks to the same robots folder and 'shards' file, and also set the <b>correct parameters</b> on each.
|
|
|
To view the parameters, type './cr -h'. Without any parameters set, you can only run one crawler (which might be all you need anyway). If necessary, you can change the database connection from 'localhost' to a different IP from inside cr.c, then rebuild.
|
|
|
<br>
|
|
|
<br>
|
wibyweb