0ct0pu5/wiby
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Add files via upload

Browse source
This commit is contained in:
wibyweb 2023-06-14 01:00:27 -04:00 • committed by GitHub
parent 3ef6a3116c
commit 8f1b7cc546
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
html/about/guide.html
View file

@ -288,7 +288,7 @@ Make sure the robots folder exists. All robots.txt files are stored in the robot
You can turn off checking for robots.txt files by commenting out the line calling the "checkrobots" function inside of cr.c.
<br>
<br>
If crawling through hyperlinks on a page, the following file types are accepted: html, htm, txt, php, asp. Links containing parameters are ignored. These limitations do not apply to pages directly submitted by people.
If crawling through hyperlinks on a page, the following file types are accepted: html, htm, xhtml, txt, php, asp. Links containing parameters are ignored. These limitations do not apply to pages directly submitted by people.
<br>
<br>
<h3>Start the core server</h3>