0ct0pu5/forkbb
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Update robots.txt

Browse source
This commit is contained in:
Visman 2021-01-12 16:14:31 +07:00
parent 3a9ebb5336
commit d2caa09585
1 changed files with 6 additions and 29 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

35
public/robots.txt
View file

@ -1,30 +1,7 @@
User-agent: *
Disallow: /a
Disallow: /d
Disallow: /c
Disallow: /e
Disallow: /j
Disallow: /l
Disallow: /m
Disallow: /p
Disallow: /r
Disallow: /s
Disallow: /u
Disallow: /*pid=
User-agent: Yandex
Disallow: /a
Disallow: /d
Disallow: /c
Disallow: /e
Disallow: /j
Disallow: /l
Disallow: /m
Disallow: /p
Disallow: /r
Disallow: /s
Disallow: /u
Disallow: /*pid=
Crawl-delay: 5
Clean-param: search_hl
Clean-param: csrf_hash
Disallow: /adm
Disallow: /log
Disallow: /mod
Disallow: /reg
Disallow: /search
Disallow: /userlist