FluxBB_by_Visman/robots.txt

31 lines
415 B
Text
Raw Permalink Normal View History

2013-11-14 15:59:04 +00:00
User-agent: *
Disallow: /a
Disallow: /d
Disallow: /c
Disallow: /e
Disallow: /j
Disallow: /l
Disallow: /m
Disallow: /p
Disallow: /r
Disallow: /s
Disallow: /u
Disallow: /*pid=
User-agent: Yandex
Disallow: /a
Disallow: /d
Disallow: /c
Disallow: /e
Disallow: /j
Disallow: /l
Disallow: /m
Disallow: /p
Disallow: /r
Disallow: /s
Disallow: /u
Disallow: /*pid=
2013-11-14 15:59:04 +00:00
Crawl-delay: 5
2015-11-10 07:29:56 +00:00
Clean-param: search_hl
Clean-param: csrf_hash