1
0
mirror of https://github.com/osmarks/mycorrhiza.git synced 2024-10-30 11:46:16 +00:00
mycorrhiza/static/robots.txt
2021-07-31 19:33:15 +05:00

8 lines
107 B
Plaintext

User-agent: *
Allow: /help/
Allow: /page/
Allow: /hypha/
Allow: /recent-changes
Disallow: /
Crawl-delay: 5