1
0
mirror of https://github.com/osmarks/mycorrhiza.git synced 2024-10-30 11:46:16 +00:00
mycorrhiza/static/robots.txt

7 lines
93 B
Plaintext

User-agent: *
Allow: /page/
Allow: /hypha/
Allow: /recent-changes
Disallow: /
Crawl-delay: 5