Google supports sitemaps:
https://www.google.com/webmasters/sitemaps/docs/en/about.html
It's an XML format for listing pages on a site, it should be fairly easy to generate this with petal, it's a simple format, eg:
<?xml version="1.0" encoding="UTF-8"?> <urlset xmlns="http://www.google.com/schemas/sitemap/0.84"> <url> <loc>http://www.yoursite.com/</loc> <lastmod>2005-01-01</lastmod> </url> <url> <loc>http://www.yoursite.com/foo/</loc> <lastmod>2004-12-23</lastmod> </url> </urlset>
https://www.google.com/webmasters/sitemaps/docs/en/protocol.html
We might as well do this as /.sitemap.xml and perhaps a plain text version would make sense:
https://www.google.com/webmasters/sitemaps/docs/en/faq.html#s9
What is a bit odd is that you have to sumbit the sitemap to google, they don't seem to intend to find them themselves:
https://www.google.com/webmasters/sitemaps/docs/en/faq.html#s4
This was posted to MKDoc-dev: