User-agent: Node/simplecrawler Disallow: * User-Agent: * Disallow: /ads/ Disallow: /admin* Disallow: /user-activation Disallow: /search* Disallow: /places/*/edit Disallow: /browse_content* Disallow: /nav/* Disallow: /random Disallow: /denver Disallow: /travelnevadahighway95tc Disallow: /newsletters/kiosk Disallow: /chase-sapphire User-agent: GPTBot Disallow: / Sitemap: http://www.atlasobscura.com/sitemaps/sitemap_index.xml.gz