#robots.txt file for https://www.iflscience.com/ User-agent: * Disallow: /search$ #bot crawl-delays User-agent: AhrefsBot Crawl-delay: 10 User-agent: Bingbot Crawl-delay: 10 #sitemap list for https://www.iflscience.com/ Sitemap: https://www.iflscience.com/sitemaps/sitemap.xml Sitemap: https://www.iflscience.com/sitemaps/sitemap-news-1.xml