User-agent: * Crawl-delay: 2 # Rule 1 User-agent: Googlebot Disallow: /nogooglebot/ # Rule 2 User-agent: * Allow: / https://www.tokamakenergy.com/sitemap_index.xml Disallow: /archive/ Disallow: /brand/