#robots.txt for http://pluralism.org User-agent: * Disallow: /news/ Visit-time: 0400-0800 Crawl-delay: 30 User-agent: Slurp Disallow: /news/ Visit-time: 0400-0800 Crawl-delay: 10 User-agent: msnbot Disallow: /news/ Visit-time: 0400-0800 Crawl-delay: 10 User-agent: bingbot Disallow: /news/ Visit-time: 0400-0800 Crawl-delay: 10 User-agent: adidxbot Disallow: /news/ Visit-time: 0400-0800 Crawl-delay: 10 User-agent: Baiduspider Disallow: / Crawl-delay: 30 User-agent: dotbot Disallow: / User-agent: Exabot Disallow: / User-agent: AhrefsBot Disallow: