User-agent: * Disallow: /wp-admin/ Disallow: /wp-includes/ Disallow: /wp-login User-agent: AhrefsBot Disallow: / User-agent: MJ12bot Disallow: / User-agent: DotBot Disallow: / User-agent: SemrushBot Disallow: / User-agent: Wget Disallow: / User-agent: msnbot Disallow: / User-agent: Dalvik Disallow: / User-agent: oBot Disallow: / User-agent: bingbot Crawl-delay: 6000 User-agent: GrapeshotCrawler Crawl-delay: 6000 User-agent: proximic Crawl-delay: 6000