User-agent: * Crawl-delay: 1.4 Disallow: /admin/* Disallow: *?*