# *.nike.com robots.txt -- just crawl it. User-agent: * Allow: / Crawl-delay: 20