# Disallow all crawlers access to certain pages. www.gpgear.com User-agent: * Disallow: /cgi-bin Disallow: /js Disallow: /class Disallow: /awstats_data Disallow: /stats Disallow: /Log Files Disallow: /icon