# Spider instructions for all web servers in the production pool # Thu, 11 Dec 2008 06:18:00 -0400 # Exclude Files From All Robots: User-agent: * Disallow: /secure/ Disallow: /cart/ Disallow: /404error.html Disallow: /Smart404.aspx Disallow: /SearchResults.aspx Disallow: /ivr/ # End robots.txt file