# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file # many of these components typically cannot be indexed anyway, since they require login, but some sites have public areas User-agent: * # Disallow: /components/event # Public Area for Carriage Trade Disallow: /components/report Disallow: /components/vehicle Disallow: /components/carfax