# Tells Scanning Robots Where They Are And Are Not Welcome # User-agent: can also specify by name; "*" is for everyone # Disallow: if this matches first part of requested path, # do not allow indexing or crawling User-agent: * Disallow: /includes/ Disallow: /graphics/ Disallow: /styles/