Robots Exclusion Protocol

A protocol that allows Web site administrators to indicate to visiting robots which parts of their site should not be visited by the robot.