Crawler

See Spider.

see "spider"

A software program that retrieves documents from data sources and gathers information that can be used to create search indexes.

Synonyms droid, Web crawler, Web spider.