Crawling

The act of accessing Web servers and/or file systems in order to extract information to feed into the enterprise search platform. By following links, a Crawler is able to traverser Web content hierarchies based on a single start URL.