Incremental indexing

Incremental indexing enables an efficient combination of freshness and scalability in a search node. The index can be partitioned into different segments (partitions) with different refresh rates. Data is initially indexed in the smallest index where it resides for a given period of time before it propagates a larger index. This procedure repeats itself; once data has entered index P, it stays there for a configurable amount of time before it moves to index P+1. This concept takes place within one search node, and is independent of the partitioning performed using multiple columns of search nodes.