The search engine's indexing "robots" (or "spiders") will examine and index suitable documents from web servers that are added explicitly to its configuration.