search engine

a computer program that searches documents, esp. on the World Wide Web, for a specified word or phrase and provides a list of documents in which this word or phrase is found.

[1990-95]

* * *

Tool for finding information, especially on the Internet or World Wide Web.

Search engines are essentially massive databases that cover wide swaths of the Internet. Most consist of three parts: at least one program, called a spider, crawler, or bot, which "crawls" through the Internet gathering information; a database, which stores the gathered information; and a search tool, with which users search through the database by typing in keywords describing the information desired (usually at a Web site dedicated to the search engine). Increasingly, metasearch engines, which search a subset (usually 10 or so) of the huge number of search engines and then compile and index the results, are being used.

* * *

      computer program to find answers to queries (query language) in a collection of information, which might be a library catalog or a database but is most commonly the World Wide Web. A Web search engine produces a list of “pages”—computer files listed on the Web—that contain the terms in a query. Most search engines allow the user to join terms with and, or, and not to refine queries. They may also search specifically for images or for names of Web sites.

      The Web is largely unorganized, and the information on its pages is of greatly varying quality, including commercial information, national databases, research reference collections, and collections of personal material. Search engines try to identify reliable pages by weighting, or ranking, them according to the number of other pages that refer to them, by identifying “authorities” to which many pages refer, and by identifying “hubs” that refer to many pages. These techniques can work well, but the user must still exercise skill in choosing appropriate combinations of search terms. A search for bank might return millions of pages (“hits”), many from commercial banks. A search for river bank might still return over a million pages, many of which are from banking institutions with river in the name. Only further refinements such as river bank and riparian reduce the number of hits to some 70,000, the most prominent of which concern rivers and their banks.

      Search engines use crawlers, programs that explore the Web by following hypertext links from page to page, recording everything on a page (known as caching), or parts of a page, together with some proprietary method of labeling content in order to build weighted indexes. Web sites often include their own labels on pages, which typically are seen only by crawlers, in order to improve the match between searches and their sites. Abuses of this voluntary labeling can distort search results if not taken into account when designing a search engine. Similarly, a user should be cognizant of whether a particular search engine auctions keywords, especially if sites that have paid for preferential placement are not indicated separately. Even the most extensive general search engines, such as Google, Altavista, Excite, and Lycos, cannot keep up with the proliferation of Web pages, and each leaves large portions uncovered. Because search engines only partially overlap in coverage, especially search engines dedicated to specialized topics, meta-search engines, such as MetaCrawler, have developed in order to direct simultaneous queries to multiple engines.

      An alternative search tool is a directory such as Yahoo, which organizes information on Web pages according to a human-developed hierarchical classification. There are also research projects to develop “smart agents” for users, programs that gradually perform more effective searches by keeping track of each user's pattern of preferences and contexts.

* * *