COM 125 Week 2: The Search is Over

Possibly one of the most widely used functions in the sphere of the Internet, are the search engines. These information retriveal systems which existed even before the birth of the World Wide Web, were designed to help find information stored on a computer system, inside a corporate or proprietary network, or in a personal computer. A search engine allows one to ask for content meeting specific criteria (typically those containing a given word or phrase) and retrieves a list of items that match those criteria (Wikipedia, 2007).

Brian Pinkerton created the first "full text" search engine - WebCrawler, which went live on April 20, 1994 (Wikipedia, 2006). Unlike its predecessors, it let users search for any word in any webpage, which became the standard for all major search engines since. It was also the first one to be widely known by the public (Wikipedia, 2007).

WebCrawler was originally a separate search engine with its own database. However, now, WebCrawler® is a metasearch engine that blends the top search results from Google, Yahoo!, Windows Live Search (formerly MSN Search), Ask.com, About.com, MIVA, LookSmart and other popular search engines. WebCrawler also provides users the option to search for images, audio, video, news, yellow pages and white pages. (Wikipedia, 2006)

In a September 1966 Scientific American article, " Information Storage and Retrieval," Ben Ami Lipetz described how the most advanced information technologies of the day could handle only routine or clerical tasks. He then concluded perceptively that breakthroughs in information retrieval would come when researchers gained a deeper understanding of how humans process information and then endowed machines with analogous capabilities. Clearly, computers have not yet reached that level of sophistication, but they are certainly taking users' personal interests, habits and needs into greater account when completing tasks.


In less than a decade, Internet search engines have completely changed how people gather information. Today, no longer must we scour shelves and shelves of books to look up something; rather we can pull up relevant documents with just a few clicks on a keyboard.

New search engines are improving the quality of results by delving deeper into the storehouse of materials available online, by sorting and presenting those results better, and by tracking your long-term interests so that they can refine their handling of new information requests.

For example, during the six years in which Google rose to dominance, it offered two critical advantages over competitors. One, it could handle extremely large-scale Web crawling tasks. Two, its indexing and weighting methods produced superior ranking results. Recently, however, search engine builders have developed several new, similarly capable schemes, some of which are even better in certain ways (Mostafa, J., Scientific American.com, 2005, p.2).

Mostafa adds that to this end, programmers have developed a class of software, referred to as wrappers, that takes advantage of the fact that online information tends to be presented using standardized " grammatical" structures (Scientific American.com, 2005, p.2). Wrappers accomplish their task in various ways. Some exploit the customary syntax of search queries and the standard formats of online resources to gain access to hidden content. Other systems take advantage of application programming interfaces, which enable software to interact via a standard set of operations and commands. An example of a program that provides access to the hidden Web is Deep Query Manager from BrightPlanet. This wrapper-based query manager can provide customized portals and search interfaces to more than 70,000 hidden Web resources.

In the future, according to Mostafa (Scientific American.com, 2005, p.1), search engines will broaden content horizons as well, doing more than simply processing keyword queries typed into a text box. They will be able to automatically take into account your location--letting your wireless PDA, for instance, pinpoint the nearest restaurant when you are traveling.

References:

History of the Internet. (2006, August 20). In Wikipedia, The Free Encyclopedia. Retrieved January 18, 2006, from http://en.wikipedia.org/w/index.php?title=History_of_the_Internet&oldid=70771572

Lipetz, B. A. (1966, September). Information Storage and Retrieval – Featured article. Scientific American.

Mostafa, J. (2005, February). Seeking Better Web Searches – Featured article. ScientificAmerican.com. Retrieved from http://www.sciam.com/article.cfm?chanID=sa006&articleID=0006304A-37F4-11E8-B7F483414B7F0000

Search engine. (2007, January 24). In Wikipedia, The Free Encyclopedia. Retrieved January 25, 2006, from http://en.wikipedia.org/wiki/Search_engine

WebCrawler. (2006, November 11). In Wikipedia, The Free Encyclopedia. Retrieved January 25, 2006, from http://en.wikipedia.org/wiki/WebCrawler