The Whole Ed Catalog


Internet Searching, Search Engines, and Search-Engine Optimization

Directories, search engines, and guides provide a means of finding information on a given topic by searching through their databases for keywords. Each service indexes and organizes data from the web according to its own plan, and results can vary widely from one service to another. The details of these plans are kept confidential to prevent webmasters from search-engine spamming -- creating content for no other reason than to take advantage of searchengines peculiarities.

Webmasters, on the other hand, try to tailor their content to be well-indexed. They use common keywords and keyword combinations, repition, emphasis, HTML "meta" tags, and whatever else they can think of to improve their search-engine rankings. this process is known as "Search Engine Optimization" or SEO. The line between SEO and spamming is ill-defined, but certain practices are clearly frowned upon.

When using search functions, the best approach is to try several products, and use the one that best fits your needs. Searching for the phrase "search engines" will often turn up a new ( or at least unknown ) search site.

Search Engine Watch provides detailed information on various services, as well as informative articles on search technology in general.

Huge Search Engines

- Index the most comprehensive data, but of necessity rely upon artificial intelligence (AI) to establish relevance. Many feel that smaller directories provide better results, especially on very specific topics which may have keywords in common with other disciplines.

Search Engine Size, (number of pages indexed) alone is not a good measure of a service's effectiveness, but it is a specific basis for comparison.

  • MSN Search -- Microsoft is a late-comer to the search world, having used Yahoo!-powered MSN Search until quite recently. Although I am nof fan of Microsoft generally, MSN search is hard to beat. Their new crawler is everywhere, providing the most up-to-date results of the major engines. Webmasters cannot ignore the traffic that MSN Search generates.

    MSN Search
  • Google: - Uses PageRank™ algorithm to produce very relevant results. Google doesn't mix paid advertising with standard results, which lends a sense of objectivity to their rankings. The Google interface is deceptively simple, but a little persistence will turn up a variety of features, services, specialized searches, and help.

    Since Google is by far the most popular search on the web (as of this update, June 2005) there is a great deal of speculation about it's current ranking algorithm.
    Web    Images    Groups    News    Froogle    Local    more »
      Advanced Search
      Language Tools

  • AOL Search -- Yet another "Powered by Google" engine. Lags behind even Googlebot's sluggish indexing.
  • Teoma -- Uses Subject-Specific Popularity™ to assign greater weight to links within a community of subject-related sites in ranking sites. Teoma indexes over 2 billion pages and is the primary supplier of Links to the Ask Jeeves search engine.

    Recently acquired by

  • Alta Vista - Founded by DEC, now a subsidiary of Compaq. Probably the most complete listing available.
  • Lycos - Pioneer of spidering technology. Very complete and well-ordered.
  • Excite - Advanced 'concept' search returns hits based on closely-related keywords with confidence rating %
  • infoseek - Fast, easy to use. Links to similar sites, Now part of the Go network.
  • Web Crawler - Owned by AOL. Scheduled for inclusion in Excite.
  • HotBot - Associated with Wired Magazine
  • Starting Point

Guides & Award Sites - Sites reviewed and selected by editors
  • Encyclopedia_Brittanica_Internet_Guide - The search function and/or breadth of coverage of this guide appears to be much improved. Browsing also requires a less "drill down" but, then -- it IS the Encyclopedia Brittanica!
  • My Virtual Reference Desk - This site by Bob Drudge - Whose son is responsible for the (in)famous Drudge Report has a little bit of everything. Emphasis is on News/Current events.

Directories - Search database refined by human reviewers
  • Open Directory Project (formerly NewHoo)

    A directory with a difference: editing is accomplished by volunteer editors. New submissions are included in a timely fashion in some categories - others may take six months (!). Unfortunately, Google is the most-used search engine on the web, and they rely heavily upon ODP. You need to be listed here.

    As with any directory, the categories in the ODP tend to be somewhat arbitrary. In some cases your page will be dumped into an undifferentiated catch-all. At other times you will be required to drill down with hair-splitting exactitude to find the "right" submission category. In the end, an editor who doesn't really understand the intended use of your site may put your page in a weird category. There is an appeals process, but it's pretty cumbersome, too. Familiarizing yourself with the ODP categories that apply to your site is time well spent.

  • Yahoo! - Employs a large staff of content specialists to verify the content of pages indexed. See related article
  • Zeal - is the member-edited directory of Reader's Digest's LookSmart family of search services.
  • Snap! - Relatively new service has some nice features including editors' guides.

Large search engines
  • Alexa - Now an company, Alexa's search is "Powered by Google" and includes some interesting additional features. Input from users who download the Alexa Toolbar is incorporated into Alexa's database to provide additional information such as related links, site traffic, and link popularity. The Alexa Toolbar is very similar to the Google Toolbar, and I wouldn't be surprised if it was a licensed version of the same. It has the very similar features, including a page rank indicator (indispensible for webmasters' Search Engine Optimization ) an excellent and inobtrusive pop-up blocker and more. In the interest of full-disclosure, I must note that any links that appear in your search will have our amazon associate ID embedded in them (we get a small commision if you purchase something). If you enjoy this site , a great way to support it is to :

  • Scrub The Web:    

  • Tower Search -


    Search Provided By TowerSearch

  • Astrofonix Web Directory
    Web Directory with comprehensive resources, search engine and web directory for internet users to find companies, products, services, news etc... and sign up for a Free email account. Add your Site FREE to our Web Directory.
Multiple-Engine Searches

  • ProFusion - "The original intelligent meta-search engine on the Web" is a project of the University of Kansas. (Go Jayhawks!) It queries a variety of online searches, evaluates and merges the results. Specialized searches are available to improve relevance.

  • CNet Search - Doesn't say much about their technology, but it is somehow related to the Open Directory Project


  • MetaCrawler - Another Meta-search engines that isn't very forthcoming about their search technology.

  • Monster Crawler - Lists the search engines it aggregates.
    Monster Crawler - The Monstrous Search Tool
    Monster Crawler  -  The Monstrous Search Tool!
    "Search All The Major Engines With Just One Click"
  • Dogpile - Search 10 popular search engines at once. Custom search lets you sort results by search engine.
Special-Purpose Search Engines
  • Filez - This site is legitimate, despite the "cracker" spelling. The place to search thousands of ftp sites.
  • BBC Education Web Guide - Absolutely fabulous guide to educational resources on the web.

Discussion Group Listings
  • Google Groups (Beta) - Extensive index of Usenet newsgroups. Formerly Deja News. If you are not familiar with Usenet, please read the FAQ before posting.

  • Nerd World - Post to any of many "Assemblies" or start your own for free.