Quick Guide to Web Searching

• Using the Web for research

• How Search Engines work

• Directory Searches

• Keyword Searches

• Google features

• Special search options on Google

• Google Scholar

• Other good search engines

• Specialty search engines

• Metasearch engines

Using the Web for research

The Web is very convenient and there is a lot of information available.  The quality of information on the Web varies wildly.  In the last lesson, we looked at using the Web to access library databases.  In this lesson, we'll look at how to search the Web effectively  

If you use the search techniques discussed below to limit your search and carefully evaluate the results, you can find information on the Web that is both credible and useful for your research. 

Use websites for

• background information
• very current information
• images or multimedia 
• information from special interest groups, associations and local and national organizations
• international and foreign language information
• popular information
• statistics

How Search Engines Work

A search engine is an automated system for finding information on the Web.  Most do the following things:

• They use automated programs called spiders, crawlers or robots (bots) to search the Web.
  • These programs are given a core group of websites and told to follow the links on each page they find.
  • They save information they find.  Sometimes entire documents, sometimes just keywords and links.
• They keep a database or index of the words the spider or crawler finds and where the words were found.
• They have software that lets users search for those words.

NOTE:   Search engines using the same software will still find different information.  Even if they begin with the same core group of websites, they do so at different times and will find different links.  Because of this, it's  a good idea to use a couple of different search engines, especially if you're having problems finding good information.

When you type words into the search box, the search engine will:

• analyze your search terms
• compare them to the websites contained in its database
• select the ones that match your search statement
• display the URLs using some kind of relevancy or ranking system

Search engines rank sites in different ways:

• some, like Google,  rank a site by counting how many other sites link to them, in other words, the more popular the site, the higher the ranking.  This system is often the most accurate.  Many other search engines have begun using Google's system.
• some list sites by how many times your keywords appear and where they appear.   A keyword that appears in the title ranks higher than one in the body of the web page.
• some engines put paid or sponsored listings on top of the list.  The good search engines clearly label these as paid or sponsored  listings.

TIP: There are many things on the Web that search engines have problems finding. For example, they usually won't find:

• the contents of a library catalog - Google is working on getting around this
• articles listed in an article database
• postings to an online newsgroup - many search engines will now do this
• individual newspaper articles - search engines will find some articles.
• information in unusual formats - particularly media files.  You may need to use a specialty search engine to find some media files.

Information that is hard for search engines to find or search is part of the hidden,  invisible  or deep web.   Many of these sites are very useful.  The best way to find them is to use directory listings.  Good general directories include:  Librarian's Internet Index , and The Internet Public Library, as well as the directory features of Google and Yahoo. 

How much of the web does a search engine search?  Latest estimates range from 1% to about 50%.  Each search engine has slightly different coverage, so if you're having trouble finding information, try  several different search engines.

How To Search using Search Engines

There are generally two ways to use a search engine, a keyword search or a directory search. 

Directory Search:  

A directory search works like the yellow pages in a phone book.  The links in a directory are chosen by people and not by a program.

Use categories, like those in Google or Yahoo,  just click on links starting with a broad category and narrowing down until you find what you want.

 For example, if you wanted to find information about the planet Saturn in the Google Directory, you would type in http://directory.google.com.  Or, you can click on the About Google link. 

Then click on Google Services & Tools. (Want to know what Google's up to?  Click on Google Labs.)

Click on directory  (but really, it's a lot faster just to type http://directory.google.com )

From the list,  look at the broadest category related to your topic. Continue to select narrower categories in the directory until your subject is reached. The categories you would probably select would be:

• Science
• > Astronomy
• > Solar System
• > Saturn

The main advantages of using a directory  or category  search include:

• Fewer results 
• Sites are chosen by people rather than computers
• Usually the results will be more relevant
• You won't waste time looking through unrelated sites about cars or the god named Saturn

TIP: Not sure of the category? Type your topic in the directory search box and check out the results. Make sure you are searching in the directory.

Google Directory is not the only Web directory available. Other examples include the Librarians' Internet Index and CompletePlanet.