The job of a search engine is to seek out sites on the Web, summarise their content, pool that information in a searchable database, and produce useful results when you search the database. Some search engines do a pretty good job of this, some not so good. The results fluctuate: The best search engine six months ago is not necessarily the best search engine today. And, of course, the best search engine for me may not be the best for you.
If you find searching for information on the Internet an exercise in frustration, one of the best things you can do to minimise the angst is to learn about the ins and outs of search engines and similar Web-based tools. To become adept at searching the Web, you need to understand the different types of search engines, the way each of the major search engines works, and the limitations of each. You'll then be in a position to choose two or three which offer distinctive features and make them your own.
There's more than one way to search
Which should you choose? That depends on your own searching preferences. But to begin with, it's important to recognise that the 'search engine' tag encompasses a variety of search tools which have as many differences as commonalities. Yahoo!, for instance, which is currently one of the most popular search sites, isn't a search engine at all. It's a Web directory. That's quite a different thing.
As well as search engines and directories, there are meta search engines, 'Deep Web' search engines, newsgroup searches, mailing list searches, and specialty search engines. The dividing line is not always clear, with some Web directories offering search engine services as well, and some search engines providing Web directories and even newsgroup searches.
This guide introduces you to each type of search tool, picks out the star performer in each category, and points you towards alternatives, both popular and obscure.
Search engines
True-blue search engines are those which make an attempt to index everything on the Web and then let you search the resultant database using keywords or phrases. They're all failures. That's because it's an impossible task. The Net is massive – we're talking billions of pages on the World Wide Web alone, which is merely the tippermost tip of the iceberg. And that mass is constantly changing, with new pages added by the semi-load each minute, and long-familiar pages disappearing when the funding runs out or the authors get tired of maintaining them.
Given those conditions, search engines such as Google and Fast do an amazing job.
One thing you should know: While the best search engines try to use objective criteria – such as a page's popularity – to order the results they return, others accept payments from sites in order to promote their listings to the top of the results page. If you want to know which search engines are which, check out the article Pay for Placement? at Search Engine Watch.
When should you use a search engine?
-
When you're searching for a specific word, phrase or concept.
-
When you want to perform complex queries.
-
When you want to unearth a broad range of possible results.
Which is the best search engine?
Google. Google's become the darling of the search set in the past couple of years. There are four reasons for its well-deserved popularity:
-
It indexes more of the Web than any other search engine (although Fast is competing head to head), making it the most complete of traditional search engines.
-
While other search engines were turning themselves into lumbering, clogged 'portals' which tried to satisfy every need of visitors, Google pared away all excess and presented itself as a clean, fast, targetted service.
-
Google uses very simple search syntax. Just enter a series of words separated by spaces and Google will do its best to find pages satisfying all terms.
-
Google's ranking technique produces extremely focussed results. Google considers each link to a Web page as a recommendation for that page, with the most-recommended pages pushed to the top of the search results. So, even if your search query yields hundreds of thousands of possible matches, you probably won't need to dig past the first page or two in the list to find really useful resources.
What are the best alternatives?
Fast, AltaVista, Lycos, Excite and Hotbot.
|
Search engine links |
Web directories
Web directories make no attempt to index the whole of the Web. Instead, these services categorise large chunks of the Web and then organise those categories into smaller and smaller sub-categories. You can use keyword searches in a Web directory, but they work best when you browse through the categories and drill down until you find relevant sites. For instance, if you're looking for photographs of quokkas, you might click the Arts category, then Visual Arts, then Photographers, then Nature and Wildlife, and then choose one of the individual photography sites.
While search engines build their indexes using special programs which explore the Web automatically, most Web directory content is hand-picked by humans. That human intervention sounds like an advantage, but it can lead to highly skewed results. It also means Web directory content is usually out of date, as it first has to go through the human vetting process.
When should you use a Web directory?
-
When you find it hard to phrase your search in specific terms or want to make a generalised search.
-
When you want to limit the resulting matches to a manageable number.
-
When you want the chaff to be eliminated before the results are presented to you.
Which is the best Web directory?
It's hard to go past Yahoo!…but I'm going to. Ask Jeeves can't rival Yahoo!'s expansiveness, but it's one of the few search sites which lets you search using 'natural language', and that's a big plus. At Ask Jeeves, you can either burrow down through the categories, or you can pose questions such as "What is meiosis?" or "How can I cook a pavlova using emu eggs?" If Jeeves doesn't know what you're talking about, he'll offer similar, alternative questions. Ask Jeeves also comes in a localised 'Oceania' version.
What are the best alternatives?
Yahoo!.
|
Web directory links |
Deep Web search engines
If you think the billion or two pages on the Web is an impressive number, the so-called 'deep Web' will leave you slack-jawed. The sites searched by even the most dedicated search engine constitute only a tiny fraction of the Web. Many of the truly useful sources of information on the Internet lurk in databases, catalogues and references which are not even scanned by the major search engines. The content on this 'deep Web' or 'invisible Web' is estimated by BrightPlanet to be some 500 times more extensive than the couple of billion pages in the visible Web.
BrightPlanet claims to have "uncovered" the deep Web. In fact, there are many people who rarely touch traditional search engines and who, instead, spend their daily lives exploring deep Web resources, and they've done so since long before BrightPlanet came along. However, BrightPlanet has done a good job of mapping the extent of this huge resource which most surfers ignore completely.
A large part of the content on the deep Web is contained in topic-specific databases. Most of this content is freely available to the public.
Some of the deep Web engines, such as CompletePlanet, attempt to catalogue the entire deep Web. Others, such as Northern Light, provide access to a chosen selection of resources. If you're interested in the deep Web, you should also check out the Gateway to Library Catalogs, which provides access to what are known as Z39.50 (it's a US Library of Congress specification) resources.
When should you use a deep search engine?
-
When you are trying to locate in-depth and authoritative sources of information.
-
When you suspect that the content you are after may not be readily available on the Internet.
-
When you're looking for academic or professional resources.
-
When you wish to search reference works and bibliographies.
Which is the best deep search engine?
BrightPlanet's CompletePlanet turns up the most useful results on a consistent basis. At CompletePlanet, you don't just search for a matching page or site; instead, your searches turn up matching resources, including entire databases and references you'd have trouble locating otherwise. You can use standard search engine keyword queries or phrase your question using 'natural language'.
What are the best alternatives?
Invisible Web, Northern Light, ProFusion and the Gateway to Library Catalogs.
| Deep Web search engine links |
|
CompletePlanet CompletePlanet help |
Meta search engines
Meta search engines leave the dirty work of searching to other search engines and then pool the results in a single list. So, with a single query you can elicit results from half a dozen or more search engines. It's a great way to save time if you're not familiar with any individual search engine. Most meta searches return a number of results per search engine, making it easy to spot sites which feature prominently on multiple search engines.
It's well worth taking advantage of each meta search's options. Most of them let you pick which search engines are used in each search, the depth of the search, the time taking for the search, and so on.
When should you use a meta search?
-
When you don't have a favourite search engine.
-
When you want a broad, but not deep, cross-section of results.
-
When you don't know the specific search syntax used by individual engines.
Which is the best meta search engine?
Ixquick's simplicity and speed make it hard to beat. Select what you want to search – the Web, news, MP3s or pictures – then type your query. The results are returned sorted according to relevance. If you want to narrow your search, you can uncheck any of the dozen search engines.
qbSearch's QuickLinks feature makes it an attractive alternative. On the results page, quick the QuickLinks button, thenclick each of the results which seems interesting, then click the QuickLinks button once more to see all the chosen matches displayed in a single window.
What are the best alternatives?
Vivisimo, DogPile and ProFusion.
| Meta search engine links |
News searches
One of the big problems with most search engines is that they're always out of date. The job of indexing all those sites simply takes too much time, and most Web sites are in a constant state of change, too. That makes search engines next to useless when you're trying to track down the latest news.
That's led to the emergence of a special category of search engines which devote themselves to tracking the latest news stories. The best of these offer hourly updates of headlines and stories culled from sources around the globe.
When should you use a news search?
-
When you want current, topical information.
-
When you want to research a story which made headlines in the past.
Which is the best news search?
The Moreover News Portal is slick. This cleanly designed search engine collates results in over 330 news categories from sources worldwide. It features plenty of Australian resources and the index is updated throughout the day.
What are the best alternatives?
Google's new News page, 1st Headlines and News Index.
News search links |
| Google News |
Specialty searches
If you can't find the information you're after with any of the main search tools, you may need to try a specialty search engine. There are hundreds of such engines on the Web, ranging from secondhand book databases to regional search engines to chemical modelling searches. Here are some of the most useful.
| Specialty search links |
|
Newsgroups: Google Groups Google Groups help |
|
Mailing lists:
Topica Topica help |
|
Books:
Alibris Bookfinder Biblioz |
|
Science: Scirus |
|
Filename extensions:
File extension search engine |
|
Shareware: Downloads.com |
Choose your search
Try to become acquainted with all the major search tools and your head will end up spinning. If you want to hone your searching efficiency, a better approach is to choose a handful of tools, each from a different category, and then get to know those search tools really well.
Most of the search sites provide tips or detailed help showing you how to make the most of their features. In the accompanying boxes, you'll find direct links to the help pages for many of the top search sites to help get you started.
© 2002 Rose Vines
|
Host your site where I host mine |
 |
Support geekgirl'sDo you find the tutorialson this site useful? If so, please show your support by kicking in a few bucks to help buy computers for the wonderful orphanages run by the Afghan organisation, afceco.org. For a small amount, it is possible to make a difference in an area of the world which is hurting badly. |
|
|
| Want to know more? Read this post on my blog. |
| top | home | net search menu |