How Search Engines Work
The
term search engine is frequently used to describe
both crawler-based search engines and human-powered
directories. These two types of search engines gather their
listings in very different ways.
A. Crawler-Based Search Engines
Crawler-based
search engines, such as Google, produce their listings
automatically. They act like a spider crawling through the
website and then people search through what they have
found. Hence the term, crawler-based. If you modify
your web pages, crawler-based search engines eventually
finds these modifications, and that can change how you are
listed. Web page titles and headings, body content and
other elements all play a very important role in achieving
this. There are three major elements to crawler-based
search engines—the spider, the index, and the search engine
software.
1.
Spider
The spider, also called the crawler, visits a
web page, reviews it, and then follows links to other web
pages within the same site. This is what is referred to
when a site is being “crawled” or “spidered”. The spider
returns to the website on a frequent basis, perhaps every
couple of months, to look for modifications. Everything the
spider finds goes into an index, the second element of a
crawler-based search engine.
2.
Index
The index, also called the catalog, is like a
huge book containing an exact copy of every web page that
the spider finds. If a web page has been altered, then this
book is updated with the current information. Sometimes it
may take a while for new web pages or modifications that the
spider finds to be added to the index. Therefore, a web
page may have been spidered, but not yet indexed.
Until it is indexed, the web page will not be available to
those searching with the search engine.
3.
Search Engine Software
Search engine
software is the third element of a search engine. This is
the program that shifts through the millions of pages
recorded in the index to find matches to a search and rank
them in order of what it believes is most relevant.
B. Human-Powered
Directories
A human-powered directory, such as the Open
Directory, relies on people for its listings. You submit a
brief description to the directory for your entire website,
or editors write one for websites that they review. A
search looks for matches only in the descriptions
submitted. Modifying your individual web pages will have no
effect on your listing. Items that are useful for improving
a listing with a search engine have nothing to do with
enhancing a listing in a directory. The only exception is
that a really good website with exceptional content might
get reviewed for free, whereas a poor website will not.
III. How Search Engines
Rank Web Pages
Search for a topic using your favorite crawler-based search
engine. Almost instantly, the search engine will sort
through the number of pages it knows about and give you the
ones that match your topic. The matches will be ranked from
the most relevant to the least relevant. Of course, the
search engines do not always hit the target. Irrelevant
pages make it through and sometimes it may take a little
more digging to locate what you want. But, by and large,
search engines get the job done. Crawler-based search
engines rank relevancy of websites by following a set of
rules, known as an algorithm. Every major search engine
programs their algorithms differently, which is why you will
be ranked differently with different search engines. But,
two major factors that are usually common among all search
engines are location and frequency.
A.
Location
The location of where the word or phrase
being searched appears in the website is a huge factor. If
the word or phrase appears in the HTML title tag, that
website(s) will be considered the most relevant. Also, if
the word or phrase appears close to the top of the first web
page, for instance, the home page, the relevancy will be
ranked accordingly.
B.
Frequency
The frequency by which the word or phrase
appears throughout the website is also a factor in rank
determination. If you mention a certain topic frequently
throughout your site, you will achieve a higher ranking.
IV. Using
META tags
Meta tags are information inserted into the head area
of your web pages. Besides from the title tag, information
in the head area of your web pages is not viewed by those
looking at your pages in browsers. Instead, Meta tags that
are in the head area are used to communicate information
that a human visitor may not be concerned with. Meta tags,
for example, can tell a browser what character set to
use or whether a web page has self-ratings in terms of
adult-only content. The syntax for a Meta tag is as
follows:
<META
name=”description” content=”…..….”>
<META
name=”keywords” content=”……….”>
<META
name=”Robots” content=”NOINDEX”>
A. Types
of META tags
As you can
probably detect from the above syntax, there are three
common types of Meta tags: description meta tag,
keywords meta tag, and robot meta tag. There are other
meta tags used by specialized search engines, but these are
the most commonly and widely used.
1.
Description
Meta Tags
The Meta
description tag allows you to influence the description of
your page in the crawlers that support the tag. The above
syntax shows the text you wish to be shown as your
description going between the quotation marks after the
"content=" portion of the tag. So in place of the dots
between the quotations marks, your description would
appear. Generally speaking, 200 to 250 characters may be
indexed, however only a small portion of this may be
displayed, depending on the search engine. It is a wise
idea to use description meta tags, because they give you a
degree of control with various crawlers.
2.
Keywords Meta
Tags
The meta
keywords tag allows you to provide additional text for
crawler-based search engines to index along with your web
site content. However, according to
www.SearchEngineWatch.com not to many crawlers support
keyword meta tags. But, it is still a good idea to use for
those that do support them. Just as in the description
syntax, you replace the dots between the quotation marks
with your chosen keywords.
3.
Robot Meta
Tags
This meta tag
allows you to specify that this particular web page should
not be indexed by crawlers.
B.
The Myth About
META tags
It is a
common myth that if you use meta tags, you are guaranteed to
be in the top ranking, but this is not true at all. Meta
tags do not guarantee anything, but they may help. Since
all search engines work a little differently from each
other, it is always wise to try everything and anything at
least once.
V.
Summary of SEO
Optimizing
refers to fine-tuning a web site to achieve optimum search
results. This is not a one-time process. Optimization is
an on-going process that requires key pages to be
resubmitted every couple of months and important new pages
as they appear. But, it’s not enough to optimize web pages
for search engines. You still have to submit them for
indexing.
There are two
types of search engines—crawler-based and human-powered
directories. There are three elements of a crawler-based
search engine—spider, index and search engine software.
Search
engines rank web pages based of two main criteria—location
and frequency.
Using meta
tags can increase your chances of being in the top ranking,
but it is not a guarantee. There are three common types of
meta tags—description, keywords, and robot meta tags.
|
Is
Duplicate Website Content Hurting your Website? Use the Page Comparison Tool
Compliments of SEO Chat ™