Search engines have been around since the early days of the web, helping Internet users to find information as websites have grown from a few hundred to millions in number. But why is it that some websites always appear on the first couple of search results pages and other websites seem to be buried within hundreds or thousands of results or don't appear at all?

How do search engines work?

Given the vast number of websites and countless individual pages, it is impossible or impractical for search enginess to fully catalog every single website page by storing all the website's contents. Search engines use shortcuts to first find out what your website is about, and to periodic update their "profile" of your website.

Search engines utilize "robots" or "spiders" which are computer applications. These robots visit websites in search of new or updated pages, usually recording small samples of text of every page within the site and following external links.

Other search engines will only utilize HTML "META" tags. Meta tags are HTML page coding not visible to the website visitor, and is used behind the page in order to classify page content, so the search engine users find specific information quickly.

While meta tags are important and helpful to most search engines, some search engines like Google use an entirely different approach.

Search engines are not created equal

All search engines use similar principles, but work differently. For instance, some search engines create website titles and descriptions from the visible text within your page.

Other search engines use the META tags so you can control the description that appears as a search result. Other search engines use the words within the TITLE tag to use for the name of your page.

Google does not really use META tags at all, but looks at your overall website content, who you are linked to, and who has linked from their website to your website. Google also looks the quality and quantity of information you have in your website. They use a relevancy "scoring" system, and have their own guidelines for website owners to go by.

What follows are some myths and realities about search engines that should help you to get your website listed properly.

9 search engine myths and realites

Myth 1: I don't need to worry about search engines since all our marketing materials contain our website address.

Reality: If you haven't listed your site in at least the top ten search engines, if someone tries to find your website by company name (maybe they've misplaced your brochure or business card) they may not find your website. If nothing else, make sure you have submitted your site with Google, Yahoo!, Lycos, Excite, HotBot, AltaVista, InfoSeek, Snap.com, Magellan, Webcrawler, GoTo.com.

Myth: 2: Submitting your site to a search engine means your website will automatically be listed within a few days.

Reality: Submitting to most of the search engines simply means you have notified them that your website exists. Usually they will then send out "spiders" or "netbots" to visit your website and index your website. However, there is no guarantee that a search engine will list your site and it may be necessary to submit your site several times.

Myth 3: All you need to do is get listed in a few search engines and customers will beat a path to your website and orders will come flooding in.

Reality: Search engines are databases. How well they work is largely dependent upon the database design and criteria built into the search engine's program, and how the user utilizes the search engine and where you are listed. Each search engine has a different approach as to how your site will be listed and where it will be listed, but most allow you use a keyword search. If a potential customer is searching a keyword that's not contained on your home page they may not find your home page.

Also, some search engines like Yahoo! default to a very general search pattern that will bring up thousands of matches whereas a more narrow search or word combination will often bring up fewer results, but more accurate results. Unfortunately, this is not something you can control.

Myth 4: By using the right keywords my website will appear on the first results page.

Reality: Each search engine may have different criteria for creating search results for a given word or phrase search. In fact, some search engines "score" website matches in their database for "relevancy." Relevancy goes beyond matching keywords or phrases, but often looks at the number of sites your website is linked to and how relevant those sites are to the phrase being searched.

Additionally, some search engines look for matches within your home page <TITLE> tag and words that appear as part of your web page content. A strategy would be to create a META tag list using a combination of general and specific keywords or combination of words. Remember, you are relying on the user to pick an appropriate phrase.

Myth 5: I need to be listed in every single search engine that exists.

Reality: While there are now hundreds of search engines or directories, the good news is the top ten are usually where people start looking for information. Specialty search engines that your customers may use to find information is well worth being listed in.

Myth 6: I can list myself--it's just a few forms to fill out and besides, there are lots of free listing services.

Reality: Manually listing with each search engine can be very time consuming, especially if you need to resubmit your website with search engines that do not list your site. There are some "free" services that claim to list you in hundreds of search engines, but often they put your e-mail name on what ends up being spam lists. You get what you pay for and then some. Consider using a site submission service like Submit-It. (www.submit-it.com) While not free, this website offers reasonable site submission services and a wealth of information.

Myth 7: Links from my site to other websites is not important to search engines, or is it?

Realty: The number of links and how relevant they are to your website content can be very helpful in getting your website from the bottom of a search results list to near the top. Link your site to industry websites and non-competitors and update this page on a regular basis.

Myth 8: I can use "phantom" pages with different keyword sets and get listed several times in a search engine.

Reality: While this approach has been used some search engines will disqualify your website and not list you. The idea is to have a number of web pages with metatag information that refresh to your main page. If only one or two search engines are bringing you all the website traffic you need, then this approach may be worthwhile, but if most of the search engines don't list your site, it may be counter-productive.

Myth 9: I don't need to list my website since the search engines send out spiders and robots to find new websites.

Reality: Submitting your site to a search and making them aware of your website will speed up the listing process. In the case of Google, Yahoo and Bing, it is a good idea to be pro-active and to get in line as there can be a waiting list.

Ten Tips to Better Search Engine Performance

1) Use "View Source"

Take a sneak peek at what your competition is doing. Use your browsers "View Source" to look at the HTML code behind the page. You can do keyword searches with several search engines and then look at the source code of those websites that are listing high on the search result page. You can spot their keywords and description and title information right at the top of the source code.

2) Use Meta Tags effectively

Make use of < META > tags. Meta tags are HTML words and descriptions that the user does not see, but which many of the search engines use to catalog your website. Example: < META name="keywords" content="circuit breakers, lighting fixtures, motor controls, lighting fixtures" > and < META name="description" content="Conway Electrical provides a full line of circuit breakers, lighting fixtures, lighting fixtures and wide range of wiring for the general construction industry" >

3) Repetitive Keywords

Try not to repeat phrases or word combinations one right after another. Some search engines will "disqualify" or not list your website at all. For example, "Plumbing,plumbing service,emergency plumbing service" can be reduced to "Plumbing,service,emergency," saving you space for other keywords.

4) Use a Few Keywords

Try to limit the number of keywords and the length of your description. If you use more than say 30 keywords, put the most important or descriptive ones towards the front of your list. If the search engines "cuts off" your list after a limited number of characters, there's a good chance the most important words will be in the search engine database.

Also, only use keywords and descriptions that are within that particular page. Using keywords for sake of listing a keyword in a metatag is not effective.

5) Use a Descriptive Page Title

Be sure to use a descriptive < TITLE > tag. Instead of < TITLE > Your Company Name < /TITLE > try something like: < TITLE >Your Company Name - Specializing in providing circuit breakers, lighting fixtures, motor controls, lighting < /TITLE >. Often search engines will look for matches to keyword searches in the title of the home page.

6) Use ALT Tags

Use < ALT > tags for photos and graphics. ALT tags are used by browsers to display the name of a graphic as it loads or is used by people using your website with Auto Load Images option turned off. Some search engines will take these into account when searching for matches. Example: < IMG SRC="/images/cb01.gif" ALT="Picture of a Circuit Breaker" > .

7) Use Frames and Flash With Caution

Do not use frames, or website structures that rely totally on Flash or special coding to deliver your content. Websites that "hide" content such as website text within frames or within a Flash graphic will be impossible for search engines to examine and list properly.

Search engines treat frames as if they are links within your main page. What happens is while your main page will be indexed, the search engine will return later to index your secondary pages as if they were external links. Be sure the main page (index.htm or default.htm) page has some descriptive text between the < NOFRAMES > and < /NOFRAMES > tag. This descriptive text should contain the most important keywords.

8) Check Your Site's Ranking

Check your website's ranking periodically using several keywords. Also check to make sure your website has not been dropped or disqualified.

Just go to the different search engines and search by your company name. If you have been dropped, go back and examine your < META > tags and other information to make sure you are not causing the search engine to not list your site.

9) Use CMS Admin Tools

If you are using a CMS (Content Management System) to manage your website content, you will have the ability to control the meta tags for each individual website page. It is worthwhile to create the page title, description and keywords for each page.

Also, most content management systems will allow you to create "search engine friendly" page urls, so that instead of an URL of: "http://www.yoursite.com/php?articlesid6+page=item22" you would get, "http://www.yoursite.com/planning-your-project.html".

10) Avoid "Tricking" the Search Engines

Try not to manipulate or trick search engines by "hiding" massive amounts of hidden keywords at the bottom of your document. Search engines will often calculate keywords by how many times they appear on a page and if you have too many repeating words, some search engines may penalize you. Google does not tolerate websites that try to "trick" the system.