Web Design Discussion

I only enjoy fishing when I catch lots of fish. Sometimes I fish for hours without catching anything, become bored and want to quit. Often the reason for not catching anything is because I am using the wrong bait. You need to use bait that is appropriate for the type of fish you wish to catch.

Read the rest of this entry »

Yahoo! recently announced a way you can specify what sections on your page you want crawled by their search engine spider. If you have parts of your web page that are unrelated to the main content (ie navigation, footer, etc) you can insert the “robots-nocontent” tag.

Read the rest of this entry »

To read my other posts about the robots.txt file, please visit:

What is robots.txt file?

How do you create a robots.txt file?

If you incorrectly format your robots.txt file, your web site files may not get indexed by the search engines.

Read the rest of this entry »

Please read What is a robots.txt file?

How to Create and Use a robots.txt file

1. Open notepad and save a file named robots.txt

2. Create records

The content of a robots.txt file consists of so-called “records”. A record contains the information for a special search engine. Each record consists of two fields: the user agent line and one or more disallow lines.

User-Agent: [Spider or Bot name]
Disallow: [Directory or File Name]

Here’s an example:

User-agent: googlebot
Disallow: /cgi-bin/

This robots.txt file would allow the “googlebot”, to retrieve every page from your site except for files from the “cgi-bin” directory. All files in the “cgi-bin” directory will be ignored by the googlebot.

3. Upload the robots.txt file to your server.

Make sure you upload the robots.txt file to the root directory (same location as your home page).

Read the rest of this entry »

Sometimes you may not want the search engines to spider specific directories of your site because you don’t want the informationto be read by the public. You can accomplish this by creating a robots.txt file and placing those files in it.

Read the rest of this entry »

I always create an html site map for every site I build. It helps search engines index all the pages of your site. This is especially important if your home page does not contain a lot of text ie has a flash introduction, includes many images, uses javascript or is a
database-driven site.

The search engine spider will find all the other pages of your site (besides the home page) by following the links that link from the html site map link on the home page.

Read the rest of this entry »

Read Web Site Content Guidelines - Part 1 No. 1-4

5. Don’t use hidden text or links

Some site owners try to trick the search engines by hiding links in the text so visitors inadvertently get taken to another site or create hidden text to get more keywords in their html code. This will be seen by Google as spamming the search engines and your site may get penalized. If penalized your site may disappear from Google and will take a long time to get it back. Therefore stay within the Google Webmaster Guidelines.

Read the rest of this entry »

Whenever you optimize a web site to improve your rankings in the search engines, it’s a good idea to be familiar with the Webmaster Guidelines from Google. If you don’t follow them you may suddenly find your site penalized for using incorrect techniques. Some of these may include hidden text or links, repeating your keywords too often in your web pages or using link schemes to artificially inflate your rankings. If your site gets penalized, it may no longer appear in Google or any of Google’s partner sites.

Read the rest of this entry »

Themed Search Engine Directories are directories related to your niche web site or specific topic of interest. They get you highly targeted incoming links. The quality and quantity of incoming links will determine the page rank of your site. The higher the page rank the more visitors you attract to your site.

Read the rest of this entry »

Paid inclusion means a search engine company will charge a fee to include your web site in its search index. Most search engine companies these days now charge a fee except for Open Directory Project and Google.

Benefits of Paid Inclusion to Search Engine Directories

Read the rest of this entry »