There's Gold in Your Website's Server Log

by: Stephen Bucaro

Many Webmasters have never bothered to view their website's server log. Ignoring your server log is like flying with a blind pilot. You may still be cruising along, but you have no idea where you are or where you're headed. If your website is on a free host, you probably don't have access to the server log. However, if you pay for your web host, or if you own the server, you almost always have some access to the server log.

In this article, you'll learn what kind of information a typical server log provides, how to analyze that information, and how to use that information to steer your website in a direction for growth and success.

The actual data in the server log is just that, data. Data is of little use without an application to compile, categorize, analyze, and turn it into useful information. The power of server log analysis software varies depending upon the web host, and it's not related to how much you pay for hosting service. I have seen some very low budget web hosts with powerful server log analysis software, and some expensive web hosts that provide only a limited utility.

The importance of the server log analysis software is second only to up-time reliabilty when it comes to web hosts. A web server log can tell you who is visiting your web site, where they're coming from, what operating system and web browser they're using, what pages of your website they are viewing, and much more. A smart webmaster performs a regular monthly analysis of the valuable information contained in the server log.

Hits

Some webmasters brag about the number of "hits" their website receives, but hits themselves are not important. For example, when a search engine spiders your website, it generates a hit for every page it visits. There are ways to generate millions of hits on a website without anyone ever actually viewing a single page.

The important statistics are "page view hits" and "unique visitors". Server log analysis software should tell you how many times each page was viewed. Anaylyzing the popularity of individual pages tells you where to place your most important advertisements, and what kind of new content you should add to your website.

The king of all statistics is the number of unique visitors your web site receives. When it comes to selling advertising or joining the most exclusive affiliate programs, a high number of unique visitors each month is the statistic that gets respect.

Good analysis software will tell you how many pages each visitor viewed and how long they stayed on your website. The goal is to develop content that gets your visitors to stay longer, a quality sometimes referred to as "stickiness".

A statistic related to hits is "bandwidth". This is a complicated statistic because you want more visitors and that causes bandwidth to go up. On the other hand, high bandwidth consumption makes your website repond slower (and most web hosts charge extra for bandwidth beyond a certain limit). What you want is to design your web pages so they use bandwidth efficiently, and you want to be on the lookout for people who are stealing your bandwidth (usually by linking to images on your web site).

Referrers

Good server log analysis software will provide you with the URLs of the webpages that people clicked on to get to your website. This is extremely valuable information. If you are receiving a high volume of traffic from another website, you can almost consider that website to be your business partner. It would be to your advantage to find ways to make your website more useful to the referring website. For example, learn what the topic of the referring website is and place more related content on your website.

One common method of aiding referrers is to place a reciprocal link on your website. However, don't bother placing the reciprocal link on a "links" page with hundreds of other links (a link farm). If you can't provide an honest reciprocal link, it's better to do nothing. An honest reciprocal link might be a page containing a review of the referrer's website, and that page should contain only a few other links.

Search Strings

Good server log analysis software will provide you with the keywords and search strings people entered at a search site to find your website. One way to get more traffic to your website is to provide more content related to the top keywords. If getting more traffic is your single goal, there are websites that will provide you with a list of the current most popular keywords entered into search engines. Then you can provide more content related to those keywords. This gets involved with the subject of "Search Engine Optimization" (SEO).

SEO is a science in itself, covered in other articles, but I will reveal a secret trick as an example of SEO tactics. Starting with a top keyword, generate every possible misspelling of that keyword and place those at the top of your webpage with the text color the same as the webpage background. (Note: I don't use that trick or any other SEO tricks myself because if you get caught the search engines will penalize your website.)

Platform and User Agent Good server log analysis software will tell you which operating systems and which browser brands and versions people are using to access your website. At the time of this writing, the most common operating system is Windows XP and the most common browser is Internet Explorer version 6. However, you will find a wide range of operating systems and browsers accessing you website. Unfortunately, they are not all compatible.

The fact is, you can't design your website to be 100 percent compatible with every operating system and web browser in use today. That would require either using only plain html (version 3) or using browser detection code and creating a dozen variations of each webpage. Use your server log's platform and user agent data to decide which browser your web pages should to be designed to be compatible with. Your statistics will probably show that 90 percent of your website's visitors are using the latest or the previous version of Internet Explorer.

Country Information

Good server log analysis software will tell you which countries your website visitors are located in. Should you create multiple variations of each webpage to accomodate visitors from other nations? It depends upon your business and your resources. The fact is that the language of business and commerce across the planet is English, and I receive frequent email, in English, from all over the planet.

Important Ways to Use Server Log Information

Website Design

Remember when the typical website had a gray background and a single image in the middle of the page? Today, that would be below the standard for a kids website on AOL. Does your server log "platform" and "user agent" statistics tell you that most of the visitors to your website are using Windows XP with Internet Explorer Version 6, while your website is designed for Windows 95 and Internet Explorer Version 4? Can you afford to be presenting a website that looks outdated?

I think it's a mistake to design a website that requires your visitors to have the latest bleeding-edge technology. On the other hand, if the rest of the web is using ASP (Active Server Pages) to generate custom webpages on-the-fly that use CSS (Cascading Style Sheets) to create webpages that resemble slick magazine pages and DHTML (Dynamic HTML) to provide interactivity, it might be time for you to investigate those technologies.

If your website is a business site or E-commerce site, the visual style and technical level of your website reflects the quality of your business or service. To determine the proper visual style and technology level of your website requires you to monitor three things.

1. Your competitors. You want to look as in-style as they do, but avoid following them down the road to bloated plugins (Macromedia Flash). 2. The portion of your audience using broad band Internet Access. Less than half of U.S. Make sure your webpages load within 20 seconds over a 56kbps modem connection. 2. Your server log platform and user agent statistics. Your web pages need to be designed to be compatible with the current most popular operating system and web browser version.

Webhost Performance

Web hosts provide two basic kinds of service, shared hosting and dedicated hosting. If your website is on a shared host, there may be hundreds of other websites sharing the same computer as your website. This limits the performance of your website. Many web host providers use "bandwidth throttling" to prevent one website from hogging all the available bandwidth.

As your website grows and attracts increasing traffic, it will need more of the host computers processor time and more bandwidth. Monitor your server log hits, and bandwidth statistics. At some point, your website may hit a performance wall and you may need to move to a higher grade of hosting, or a different web host company to continue growth.

When moving to a different web host company, don't trust the specifications they provide. Check several different web host directories that let visitors rate web hosts. If several different web host directories have many users that give a web host company a high rating, that's the company you want to trust with your website.

Website Topic

As you analyze certain website server statistics (for example, "page view hits", and "top search engine keywords") and make adjustments, you may find the content of your website drifting away from your original intent. At some point, you may have to decide, Am I going to chase traffic and change the topic of my website, or am I more interested in focusing on the topic of my choice?

If the original objective of your website was to make money above all else, then let your visitors steer your website's content. There's nothing wrong with that. Everybody's got to make a living. But, if the original objective of your website was to be a resource on a subject that you are passionate about, then you may need to reassess the content of your website.

I would reccommend having the best of both worlds. Stick to content related to the subject of your website, but delineate a wide enough scope of the topic so that when your server statistics point out an opportunity to increase traffic, you have rules to guide you.

In this article you learned some of the types of information you can mine from your web server log and some powerful ways to use that information to steer your website in a direction for growth and success. If you are one of those webmasters who has never bothered to view their website's server log, you now know that you have been overlooking some gold nuggets.

--

Permission is granted for the below article to forward, reprint, distribute, use for ezine, newsletter, website, offer as free bonus or part of a product for sale as long as no changes are made and the byline, copyright, and the resource box below is included.