Analyzing Web Traffic

Gathering information on web site traffic is a crucial element in the planning of marketing and further development. When analyzing web site traffic, it is also important to be able to understand the associated statistics terminology.

A question that webmasters are often asked is "how many hits does your site get?". You then need to ask "do you mean hits, requests, unique visitors or page views?".

Many hit counters, such as the FrontPage 2000 component can be a little misleading and display grossly inflated statistics. This particular hit counter only measures page views and if you sit on a page clicking the "refresh" button, it will increment.

As FrontPage and similar counters only reports numbers, it is a very poor method of gaining an accurate view of traffic and the way visitors interact with your site - you'll need a more comprehensive tool.

The following is an outline of different methods of statistics collection, plus definitions of associated statistics terminology to help in analyzing web traffic:

 
Raw server logs - manual statistics gathering

One of the best methods of studying traffic, particularly if you wish to really narrow down on a certain page, browser, IP address or search engine referrals is to download your server logs. A server log is basically just a text file that contains every request that has been made to your server. A request may be for a page, an image or any other element contained on your web site.

A server log entry looks something like this:

216.154.251.84 - - [14/Aug/2005:08:04:14 -0800] "GET /animate.js HTTP/1.1" "http://www.sfldfkfdk.com/" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"

It all looks a little confusing, but it's really pretty simple when you break it down:

216.154.XXX.XX - This is the IP address of the person requesting an element from your site (x's will display as numbers). You can free tools from sites such as DNS Stuff to then translate those numbers to a geographical point.

[14/Aug/2005:08:04:14 -0800] - The date and time of the request

"GET /index.htm HTTP/1.1" - The file being requested and the method of transfer.

"http://www.sfldfkfdk.com/" - Where the request came from. This may be another web site or a search engine. If it was a search engine referral, it's most likely that the search string will also be included.

"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)" - The user agent (browser or search engine robot) and operating system of the person/computer requesting the file.

The manual study of server logs can reveal many interesting trends and habits, but it's also very time consuming. Each visitor to your site may generate hundreds, perhaps thousands of requests on each visit depending on the complexity of your site.

Manually dissecting your server logs is great for in-depth studies, but you'll also want a tool for "at a glance" statistics. There are many server log interpretation tools on the market, but the good ones are generally very expensive. Many hosts include statistics tracking with their hosting packages, but I have found many to be inaccurate or don't update often enough. A third party service is also useful as a comparison tool.

Also, while you'll have access to logs if you are commercially hosted (ask your hosting service about their location), webmasters hosted on free servers usually won't have this privilege.

If you don't have access to your server logs, or are looking for to complement your own server log analysis, there are many remotely hosted services available. Remotely hosted services are usually very simple to set up - simply copy and paste a few lines of code into your pages. Learn more about these services.

Web site statistics terminology

Webmasters and site owners tend to use different terminology in relation to statistics and it's important to be able to differentiate between the terms - especially if you are wanting to attract advertisers to your site. Savvy advertisers will want to know exactly what your traffic rates are, where it comes from, what your visitors come to your site for and how long they stay.

What is a hit or request?

A hit is the result of a file being requested and served from your web site. This can be a html document, an image file, an audio track etc. Web pages that contain a large number of elements will return high hit scores. Hits are of very little consequence when analyzing your visitor demographics.

What is a page view?

A page view means just that. Once again, it is not a true indication of how many different people are visiting your web site, but it is a good way to judge how "sticky" (the ability to retain the interest of visitors) your web site is and is an important consideration regarding the possibility of attracting high paying advertising.

What is a unique visitor?

A unique visitor (combined with page view data) is where stats really count. It is someone with a unique IP address who is entering a Web site for the first time that day (or some other specified period). When you log onto the Internet, you are assigned a unique IP address, or if you are a cable modem user your IP address is usually "static", meaning that it never changes.

Your IP address is an identifier and while you are using it, no one else on the Internet can utilize that particular set of numbers. Your number is counted once, usually for a 2-24 hour period, dependent upon the tracking software.

So no matter how many times a visitor refreshes or navigates through your web site, they will only be counted once for the specified time period. This is by far the more accurate way of analyzing web site performance.

Note: the notion of a "unique" visitor as described above can be somewhat misleading in the instances of multiple users sharing the same IP. For example, the proliferation of home networks in recent years may see you having 2 or more people from the same network accessing your site via the same originating IP. This can also occur in the cases of universities accessing your site. In these circumstances, it is more difficult to tell if 1 person from the same IP is viewing 50 pages, or if it's 10 people.

If you really need to break down figures from these sources further, you'll need to do some more detailed log analysis; comparing items such as access times, browser/OS configuration etc.

The other way to get a *reasonably* accurate visitor count is to look at the overall page view average trend for your site. For example, if the page view average for your site is 3 pages per visit, and you have one IP showing a page view count of 200 for a period, then you could divide it by 3 to give you a "guesstimate" of how many visitors originated from that IP.

What is a referrer?

A referrer is simply the origins of the visitor to your site, i.e. the last site visited and the page on that web site.

Gauging web site traffic

When you divide the number of page views by the number of unique visitors, this can give an excellent indication of whether traffic is transient or is staying on your site. If the average is one page or under, you can be pretty sure that there is something on your pages that is scaring people off.

Perhaps the load time is too slow or your landing page isn't well structured. Remember that due to bandwidth considerations, those first few elements that display as your page is loading may be the deciding factor as to whether a visitor waits around for the entire page to load.

Analyzing web traffic can take up a fair amount of time, but it is definitely worth it.

Further learning resources:

Free web site traffic reporting services

Identifying search engine spiders

Creating effective landing pages

 

Article by Michael Bloch
Taming the Beast
Tutorials, web content, tools and software.
Web Marketing, Internet Development & Ecommerce Resources