Zen & The Art of Your Raw Access Logs

I am constantly discovering new things about my network of Web properties just by simply looking at the raw access logs on my different servers. Sure, I use Google Analytics for getting the daily ‘big picture’ of all my sites, but I get the most valuable data (and definitive action items) from actually looking at my weekly raw logs line-by-line.

For one thing, in the raw logs, I can see user activity on my sites on an extremely granular level…I can see every broken link, every query typed into an engine, every 304, every 404, every 500, and every path a user takes…absolutely everything!

With a service like Google Analytics, you sort of need to know what you are looking for, and if you want specific information, you have to know how to ask for it. By looking at your raw access logs, you are really putting yourself in the shoes of your visitors…you are actually taking the exact same path and basically having the same experience that they did (in your minds eye, at least). Sure, it takes a little while to get used to “seeing” user behavior by looking at time stamps and cryptic server codes, paths, etc., but if you do it often enough, it becomes second nature and simple to do.

You really should practice this once a week as I do…I actually find it very relaxing and quite Zen (as well as financially rewarding).

Google Webmaster Tools Quick Start Guide

If you’re not using Google’s free webmaster tools to understand how Google indexes your domains and sites you’re missing an important source of data. Google has created a simple Quick Start Guide for Webmaster Tools that can get you started.

Even better, FeedTheBot offers a great tour through Google’s webmaster Guidelines created by an SEO guru who really knows his stuff (and it was supposedly written while he was homeless and living in a tent!). A very clear and simple reintepretation of the guidelines.

The Value of a “View”

Running the numbers on a developed site can be a little confusing but understanding how they work can really help you value a site and determine where to spend your time and money.

I track AdSense on quite a few sites. Here’s how I parse the data:

  • Make sure you set up channels for each site and ad unit. This is absolutely essential.
  • Look at your thirty day totals by channel/site to determine highest performing site(s)
  • Remember that Google measures by impression not page view. Three units per page (the max they allow) means 3 impressions per page view
  • Google eCPM rate is your earnings per 1000 impressions. If you have three units per page then you have to triple that number to get your average revenue per page view

An example: We have a top performing site that, using the tools above, returns 4.8 cents per page view or $48 per 1000 page views. This was surprising to me but the knowledge meant that we needed to focus a lot more attention on driving traffic to this site since even a slight daily bump adds up really fast.

The same site averages 2.9 page views per visit so a visit is worth an average of $.14. A thousand visits a day is worth around $140 so spending anything up to that amount daily to drive traffic is worth it, remembering that sites generating $50,000 in annual revenues (which is about what those 1000 visits per day gets you) are worth a multiple of those revenues to a buyer, probably at least $300k. You could argue that spending far more to drive traffic makes sense because you’re losing in the short term (aka, investing) but building equity in the long term.

Know your numbers- you can easily set up a spreadsheet to track them.

BTW, the site I’m using as an example is in an enormous, information-hungry consumer market involving a lot of spending. I’ve also left affiliate revenue out of the equation to simplify things.