Analytics vs Analysis
A gentleman whom I’ve learnt a lot from over the last few years, once mentioned to me that he wanted to write a book, called the Business of Measurement. Since he hasn’t written it yet, I will humbly provide this as a potential preview to what the book might be about…
Most sites have analytics so they can track whatever statistics they consider important.
Most times, this falls into 3 categories :
- Page Views
- Visits
- [Unique] Visitors
If you’ve dealt with me ever you are aware of my views on the flaws inherent to the definitions of the above. For those of you lucky enough to have never had to deal with me, here’s a quick definition of each
Google Analytics Dashboard
Page Views : :
Let’s assume that you came to this blog, via a search engine, and landed on this particular post (http://dragonsbehere.com/weblog/e-commerce-platform/analytics-vs-analysis). In my Analytics solution now this page will record an increased Page View count of 1.
If you were to reload this page, that number would go up again.
If you were to find my writing fascinating, and go to other posts, those pages would also incur an increased Page View Count.
Page View has become a slightly muddy issue with the rise of AJAX. What exactly counts as a Page View, when sections of the page change without the page reloading? What percentage of the page has to change before it is a new Page View? What about just counting the elements of the page?
There’s also the issue of Javascript vs Cookies vs Sessions etc. What happens if someone disables Javascript, or third-party cookies? Some of these issues were brought up at a panel a while ago (a long while ago). You can read the answers here.
Visits ::
You’re a visitor. You opening this page on your computer constitutes a visit to my weblog (for which I thank you). Visits and Page Views usually have a 1 to many relationship – i.e. for every visit, at least one page view is recorded, and possibly more (see above regarding my fascinating writing skills). It is impossible to have more Page Views than Visits.
Obviously Visits suffers from the same problems as Page Views (Ajax, Javascript & cookies disabled) etc. It also has its own set of new issues – what constitutes a visit? What happens, if I leave the page open for session.length + 1 minute? Maybe we could get around that with an infinite session, but then, what is that session based on? Cookies? What if they are disabled, or deleted? How about storing session data on the server? (Note, for the “Shared Nothing” purists, I support your views, just making a point. Also I think it is feasible to do sessions via Db – even in a shared nothing scenario.)
Regardless – as you can see, all approaches have a positive and a negative side to them
[Unique] Visitors::
Visitors, fairly self-explanatory – for example, you’re a visitor to my blog. Unique Visitors is meant to distinguish between people making multiple visits to the site, and just the number of people visiting the site. Visitors should have a 1 to many relationship with Visits. i.e. for every Visitor, we should record at least one visit. Similarly, Unique Visitors should have a 1 to many relationship with Visitors. You can’t have more Unique Visitors than Visitors. Similarly you can’t have more Visitors than Visits.
Visitors have their own share of issues – what happens if you visit the website from home, and then again from work? What happens if you access it from a public computer (such as a library)? What happens if you can’t store cookie (because you work for the Government) and all visits from your office (lets say – the Pentagon for example) come from a single IP address? Should all those visits be classified as a single visit?
Now, these are just the start of the Analytics part of the discussion. We still have to get to sources of traffic, pathing, landing pages, bounce rates, goal achievements etc. and we will – in future blog posts.
More importantly, what is the point of all these statistics? How does it matter how many people visited your website? Or how many Page Views you generated?
Now, that’s the analysis portion. Sure you can get a metric ton of information on what is happening on your website. Yes, there are plenty of flaws in the gathering of data inherent to the fact that the Internet is a fault tolerant system, and analytics is traditionally not.
What you do with this data is where the secret sauce lies. For example, Page Views could indicate what pages are popular, which in this case (a blog) indicates what content my audience is interested in, and what it isn’t.
Maybe I want to cater to what my audience is interested in, or maybe it indicates that I need to go seek out the audience that would appreciate the rest of my content as well. For example, in my case, the Technology posts get much higher traffic than the E-Commerce posts.
Regardless, analytics without analysis is almost pointless. I say almost because the mere act of measuring something, can cause us to change our behaviors to improve it. Which is the very point of this post. Make sure you are measuring what you want to improve. Make sure your analytical output is closely tied to your business goals.
Let’s take a rather simplistic example :
You run an e-commerce store that experiences the following basic traffic patterns :
- Visitors – 100 /day
- Visits – 200 / day (or two Visits per Visitor)
- Page Views – 400 /day (or two PV per Visit, or four PV per Visitor)
The website experiences a conversion rate of 1% based on Visitors (so 1 out of every 100 Visitors will engage in some action that the business considers worthwhile.)
Now, lets say that the business is measuring Page Views for instance, and decides to increase that number, hoping for it to all trickle downstream – leading to an increased bottom line. Well, there’s a number of ways to do this – let’s consider two of them that I’ve seen used in multiple places :
- Buy Traffic
- Change your UI to require multiple Page Views to achieve the same result.
Both these strategies should increase Page Views. However, the trickle down effect of increasing Page Views for an act of conversion is debatable. Will the bought traffic actually convert at the same rate as the previous (possibly organic) traffic was converting? Maybe, if its bought from a fantastically targeted program. Changing your UI to require more Page Views seems like it would definitely hurt your conversion rate – but in some cases (usually involving eyeballs on ad’s) it might help your bottom line.
As you can tell, there is no silver bullet for the analysis aspect. I can only suggest what my mentor told me. Understand the bottom line. Once it is known what the business measures itself against, whether that’s sales, or profitability, or user happiness, or some combination of those : once that can be succinctly explained, then you know what to measure.
You can’t improve what you can’t measure. Corollary : measuring the ‘wrong thing’, will lead to improving the wrong thing, possibly at the cost of the ‘right thing’.