Posts Tagged ‘web analysis’

Is web analytics an art or a science? Is it primarily a creative or a methodical endeavour? Is it left-brain or right-brain?

Or, is it both? Does it rely on some kind of balance between the two?

I initially started pondering this after reading Steve Jackson’s Cult of Analytics. The book primarily describes an organisational structure and process that can be used to put web analytics at the heart of an organisation. However, it goes much further than this in that it attempts to create an intensely rigorous system of scorecards that can be used to police the delivery of this framework. I also noticed similar thinking in Akin Arikan’s recent call to create an expert system for web analytics, which argues that web analytics should operate much like the field of medicine or mechanics, with concrete processes followed to the letter.

I can certainly see where they are both coming from, but something about this whole thing just makes me feel really uncomfortable. It is undeniable that web analytics is a form of science and computing (it’s got the word ‘analytics’ in it for a start!), but something inside me constantly cries out “no, there’s more to it than that, web analytics is about creativity and intuition and sales and the passion for opportunity!”. You might need the science and the tech in order to understand what’s happening, but can this ever really tell you what to do next? Doesn’t this require a fundamental and intrinsic understanding of business strategy that can’t be reduced to statistics and data; or made into some kind of rules-based process?

But if this is true, and web analytics is a balance between science and art; analysis and intuition – then today the field seems woefully lacking in the art and intuition. But why is this?

Web analytics has a ‘tech’ heritage, but does this fit?

Web analytics originally ‘emerged’ from the field of IT, and was later integrated with the field of business intelligence. This produced a group of people with a huge amount of technical and analytical knowledge, but their role is to report things to other people. They don’t typically get involved with what that information is used for.

But web analytics IS the use of the information. You can’t divorce the information from what it needs to be used for. Anyone who has witnessed first hand an organisation where reporting is handled by IT and optimisation by marketing will know exactly how disastrous this can be. Separating the two creates an uncrossable chasm in the middle. You either need one person with both skill sets (unfortunately quite rare), or a well managed team with both camps working together.

Web analysis is not the same as traditional data analysis

The word ‘analysis’ in web analysis persuades most companies that they should fill their senior web analyst positions with hardcore data analysts. Some companies even go as far as employing people who have previously been analysing things like meteorological or geological data sets. However, whilst it is important to have at least some hardcore stats knowledge in a team, it isn’t necessary at the senior level.

Web analytics tools are easy to use, at least from a functionality perspective. The vast majority of the stats and number crunching has already been done by the software. Anyone who has crossed over from using something like SAS to something like Site Catalyst will understand this. It isn’t analysis in the same sense; it’s report viewing – so the ‘analysis’ is in fact the interpretation and translation of the reports into action, which starts to get much closer to marketing and general business performance than any kind of traditional analysis.

People only see the means, not the end

If someone came to your house to sell you double-glazing and spent an hour showing you the tools they planned to use, and talked about how the plastic was manufactured, you wouldn’t buy the windows! If, on the other hand, they talked to you about the reduction in noise, the increased warmth and the lower fuel bills that you would get, then you would be more interested, right?

For some reason, we have an endemic problem in this industry whereby people obsess over the analysis and data, and not the benefits of the analysis. This results in a perception of web analytics as boring and difficult to understand. If you are presenting recommendations of analysis to senior management, do you really even need to show the analysis? Web analytics is decision support, not a sleep-aid!

—-

The problem then it seems, is that you often have senior people who are too scientific in their approach and lack the spark of commercial intuition and business acumen that can drive truly actionable analytics. This was less of a problem in the old days of data analysis for direct marketing because there were clearly defined process frameworks through which marketing folk could receive standard reports and make decisions based on those reports – but, and here’s the crucial point, there isn’t really anything standard about digital marketing!

I have always said that, one day fairly soon, the word digital will cease to exist; it’s a term used to describe the fact that some things are analogue and other things are digital, but already there isn’t so much left that can reasonably be called analogue. Therefore, if digital marketing simply becomes marketing, and digital business simply becomes business, and these things are as data-driven as we all hope they will be, then ‘analytics’ is a huge central discipline to an entire business operation – to reduce it as a discipline to a fixed process is like trying to create a fixed process for all aspects of the management of a business. Is this possible and, even if it were, is it not suffocating to the organic growth and development of the business?

On a final note, I was curious to see that Avinash Kaushik’s new book, which I haven’t actually read yet, holds the strap-line “The Art of Online Accountability and Science of Customer Centricity” – I wonder what, if anything, he has in store for us on this question?

How do you know whether something is working if you don’t know what ‘working’ means? If your site gets 25,000 visitors a week, is that good or bad? You might reply that 6 months ago you only got 17,000 monthly visitors and therefore it must be fantastic, but how do you know that you haven’t simply moved from really terrible to slightly less terrible?

The way to get out of this is to set good, robust goals for your KPIs – ones which are achievable and based on sound insights and benchmarks. This isn’t just a ‘nice to have’, it is the foundation of a good web analytics measurement strategy or the downfall of a bad one if it’s missing!

Why goals are so important in web analytics

To illustrate the importance: imagine that your website drives offline sales of retail products. Recently you have gradually but consistently improved visitor numbers and congratulate yourself that this will mean more sales in stores. However, unknown to you the market for your particular product has just exploded and there are floods of eager new customers looking for ways to buy it, almost all of whom end up at your competitors. The ultimate effect of this is that you rapidly lose market share in the stores, but of course you never see any of this because you’re too busy congratulating yourself on your 2.3% increase in visitors.

To simplify this a bit – imagine if you owned a shop but never ever left the building or looked outside. You might think you’re doing well because your 10 average customers a day is more like 12 these days, but what if in reality the street outside was teeming with thousands of people and all the other shops had mile-long queues? Would you still think you’re a success?

So, goals – they’re really important! Here’s some tips on how to set them and use them:

How to create goals for your website KPIs

Creating a goal for a KPI is simply about asking first ‘where are we now?’ and then ‘where do we want to be?’ – there are a variety of data and insight sources that you can use to do this. In an ideal world you will use all in combination:

Corporate goals – This is so obvious that it shocks me how often it is ignored by web analysts and even marketing folk. Put very simply: how does your website relate to the overall goals of the business, and what does it need to do to help achieve them? For example, if you have a content site that makes money through ad revenue, how much ad revenue do your shareholders want/need in the next year? You can easily work backwards from this to understand what your goals should be: if you need an extra £85K revenue, how many additional visitors do you need and/or how many extra pages do you want people to view? Simple!

Competitive benchmarking - how do your competitors perform against the same KPIs? Whilst this can be difficult and sometimes impossible to find out, what information can be got is insanely useful. Ideally the data should be linked to corporate goals: if you know that your arch nemesis achieves 5 times the traffic that you do – but you also know that they target a larger and less profitable market segment than your strategy dictates – then you can use your market share estimates to work out what your traffic should be, rather than blindly trying to follow them on a pure number. Good sources of competitive data are commercial providers like Compete, Hitwise and Nielsen; as well as free tools like Google Trends. Failing that you could just ask them. You might be surprised how open they would be if you offered to share your stats in return.

Targeted improvements - Sometimes you just want to push yourself. Even if you have great competitive benchmarking, why stop at matching the competition? Push yourself further! This is about understanding what is achievable and stretching yourself slightly beyond it. Very useful if you have staff that work to bonus targets.

Common sense – if you know your business very well and have a good gut feel about where it can go, you might be brave enough to just use your intuition. Be very careful with this though, an unachievable and unrealistic target is often worse than having no targets – it will lead you on a wild goose chase!

How not to benchmark your performance

I have often seen people set up KPIs and then monitor them using a statistical method called standard deviation. In simple terms this is just a way of making sure that your KPI figures don’t fluctuate dramatically; i.e. they don’t deviate from the average in a given period. If the number improves you can learn from it and roll with it; if it declines you can understand why and rectify it. Whilst this has it’s uses on a day-to-day basis, this is a disastrous way to handle goals. Why would you not want to deviate from the average (mundane)? What else is business except the striving for improvement? No brainer!

Further reading

Avinash Kaushik on benchmarking, goals and more

Coremetrics on industry benchmarking

The Big Book of KPIs by Eric Peterson