Actionable Analytics

Posts Tagged ‘Measurement’

I have just been reading Jason Carmel’s post on Optimization and Analytics, which quite rightly argues that performance marketing may be a better and less ambiguous term to describe what we web analysts actually do on a day-to-day basis. I couldn’t agree more, but the issue obviously goes way beyond terminology; and the post actually reminded me of a recent discussion with a client on exactly the same topic, which might be worth sharing.

What is web analytics anyway though?

The real root of this issue is the fact that many companies fail to see what the real goal of web analytics is. They see it as something extra that might be useful, but only when they get around to it and when they don’t have anything more important to do. In the mean-time they carry on as normal; churning out emails, scheduling site updates, adding and removing things to the home page – all based on gut feel or, as Avinash Kaushik likes to put it, the HiPPO (Highest Paid Person’s Opinion). What they don’t understand is that all this stuff they do and web analytics are actually one and the same! Talking about performance marketing not only makes web analytics seem less geeky, it brings to light the fact that our ultimate output IS marketing.

Performance Marketing is a much better clarion call

The specific client I was talking to had exactly this problem; because they didn’t understand web analytics they just couldn’t connect it mentally to their own jobs. In this situation it’s no good talking to people about maturity models or measurement frameworks, or trying to train them on tools, because they still won’t get it. You need to educate them about why they should even listen to you and, more importantly, you need get them excited about why they need to be involved. This is how I went about it on this occasion:

Step 1 – Show them why they need it

The client was under immense pressure to deliver results with a reduced budget, and couldn’t see any way of doing it. They dismissed all notion of ‘web analytics’ because it sounded expensive, time-consuming and like something that wouldn’t deliver immediate results – i.e. they didn’t get it. The first step was to try and show them (without talking about analytics) that they needed to be cleverer about their marketing:

The Need for Data-Driven Marketing

This chart is specific to this client’s market and situation, but what it actually says isn’t so relevant. The key point is that mass marketing is no longer effective, even if you have got the cash for it. Customers are more individual than they use to be, and so you need to get closer to them and have more genuine conversations with them.

Step 2 – Make the connection

Nobody can really argue with what you’ve just said, and then the line of argument progresses in this fashion:

1. The ability to be pro-active and to successfully affect consumer decisions is reliant on the ability to listen, learn and to communicate genuine value through intelligent dialogue.
2. In an online environment listening and learning is achieved through web analysis, measurement and research; understanding how customers currently interact with us and how they want to interact with us.
3. Intelligent dialogue is achieved by optimising the customer experience in order to communicate our message in the most appropriate way, based on what we have learned by listening and understanding.
4. The process is only possible if the data, tools, capabilities and the methods for using them are available and tuned in to what we want to know, and so careful planning is required in order to ensure that insight can become actionable.

This describes holistically the whole process of marketing based on listening, which can also be called performance marketing. Then you can start talking about how to enable them with the ability to actually do it. At the heart of this is the ability to streamline and simplify the flow of data so that decisions can be made:

Enabling Performance Marketing

If data and tools are faster, easier, better and generally more efficient at providing meaningful insight, then your staff are able to spend more time generating action based on that insight and less time trying to work out what it means. This, in turn, means that more attention can be focused on optimisation and improvement initiatives that drive increased performance; and the final result of this is that dialogue and relationship with the customer becomes more tailored, more meaningful and more effective.

Now you can talk about analytics!

Only then can you start to have discussions about maturity models, vendors, internal or external consultants etc etc. It might still be a very long slog, but at least your client (or boss or whoever) can understand what the end game really is.

The other day I was presenting the findings of some analysis to a client. The focus of this analysis was to discover the behavioural factors affecting checkout completion rates in order to shed some light on why people drop out. For example, within this analysis I was able to say fairly basic things such as:

• Visitors who spend a lot of time on the site before their purchase are less likely to drop out of the checkout process than those who’s session is shorter
• Visitors who land on the homepage are more likely to drop out of the process than those who land on a product page

Now, the client immediately got rather excited about this and began to say, regarding the first point, “Wow, so if we can increase the time on site then we can improve our drop-out rates. Excellent, how do we increase time on site?”. Had I allowed it, this person would no doubt have been rushing back to the marketing team with a new objective to get the time on site up!

So what’s wrong with this? Well, apart from the very numerous issues with dwell-time associated with this specific example, this represents a very common misunderstanding in web analysis. Put very simply:

Your customer didn’t complete their purchase because they were on your site for a long time. They were probably on your site for a long time because they are interested in your products and your site is relevant to them which, in turn, means they are more likely to complete their purchase. Increasing dwell-time per se doesn’t make any sense in this example because it isn’t the cause.

To provide a simpler example of this: you might notice that people who dress smartly often have quite tidy hair as well. Does this mean that dressing smartly causes tidy hair? If I put a suit on, will my hair instantly become much tidier because I’m wearing a suit? No, there is some other factor (the person’s need to look smart) that is driving both of these things.

This leads to all kinds of problems in web analysis, some of which are quite subtle. Furthermore, this problem is inextricably bound up with our obsession with click-stream; if we can’t see beyond the web analysis tool then we have to find our causes within it. The biggest danger is that we stop being able to see our visitors as real people with real needs, and instead just view them as lines of data or collections of behaviours.

A couple more examples of how this can cause problems:

• You notice that direct traffic is of a higher quality than other sources. Does this mean that you should simply get more people to come to you direct? You could do this by displaying your URL as a static image in non-clickable banners, meaning that people have to physically type it into the browser. Again, no. Real direct traffic is direct because of brand familiarity and relevance, which may have nothing to do with advertising. The pure fact that it’s direct is of little relevance. (by the way, be careful – direct traffic isn’t always what it seems)
• You notice a correlation between downloads of your latest white paper and calls to your salesteam. Excellent, the white paper is a succesful acquisition tool and is driving leads! Or is it? Which way round is it really? Are people calling you because they downloaded the white paper, or did they look at your site and dowload the white paper because they called you?

Remember, correlation does not imply causation! You can avoid this by remembering that your customers are real people with needs, desires, habits and lifestyles. They are not lines of data with dwell-times, page counts and completion rates. These things are only behavioural indications of something else more complex that is happening. Look beyond the click-stream and understand how your customers think and feel.

Drowning in Data

The concept of ‘drowning in data’ cannot be understated when it comes to web analytics. Apart from the sheer quantity of information available, the situation is worsened because the tools we use are so terribly fast and effective; it has never been easier to slice, dice and peel (?) your way through such huge mountains of click-stream data. But just because it’s there and easy to access certainly doesn’t mean it’s easy to make sense of. I believe most companies that fail in this arena do so because they simply don’t know what to look at, but rather flail around in the data following endless and infinite pathways that, whilst ‘interesting’, ultimately lead nowhere fast.

This post describes how to get your head above the water and start swimming in a straight line. The answer lies in what I call a ‘Measurement and Optimisation Framework’, which might sound complicated but is, in fact, simply a strategy for: what you should be measuring; how to do it; and what you should do with the information once you get it.

Developing a Measurement & Optimisation Framework

The process of developing a measurement and optimisation framework is simply about answering 3 key questions:

1. Why does my website exist?
2. How can I measure the success of that existence?
3. What can I do to make it more successful if I find it isn’t achieving what I want it to?

For very simple websites (such as a personal blog), you could probably get away with just spending an hour or so thinking about this. For more complex business sites it could take some time! Following is a brief summary of the top-level part of the process through which I would typically take a client in order to get this up and running:

1: Define your site’s KPIs

How can you fix something or make it better if you don’t know what it was meant to do in the first place? Not setting proper objectives and goals is the most serious and fundamental mistake anyone can make, and not just in web analytics!

Most companies fail to do this because they assume that they intrinsically know what their site is for and what needs to be done to improve it. Take the example of a site selling CDs – it’s for selling CDs, right? What could be more complicated than that?

But, think about it for a moment, who is it trying to sell CDs to? Is it trying to achieve the lowest price possible or is it selling at a premium because it caters to a niche? And where does the company want to be in 5 years time, and what does that mean in terms of the brand that needs to be built? Is it important that people tell their friends about it? Oh, and how does the profitability work? Do we need to reduce the cost per sale by increasing the number of repeat buyers and therefore reducing media spend? And what about our other sales channels? Sales on the web cost much less than those that go through the call centre, so do we need to persuade some of those customers to get on-line? etc etc etc…

The point is, what your website means strategically is not necessarily all that easy to articulate. You need to get a really firm grasp on what your companies corporate goals are and work downwards. For big companies this generally means using something like a Balanced Scorecard approach. The system you use isn’t necessarily important, the point is that you align the goals of your site with the strategic goals of the company or, better still, the strategic goals of your customers!

2: Set targets

Once you have defined how to measure success (your KPIs), you then need to determine what that success IS. Again, this goes back to your corporate objectives: if your site is there to generate advertising revenue, how much revenue do your shareholders need next year? And what does that mean in terms of the number of visitors you need and the number of pages they need to look at? This is how you set targets.

Even if you can’t get anyone in your company to give you these targets, you should make them up yourself! It is incredibly difficult to optimise something to work better if you don’t know what ‘better’ means. If you are not able to prioritise which areas of the site need the most attention at any one time, you will drown – and you cannot do this without a sense of the goal for each KPI. Just do it!

At this point in time you might be able to produce something like this:

Typical Web Analytics Measurement Framework

[Please note: I doctored this a lot to protect the identity of a client, so it won't necessarily make complete intuitive sense and is provided more as a visual example]

3: Guide your analysis with a KPI dashboard

Now you know what your KPIs are and how to measure them you can produce a dashboard report showing where they are against where they need to be. This is incredibly important because it is the guiding light of your analytics and tells you exactly what to look for. If, taking the example KPIs in the chart above, I produce my weekly or monthly dashboard only to find that my unique visitors are dangerously below target but that all other KPIs are OK, then all my analysis for that week/month will be guided by a very specific question: what drives unique visitors and how can I improve the volume?

By investigating this you might find, for example, that you have saturated your search market and therefore need to optimise the site for different, non-branded keywords – or that the TV campaign you tested sent lots of high quality traffic and should be repeated. The point is that, without the KPIs, targets and the dashboard, you have nothing to focus you and, more importantly, have no solid way of telling your marketing director why they need to spend more on TV!

4: Optimise, optimise, optimise!

Remember finally that web analysis is not about understanding, its about doing. If you think your job is to report figures to someone else so that they can make sense of them, then you are not an analyst. The output of everything you do is about making changes to your site, media strategy, internal processes or whatever. Analysis and optimisation are essentially the same thing!

Empower yourself!

So what’s the benefit of all this? If it isn’t already obvious think about these two possible scenarios in which you are presenting your ‘analysis’ to your wider team:

• You hold a meeting in which you present 30 charts of data from your analytics tool, moving through geography, time on site, hour of day, browsers, screen resolution and lots of other fascinating charts. At the end everyone agrees that it was really interesting and goes back to their jobs.
• You hold a meeting in which you state that you can make the company an additional £1.5m per year in sales revenue and then proceed to present a road-map for implementing changes to make it happen, with a full ROI justification of likely costs. You get promoted and paid more!

Which one would you prefer?

I was driven to write this post after chatting to the online marketing manager of a large international company, who proudly told me that ‘dwell-time’ was now one of their most important KPIs; and that they had issued instructions to all local marketing teams that the primary focus for the coming year was to ‘increase dwell-time’, thereby getting customers ‘more engaged’. I suggested that they make the pages take longer to load. He didn’t get the joke!

In seriousness though, this is a very common example of the way many companies view their websites. Personally I think it might come from too many years dealing with traditional offline media – “if only we could find a way to get people to look at our bill-board for longer, and pay more attention to it!” But beware…

The danger of dwell-time

In most cases measuring dwell-time as ‘engagement’ (or even at all) is not only wrong, but is frankly dangerous. Just a few of the reasons for this are as follows:

• A lot of your visitors are at your site because they want to get something done, quickly: place an order for something they decided to buy last week; find your address; get help; and so on. Why do you want this to take longer? If you ran a supermarket you might want people to spend longer browsing the aisles, but would you want them to have to queue for longer at the check-out??
• I might spend 2 hours ‘engaging’ with every aspect of your site, but that might be because I despise you and am learning everything about you so I can destroy you! This is extreme, but the point is that engagement isn’t necessarily positive engagement.
• Most companies find, if they run the analysis, that people who buy things spent longer on the site than people who didn’t. This leads them to think that if they can get people to spend longer on the site then they will surely buy more stuff. This is one of the biggest errors I see in web analytics, and not just regarding this example. People who buy things don’t buy things because they were on the site longer, they were on the site longer because they were in the mood to buy something, or because your site was relevant to them. Simply getting people to stay on the site longer doesn’t change their state of mind, and by obsessing over it you ignore the real underlying drivers.
• If what you really want to do is get people more engaged with your content, and get them to think positively about it – why not just measure that? Do a survey or run some focus groups; ask them what they thought and, if they don’t like it, ask them why not and how you can improve it. This kind of brand engagement is a deeply emotional and qualitative thing – how on earth do you expect to correlate it to something so cold and bland as the time they spent on your site?

But there is something more fundamental underlying all this. I think in most of these cases companies (especially non-ecommerce sites) are unsure what their website IS; what it means to them strategically and, more importantly, the role it plays in the overall journeys taken by their different customer segments. How exactly do you want the content on your site to influence your customers’ behaviour? Do you even know how your customers are using the site at the moment? Until these questions are answered (quantitatively and qualitatively) you will never be able to meet them in relevant dialogue through your site. And if you really think this through, and then think back to the concept of pure dwell-time – how absurd does that sound now? It’s like locking the doors of the shop and not letting people out!

But we are trying to achieve something, so what is it and how do we go about it?

Nevertheless, websites do have a communicative role to play. Our visitors need to be influenced, motivated, persuaded, dazzled, awed – not just to make them buy something, but so that we become part of their lives in whatever way is relevant to them. So how do we do it? Well, unfortunately the answer to this question is deeply unique to every single business – you need to go on your own voyage of discovery in order to understand exactly what ’success’ and ‘performance’ mean to you and therefore how to influence them. However, here are some tips to set you off:

• Push the site itself (and especially anything to do with click-stream data) out of your mind temporarily. Work out who your customers are and why and how they want to interact with you as a business. Similarly, work out how you want them to think of you, and what role you want to play in their lives. Now, in the middle of all this – what does/might the website mean to them; how does it help them; what would make it important to them? If you have the budget I would strongly recommend this being a major research project.
• Remember that you don’t just have one type of customer, and even similar customers want different things at different times. Segment your customers by who they are and what they want to achieve, and make sure you understand the above question according to these different types of customers. What role does the site play for them at the current stage in their journey with you?
• Ensure that your objectives and KPIs reflect this understanding. If by engagement you really mean that all visitors successfully completed what they came to do, then ask them whether they did or not and use this as a KPI. If the journeys and tasks that people want to perform are totally different, then you need different KPIs.
• If things like dwell-time are still relevant to some of these journeys then use them, but remember and take heed: these are indicators of other behaviours or attitudes. You cannot influence this metric directly. Know what drives it!
• Never rely solely on click-stream data as your source of insight. Sometimes it is easier for continual reporting if all KPIs are based on click-stream, but if this is the case then you need to make sure you explain and drive these metrics using other, qualitative sources of data. Click-stream is the what, not the why!

Above all, remember that your website is not and will never be a ‘pamphlet on the web’. You might think of it like this, but your customers most certainly don’t. These days brands sink or swim based on how effectively they ‘engage’ with people through digital channels, but this ‘engagement’ is a million miles away from ‘dwell-time’!