Actionable Analytics

The other day I was presenting the findings of some analysis to a client. The focus of this analysis was to discover the behavioural factors affecting checkout completion rates in order to shed some light on why people drop out. For example, within this analysis I was able to say fairly basic things such as:

• Visitors who spend a lot of time on the site before their purchase are less likely to drop out of the checkout process than those who’s session is shorter
• Visitors who land on the homepage are more likely to drop out of the process than those who land on a product page

Now, the client immediately got rather excited about this and began to say, regarding the first point, “Wow, so if we can increase the time on site then we can improve our drop-out rates. Excellent, how do we increase time on site?”. Had I allowed it, this person would no doubt have been rushing back to the marketing team with a new objective to get the time on site up!

So what’s wrong with this? Well, apart from the very numerous issues with dwell-time associated with this specific example, this represents a very common misunderstanding in web analysis. Put very simply:

Your customer didn’t complete their purchase because they were on your site for a long time. They were probably on your site for a long time because they are interested in your products and your site is relevant to them which, in turn, means they are more likely to complete their purchase. Increasing dwell-time per se doesn’t make any sense in this example because it isn’t the cause.

To provide a simpler example of this: you might notice that people who dress smartly often have quite tidy hair as well. Does this mean that dressing smartly causes tidy hair? If I put a suit on, will my hair instantly become much tidier because I’m wearing a suit? No, there is some other factor (the person’s need to look smart) that is driving both of these things.

This leads to all kinds of problems in web analysis, some of which are quite subtle. Furthermore, this problem is inextricably bound up with our obsession with click-stream; if we can’t see beyond the web analysis tool then we have to find our causes within it. The biggest danger is that we stop being able to see our visitors as real people with real needs, and instead just view them as lines of data or collections of behaviours.

A couple more examples of how this can cause problems:

• You notice that direct traffic is of a higher quality than other sources. Does this mean that you should simply get more people to come to you direct? You could do this by displaying your URL as a static image in non-clickable banners, meaning that people have to physically type it into the browser. Again, no. Real direct traffic is direct because of brand familiarity and relevance, which may have nothing to do with advertising. The pure fact that it’s direct is of little relevance. (by the way, be careful – direct traffic isn’t always what it seems)
• You notice a correlation between downloads of your latest white paper and calls to your salesteam. Excellent, the white paper is a succesful acquisition tool and is driving leads! Or is it? Which way round is it really? Are people calling you because they downloaded the white paper, or did they look at your site and dowload the white paper because they called you?

Remember, correlation does not imply causation! You can avoid this by remembering that your customers are real people with needs, desires, habits and lifestyles. They are not lines of data with dwell-times, page counts and completion rates. These things are only behavioural indications of something else more complex that is happening. Look beyond the click-stream and understand how your customers think and feel.

3 Responses to “Errors of causation in web analytics”