Once upon a time the objective of a digital advertising brief was much the same as an off-line brief: to communicate a sales message to people in order to get them to buy a product. The only real differences were the way in which that message was delivered and the way in which the consumer could purchase the product.

However, in recent years there has been a dramatic shift away from this model. The traditional set-up of websites, micro sites and banner ads is increasingly being replaced by a new breed of apps, networks, engagement platforms and digital tools. Interrupting people with overt sales messages is out of style. Most of the time these things have almost nothing to do with the actual product the company is selling.

But how then do we measure these new experiences? The old model of reach, click-through and conversion isn’t really any different to direct-mail and was easy, but our brave new world often has very little to do with that old model. In order to understand how to measure this we first need to look at what these tools actually are and why they came about in the first place.

[N.B - I have never directly worked on any of the case studies I use in this post and use them only as examples or for hypothetical ideas]

The need for branded utility and customer engagement

Branded utility, and indeed the whole ‘customer engagement’ movement is born out of the increasing commoditisation of products. In the old days the only sources of information available to customers were advertising (and sales) and the advice of a very small peer-group. This meant that marketers had full control over who knew about their products and, more importantly, what they thought about them. This one-way communication to a captive audience allowed the ad-men to conjure up a complex meta-narrative of value and meaning around essentially rather dull products. People weren’t loyal to a product because they loved it, it was because they were brainwashed into believing that their lives would be so much the worse without it.

Skipping forward to the present day: in a world where information is available before we even know we need it, and where brands live and die based on the the ebb and flow of social media sentiment, we no longer need adverts to tell us how to think and feel about products. If I want to know which mobile phone provider is right for me a focused 20 minutes on the Internet reading reviews and asking my friends will form an infinitely more genuine and useful opinion than an ambiguous TV advert of people rolling around in fields of corn.

So marketers no longer control the message, and therefore a product is merely a product. Other than the actual efficacy and quality of the product itself our ability to influence people’s purchase decisions is very quickly vanishing. This is the central business issue behind this new move towards customer engagement online: as someone who works for an agency I get to see this first hand in the challenges presented to us by our clients. At bottom they all say the same thing:

Help!! We need to be more than just a product!

How to be more than just a product

“Stop interrupting what people are interested in and be what people are interested in” – David Ogilvy

If brands want to mean more to their customers than just the products they sell, then it follows that they need to develop a relationship with the customer over and above the experience of that product. This means doing and being something else in such a way that the brand itself becomes a meaningful part of our customers’ everyday lives.

The most famous and most often cited example of how this can work is Nike Plus. Nike sell running shoes, which people probably buy once or twice a year and which they probably never really think about when they’re wearing them to run in. Nike Plus transcends the product by providing a unique service which enhances the entire experience of running. It’s completely relevant to shoes but it isn’t shoes, and it allows Nike as a brand to own running and not just running shoes. This therefore becomes a platform whereby Nike are continually engaged with the running community and through which they have an immensely valid stage on which to communicate to this community about products when it’s appropriate.

Another excellent example is Charmin’s Sit or Squat mobile phone app, which allows a user to locate nearby public toilets when ‘caught short’. It even allows you to upload photographs and rate/review different locations. Obviously this has nothing to do with selling the product directly in the traditional sense (features and benefits), but at the same time it is completely relevant to toilets and therefore toilet paper – it’s a genius piece of advertising which the ‘audience’ keeps with them and uses again and again.

Lastly, there is one more example of this which I am sure will be familiar to readers of this blog, and which may come as a surprise – Google Analytics. At first this might seem odd, but if you think about it GA is actually one of the cleverest engagement platforms in existence: it’s free and ultimately exists to benefit Adwords revenue, but what more effective way could there be to engage, captivate and learn about businesses regarding their use of the web?

How does branded utility differ?

If we want to develop a framework for this stuff which differs to the usual model, we first need to understand how it differs in concept. Here are the key pillars:

Product disassociation – traditional advertising almost always points directly to a product. Even if we can’t measure the sale directly as we can with on-line conversion, it is generally possible to show some correlation between the communication and it’s impact on sales. However with branded utility the picture isn’t always so easy. What actual impact on Charmin’s sales does the Sit or Squat app really have?

Quality not quantity – branded utility is all about relationships, engagement, loyalty and advocacy. Unlike traditional advertising, it doesn’t necessarily matter how many people see the app or service, it only really matters how many people find it useful; feel an affinity to the brand because of it; and talk about it to their friends. Whereas hateful and irritating TV ads can still create positive brand equity through recall, it simply doesn’t work like that here.

Marketing as a service – increasingly people expect something in return for receiving messages, and not only where branded utility is concerned. SAAS tools like Spotify, which allows a free version with ads and a paid version without, has created a popular belief that advertising is a choice – if I have to see it then I want something in return. This is another reason why branded utility is increasingly in demand for brands. Marketing must offer a genuine service in order for any message to be accepted by the consumer.

Advocacy is more than a passing comment – in the digital age it isn’t enough for people simply to tell their friends about your product in the pub. True digital advocates are somewhere slightly closer to a mobilized sales-force; they must be proactive in sharing their experiences and bringing others in to the fold. This can only happen if doing it is easy, meaningful and if they have the right tools to go about it.

Measuring and optimising branded utility

Ultimately, branded utility differs from traditional advertising because it is about creating and driving owned and earned media, which behave incredibly differently to bought media and all the models of traditional advertising. But how then do we measure these strategies?

Aligning corporate goals

Very simply, why the hell are you even doing this really? What exactly do you think it will achieve in terms of profit, retention, sales blah blah blah. If you can’t align the initiative to what it is ultimately supposed to do then your measurements won’t offer any true value in terms of how effective it is for the business. Start from the most obvious and top-line corporate goals and trace these down to your metrics for the initiative. This will become more apparent in the following sections.

Differentiate impact and effectiveness

The natural tendency with branded utility is to measure it like a website. Let’s take the Charmin app as an example. Depending on how it was tagged we could tell how many people download the app; how many times they use it and for how long; to what extent they share, upload and rate content; and also perhaps what they think of it with the aid of some qual research. We can also identify key behavioural goals which in some way mirror a ‘conversion point’, thus giving us a kind of funnel analysis.

However, all this tells us is what the impact of the app is. We can track lots of data on how well it works and how much people love it as a toilet app, but this doesn’t tell us whether it is effective a as a brand communication. It could be number one in the iTunes app store and raved about on national TV, but does this really mean that people will buy more toilet paper? This is the effectiveness of the entire initiative and relies much more heavily on controlled qualitative studies and business modeling.

Build a measurement framework

Create a framework for the metrics you will report on, and ensure it reflects the distinction between impact and effectiveness described above. I have written more extensively on how to build a measurement framework on another blog post, but I have also sketched out a brief hypothetical example based on the Charmin example here. This is, in part, based on Forrester’s general framework for measuring engagement, which fits very nicely with this type of digital activity

Beware correlation and causation

It is amazing how many mistakes are made regarding correlation and causation in web analytics, but it is especially easy with branded utility. For example, take the example of Mazda’s online community forum for MX5 owners, which is certainly a form of branded utlity. They would be interested in to what extent people who use the forum are likely to renew vs. people who don’t. This is easy to do: take a sample of user-drivers vs a sample of non-user drivers and compare the renewal rate. However, this is incorrect. People who use the forum are already more engaged with the brand than people who don’t; this is the very reason that they find it and sign up to it. So the fact that they are more likely to renew could have nothing to do with the forum at all. The real question is: what is the incremental value of the forum or, more clearly: how many people renew because of their experience on the forum. This is not as easy to answer.

Don’t be afraid of intangibles

The answer to the above question cannot necessarily be answered with straight data, and will seem like an intangible business question, but unfortunately it is exactly these kind of questions which we need to answer to address the effectiveness part of the measurement. Rather than go into detail here, I will point you in the direction of a fantastic book: How to Measure Anything by Douglas Hubbard – if you haven’t read this I strongly recommend it. It will enlighten you about just how much of the supposedly unmeasurable can indeed be measured.

In brief summary

The digital relationships we have with our customers are changing. The old advertising and sales modelsdon’t apply in this new world. Embrace the future!

Further reading:

Contagious Magazine Special Report – Branded Utility

Brand Utility – a neat presentation from brandutility.net

Branded Utility Day at Ogilvy – presentation on Youtube

Another branded utility slideshare presentation

Brand Utility, the blog

Recently I read Michael Notte’s excellent post on web analytics and the automotive industry in Europe, and then ended up getting drawn into a conversation about the tool Sophus3, which I used for a long time when I was working with a key automotive manufacturer. I ended up writing a bit of a review for it, so thought I would expand this out a bit on my own blog. I had a lot of problems with this tool and never really felt able to voice my issues about it due to client politics, but now I no longer work with this client I feel it is time my opinions were voiced. Hopefully this will assist with other client’s decision making processes.

Firstly, a disclaimer: these are my opinions alone and have nothing to do with any employer or client I have ever worked with – nor can I say that these are necessarily universal truths that others would have had as well; it’s simply a statement of my experiences working with the tool and supplier. Similarly, the client in question has an excellent relationship with Sophus3 and does get value from the tool; my beef is simply that they could do a hell of lot better and don’t realise it.

Furthermore, it really isn’t all bad – so just to avoid coming across too negative I will start with the positives:

Pros:

• They DO understand the automotive industry better than any other supplier. No other vendor targets vertical market segements in this way. It means that they have a lot of good insights on how the measurement framework for the sector should work and how reporting should be built.
• Their customer service is very good and their staff are very dedicated to client support (at least for their end clients, not so much if you’re an agency though).
• Their back end analytics interface, whilst extremely slow, is actually very flexible and feels more like querying a proper database than an OLAP set-up. Complex cross-tabs and tables can be built in a way which isn’t really possible in other tools.

Cons:

• Speed. The tool is very very slow to use in comparison to other tools, so much so that most of the time you just won’t bother. At times it is more like querying a huge SQL database than a web analytics tool.
• Accuracy of the competitive tool. The eData Exchange tool is supposed to provide benchmark data of all automotive suppliers. However, only key pages of the sites are tagged and non-standard stuff like microsites are ignored. This in my opinion makes the data too inaccurate to use. Some manufacturers rely heavily on campaign microsites and the customer never actually hits the main website. Other manufacturers do everything in the main site.
• Tagging. This does not work like any other web analytics tool. Tags are simply bits of code that are placed on the site – then Sophus3 themselves have to sort out all the meta-data and naming conventions at their end. This is a nightmare and removes vital control over how the tool is set up. It also creates a enormous possibilty for error that just doesn’t happen in other tools.
• ALL configuration has to be done by Sophus3. They have to set up all the dashboards, custom metrics etc etc. This wouldn’t be so bad, but personally I never really felt that this was done right and therefore wasn’t easy to use. No vendor, despite what they say, really has a proper handle on measurement strategy. This is something that needs to be handled by either the client or the agency; and they need the hands-on flexibilty to make the tool bend to this strategy.
• No proper page path analysis; no site overlay; no on-the-fly segmentation; and various other missing fundamental bits of functionality.

In summary, at this point in time I can honestly say that I would not recommend Sophus3 to any company, not even automotive manufacturers. The tool isn’t completely useless, but the point is that even free tools like GA are leaps and bounds ahead, not to mention the giants like Omniture and Webtrends. Sophus3 have a good organisational foundation, they just need to seriously update their tool to bring it in line with other players.

Is web analytics an art or a science? Is it primarily a creative or a methodical endeavour? Is it left-brain or right-brain?

Or, is it both? Does it rely on some kind of balance between the two?

I initially started pondering this after reading Steve Jackson’s Cult of Analytics. The book primarily describes an organisational structure and process that can be used to put web analytics at the heart of an organisation. However, it goes much further than this in that it attempts to create an intensely rigorous system of scorecards that can be used to police the delivery of this framework. I also noticed similar thinking in Akin Arikan’s recent call to create an expert system for web analytics, which argues that web analytics should operate much like the field of medicine or mechanics, with concrete processes followed to the letter.

I can certainly see where they are both coming from, but something about this whole thing just makes me feel really uncomfortable. It is undeniable that web analytics is a form of science and computing (it’s got the word ‘analytics’ in it for a start!), but something inside me constantly cries out “no, there’s more to it than that, web analytics is about creativity and intuition and sales and the passion for opportunity!”. You might need the science and the tech in order to understand what’s happening, but can this ever really tell you what to do next? Doesn’t this require a fundamental and intrinsic understanding of business strategy that can’t be reduced to statistics and data; or made into some kind of rules-based process?

But if this is true, and web analytics is a balance between science and art; analysis and intuition – then today the field seems woefully lacking in the art and intuition. But why is this?

Web analytics has a ‘tech’ heritage, but does this fit?

Web analytics originally ‘emerged’ from the field of IT, and was later integrated with the field of business intelligence. This produced a group of people with a huge amount of technical and analytical knowledge, but their role is to report things to other people. They don’t typically get involved with what that information is used for.

But web analytics IS the use of the information. You can’t divorce the information from what it needs to be used for. Anyone who has witnessed first hand an organisation where reporting is handled by IT and optimisation by marketing will know exactly how disastrous this can be. Separating the two creates an uncrossable chasm in the middle. You either need one person with both skill sets (unfortunately quite rare), or a well managed team with both camps working together.

Web analysis is not the same as traditional data analysis

The word ‘analysis’ in web analysis persuades most companies that they should fill their senior web analyst positions with hardcore data analysts. Some companies even go as far as employing people who have previously been analysing things like meteorological or geological data sets. However, whilst it is important to have at least some hardcore stats knowledge in a team, it isn’t necessary at the senior level.

Web analytics tools are easy to use, at least from a functionality perspective. The vast majority of the stats and number crunching has already been done by the software. Anyone who has crossed over from using something like SAS to something like Site Catalyst will understand this. It isn’t analysis in the same sense; it’s report viewing – so the ‘analysis’ is in fact the interpretation and translation of the reports into action, which starts to get much closer to marketing and general business performance than any kind of traditional analysis.

People only see the means, not the end

If someone came to your house to sell you double-glazing and spent an hour showing you the tools they planned to use, and talked about how the plastic was manufactured, you wouldn’t buy the windows! If, on the other hand, they talked to you about the reduction in noise, the increased warmth and the lower fuel bills that you would get, then you would be more interested, right?

For some reason, we have an endemic problem in this industry whereby people obsess over the analysis and data, and not the benefits of the analysis. This results in a perception of web analytics as boring and difficult to understand. If you are presenting recommendations of analysis to senior management, do you really even need to show the analysis? Web analytics is decision support, not a sleep-aid!

—-

The problem then it seems, is that you often have senior people who are too scientific in their approach and lack the spark of commercial intuition and business acumen that can drive truly actionable analytics. This was less of a problem in the old days of data analysis for direct marketing because there were clearly defined process frameworks through which marketing folk could receive standard reports and make decisions based on those reports – but, and here’s the crucial point, there isn’t really anything standard about digital marketing!

I have always said that, one day fairly soon, the word digital will cease to exist; it’s a term used to describe the fact that some things are analogue and other things are digital, but already there isn’t so much left that can reasonably be called analogue. Therefore, if digital marketing simply becomes marketing, and digital business simply becomes business, and these things are as data-driven as we all hope they will be, then ‘analytics’ is a huge central discipline to an entire business operation – to reduce it as a discipline to a fixed process is like trying to create a fixed process for all aspects of the management of a business. Is this possible and, even if it were, is it not suffocating to the organic growth and development of the business?

On a final note, I was curious to see that Avinash Kaushik’s new book, which I haven’t actually read yet, holds the strap-line “The Art of Online Accountability and Science of Customer Centricity” – I wonder what, if anything, he has in store for us on this question?

People who don’t work with optimisation on a day-to-day basis sometimes have a hard time understading why link building is so important for organic search. It is though – it’s really really important. This is an essential guide to the basics aimed at people who perhaps don’t do it but who can make it happen (because they have the cash or whatever). Listen carefully…

What is a backlink?

Any link to your site from an external domain is classed as a backlink, although in SEO terms this usually refers to editorial links (PR, blogs, articles etc) as opposed to those from search engines, banner adverts or other paid traffic sources. Despite being a source of traffic for the site, these links are vital to organic search rankings.

Search engines index web pages and store them in a database, but the order in which they serve these pages is dependent on the perceived relevance of the page to the search term used; and to the quality of the site itself. The search engines use various pieces of information in order to determine this, but the most important is the volume, quality and nature of backlinks to the site and to the specific domain.

Whilst no-one really knows exactly how the search engine algorithms work, studies are regularly carried out to assess which factors have the greatest impact on rankings. The latest ranking factors survey from SEOmoz identified the following as the top most influential factors:

1. Keyword focused anchor text from external links
2. External link popularity (quantity/quality of external links)
3. Diversity of link sources (links from many unique root domains)
4. Keyword use anywhere in the title tag
5. Trustworthiness of the domain based on link distance from trusted domains

As you can see, 5 of the 4 relate to external links. Almost all aspects of the site itself and its content are considered less important.

Therefore, if the site is to rank highly for high volume keywords, it is imperative that the right kind of backlinks are created and encouraged throughout the internet.

What makes good backlinks?

The following describes the characteristics of the best possible backlink (in no particular order):

• Relevant site – the domain/site on which the link exists is relevant to the destination site. It has appropriate and relevant content and appeals to the same target market.
• Quality and reputation of the site – the domain/site is trustworthy, popular and well respected in its field. For example, non-commercial sites such as Wikipedia will always score higher than content aggregators and directories.
• Relevant content – the editorial content of the page on which the link exists is relevant and similar to the content of the destination page.
• The link is embedded in anchor text which uses a term for which you need to rank highly. For example, “I highly recommend reading my web analytics blog if you get chance”
• The body text around the link and in the article in general uses keywords and phrases for which your site needs to rank highly.
• The destination of the link is a page appropriate to the article, context and the anchor text used.
• The sentimental context of the link is positive. Negative descriptive words around the link lead the engines to believe that the site is unpopular.

Backlink PR : Creating and improving backlinks

Links will appear naturally throughout the internet as the site gains popularity. But these links will not necessarily match the above criteria. There are several options for building new links and persuading content owners to amend or remove existing links:

• Press Releases – PR that will feature on-line can be crafted and distributed according to all the above guidelines. It may even be possible to pre-populate the copy with links and anchor text which can be used by the content/media owners. If this is not possible, guidelines can be supplied along with the press release. These may not be used, but it doesn’t hurt.
• Blogs (internal/affiliated) – blogs created by you or your associates on 3^rd party domains can be fully crafted according to the above guidelines.
• Blogs (created by others) – if backlinks are found within 3^rd party blogs, a polite and descriptive email to the blog owner can often persuade them to make small changes to the link, especially elements such as the anchor text. It is not appropriate to try to influence the editorial nature of the copy.
• Blog comments – other people’s blogs almost always include the ability to make comments. If the blog is relevant to the subject matter of your site, then a comment can be made including a link to the appropriate page. Some blog sites also provide the ability to use html href tags, therefore allowing anchor text. This type of link-building should be approached with caution. Absolute transparency about who is making the comment is essential. Also, the comment should be appropriate, relevant and meaningful. Bloggers are very quick to pick up spamming and underhand link building techniques.
• Articles/editorial created by others – relevant articles and editorial content are a prime target for link building. If the article already contains a link to the site, then (as with blogs) a polite email to the webmaster or editor can often result in subtle changes being made to the link copy. If the site is relevant and does not include a link, a similar email can be crafted to persuade the inclusion of a link. Media owners may get a large volume of these emails, therefore the email must be personal, the content relevant and the link must add something to their article
• Wikipedia etc – has high kudos with the search engines. If articles can be created or contributed to, then links and/or foot references can be placed. As with other links, these must of course be relevant.
• Directories – bespoke directories for your particular industry/topic/hobby often allow short descriptions to be created along with site submissions. However, only the most relevant and well respected directories should be used.

A note on etiquette

Aggressive link-building techniques almost always backfire and result in lower search rankings or bad publicity from bloggers and other online commentators. Link building communications should be approached with the same respectfulness as any other marketing communication. For example, the motivation for commenting on a blog post may be to create a link, but the content and relevance of the comment itself is far more important. If the comment is not relevant and does not reflect your brand then this is far worse publicity than anything that can be incurred by lower search rankings if the link were not there.

I have just been reading Jason Carmel’s post on Optimization and Analytics, which quite rightly argues that performance marketing may be a better and less ambiguous term to describe what we web analysts actually do on a day-to-day basis. I couldn’t agree more, but the issue obviously goes way beyond terminology; and the post actually reminded me of a recent discussion with a client on exactly the same topic, which might be worth sharing.

What is web analytics anyway though?

The real root of this issue is the fact that many companies fail to see what the real goal of web analytics is. They see it as something extra that might be useful, but only when they get around to it and when they don’t have anything more important to do. In the mean-time they carry on as normal; churning out emails, scheduling site updates, adding and removing things to the home page – all based on gut feel or, as Avinash Kaushik likes to put it, the HiPPO (Highest Paid Person’s Opinion). What they don’t understand is that all this stuff they do and web analytics are actually one and the same! Talking about performance marketing not only makes web analytics seem less geeky, it brings to light the fact that our ultimate output IS marketing.

Performance Marketing is a much better clarion call

The specific client I was talking to had exactly this problem; because they didn’t understand web analytics they just couldn’t connect it mentally to their own jobs. In this situation it’s no good talking to people about maturity models or measurement frameworks, or trying to train them on tools, because they still won’t get it. You need to educate them about why they should even listen to you and, more importantly, you need get them excited about why they need to be involved. This is how I went about it on this occasion:

Step 1 – Show them why they need it

The client was under immense pressure to deliver results with a reduced budget, and couldn’t see any way of doing it. They dismissed all notion of ‘web analytics’ because it sounded expensive, time-consuming and like something that wouldn’t deliver immediate results – i.e. they didn’t get it. The first step was to try and show them (without talking about analytics) that they needed to be cleverer about their marketing:

The Need for Data-Driven Marketing

This chart is specific to this client’s market and situation, but what it actually says isn’t so relevant. The key point is that mass marketing is no longer effective, even if you have got the cash for it. Customers are more individual than they use to be, and so you need to get closer to them and have more genuine conversations with them.

Step 2 – Make the connection

Nobody can really argue with what you’ve just said, and then the line of argument progresses in this fashion:

1. The ability to be pro-active and to successfully affect consumer decisions is reliant on the ability to listen, learn and to communicate genuine value through intelligent dialogue.
2. In an online environment listening and learning is achieved through web analysis, measurement and research; understanding how customers currently interact with us and how they want to interact with us.
3. Intelligent dialogue is achieved by optimising the customer experience in order to communicate our message in the most appropriate way, based on what we have learned by listening and understanding.
4. The process is only possible if the data, tools, capabilities and the methods for using them are available and tuned in to what we want to know, and so careful planning is required in order to ensure that insight can become actionable.

This describes holistically the whole process of marketing based on listening, which can also be called performance marketing. Then you can start talking about how to enable them with the ability to actually do it. At the heart of this is the ability to streamline and simplify the flow of data so that decisions can be made:

Enabling Performance Marketing

If data and tools are faster, easier, better and generally more efficient at providing meaningful insight, then your staff are able to spend more time generating action based on that insight and less time trying to work out what it means. This, in turn, means that more attention can be focused on optimisation and improvement initiatives that drive increased performance; and the final result of this is that dialogue and relationship with the customer becomes more tailored, more meaningful and more effective.

Now you can talk about analytics!

Only then can you start to have discussions about maturity models, vendors, internal or external consultants etc etc. It might still be a very long slog, but at least your client (or boss or whoever) can understand what the end game really is.

How do you know whether something is working if you don’t know what ‘working’ means? If your site gets 25,000 visitors a week, is that good or bad? You might reply that 6 months ago you only got 17,000 monthly visitors and therefore it must be fantastic, but how do you know that you haven’t simply moved from really terrible to slightly less terrible?

The way to get out of this is to set good, robust goals for your KPIs – ones which are achievable and based on sound insights and benchmarks. This isn’t just a ‘nice to have’, it is the foundation of a good web analytics measurement strategy or the downfall of a bad one if it’s missing!

Why goals are so important in web analytics

To illustrate the importance: imagine that your website drives offline sales of retail products. Recently you have gradually but consistently improved visitor numbers and congratulate yourself that this will mean more sales in stores. However, unknown to you the market for your particular product has just exploded and there are floods of eager new customers looking for ways to buy it, almost all of whom end up at your competitors. The ultimate effect of this is that you rapidly lose market share in the stores, but of course you never see any of this because you’re too busy congratulating yourself on your 2.3% increase in visitors.

To simplify this a bit – imagine if you owned a shop but never ever left the building or looked outside. You might think you’re doing well because your 10 average customers a day is more like 12 these days, but what if in reality the street outside was teeming with thousands of people and all the other shops had mile-long queues? Would you still think you’re a success?

So, goals – they’re really important! Here’s some tips on how to set them and use them:

How to create goals for your website KPIs

Creating a goal for a KPI is simply about asking first ‘where are we now?’ and then ‘where do we want to be?’ – there are a variety of data and insight sources that you can use to do this. In an ideal world you will use all in combination:

Corporate goals – This is so obvious that it shocks me how often it is ignored by web analysts and even marketing folk. Put very simply: how does your website relate to the overall goals of the business, and what does it need to do to help achieve them? For example, if you have a content site that makes money through ad revenue, how much ad revenue do your shareholders want/need in the next year? You can easily work backwards from this to understand what your goals should be: if you need an extra £85K revenue, how many additional visitors do you need and/or how many extra pages do you want people to view? Simple!

Competitive benchmarking - how do your competitors perform against the same KPIs? Whilst this can be difficult and sometimes impossible to find out, what information can be got is insanely useful. Ideally the data should be linked to corporate goals: if you know that your arch nemesis achieves 5 times the traffic that you do – but you also know that they target a larger and less profitable market segment than your strategy dictates – then you can use your market share estimates to work out what your traffic should be, rather than blindly trying to follow them on a pure number. Good sources of competitive data are commercial providers like Compete, Hitwise and Nielsen; as well as free tools like Google Trends. Failing that you could just ask them. You might be surprised how open they would be if you offered to share your stats in return.

Targeted improvements - Sometimes you just want to push yourself. Even if you have great competitive benchmarking, why stop at matching the competition? Push yourself further! This is about understanding what is achievable and stretching yourself slightly beyond it. Very useful if you have staff that work to bonus targets.

Common sense – if you know your business very well and have a good gut feel about where it can go, you might be brave enough to just use your intuition. Be very careful with this though, an unachievable and unrealistic target is often worse than having no targets – it will lead you on a wild goose chase!

How not to benchmark your performance

I have often seen people set up KPIs and then monitor them using a statistical method called standard deviation. In simple terms this is just a way of making sure that your KPI figures don’t fluctuate dramatically; i.e. they don’t deviate from the average in a given period. If the number improves you can learn from it and roll with it; if it declines you can understand why and rectify it. Whilst this has it’s uses on a day-to-day basis, this is a disastrous way to handle goals. Why would you not want to deviate from the average (mundane)? What else is business except the striving for improvement? No brainer!

Further reading

Avinash Kaushik on benchmarking, goals and more

Coremetrics on industry benchmarking

The Big Book of KPIs by Eric Peterson

The issue of data quality and accuracy in web analytics is something that most web analysts have no option but to learn and internalise very quickly, especially when people start asking why numbers don’t match. However, it is often easy for us to forget that our clients, business users and marketing teams don’t live and breath this data as we do. This post is therefore a reminder of the essential (by no means definitive) facts about why web analytics data can’t necessarily be taken as fact.

Why are the numbers different?

Most people first recognise a problem with web analytics data because they are trying to reconcile absolute numbers between two different systems, for example when comparing visits in Google Analytics with clicks as reported by Atlas (or some other ad tracking tool). The following are the key reasons why these numbers don’t match:

• The terminology used to calculate metrics usually differs slightly. For example, unique visitors must always be unique visitors within [a certain time frame]. Different vendors may use different time frames. Neither is right or wrong; they are just different. This same principle can also apply to lots of other metrics, and sometimes on a much more subtle level.
• Whilst advances are constantly being made, there are currently no agreed standards to these definitions. Analytics vendors often try to name-drop ABCe standards (at least in the UK), but these are generally considered to be outdated and were created for reporting on visits that derive from banner advertising and search; not for web analysis. Here is a good synopsis of the current state of standards.
• Tracking methodologies, such as cookies, packet sniffers and IP addresses all collect data in different ways and all have pros and cons to the way in which they do this. See example below for further info on this one.
• The Internet is composed of a huge array of different technologies, which are all constantly evolving and changing. These technologies play a big part in the accuracy of data collection.
• New browser versions invariably feature new types of technology that allow increasingly savvy web users to hide their on-line behaviour, or even block this behaviour by default.
• Robots and spiders crawl Internet pages in order to e.g. index what is in them for search engines. Data quality in web analytics is a race to keep up with these creatures!

Cookies and Unique Visitors – An Example

The issue of cookies is generally the biggest area of confusion. A client of mine was recently comparing Google Analytics to their incumbent provider, Sophus3. They noticed large differences in unique visitors and wanted to understand why. Whilst this issue is in some respect the product of all the points raised above, the main cause is the type of cookie used:

With Google Analytics, visitors are tracked using 1st party cookies. Estimates suggest that around 1% of users block these cookies and a further 4% block JavaScript. GA is therefore unable to track these users, so real visitors may be under-counted by about 5%.

Sophus3, on the other hand, uses 3rd party cookies. Many browsers block these by default, so estimates suggest that around 65% of traffic is lost due to the combination of this and JavaScript blocking.

Sophus3 then use IP address to track visitors who have blocked cookies. However, most broadband providers use dynamic IP addresses, which change periodically. In some cases, the IP address could change every time the person switches on their computer. Therefore, Sophus3 will register individual people as multiple visitors, and overall numbers will therefore be inflated.

The following chart illustrates this issue in a more visual way (numbers are rough estimates to illustrate a point, and are not meant to be accurate):

How cookies can affect data accuracy in web analytics

Whilst 1st party cookies are generally considered in the industry to be best practice, in truth neither is perfect. For more information, here is a more detailed overview of how cookies affect web analytics data.

Get over it!

The issue of data accuracy can cripple companies and cause vast amounts of wasted time. In truth there is no solution, it is much better to:

• Understand the limitations in as much detail as possible and ensure that all recipients of web reporting and analysis are familiar with what the numbers do and don’t tell them.
• Focus on trends and segments, and not on absolute numbers. This is easy to do when the focus is on analysis and not pure reporting; insight never comes from pure numbers.
• Where numbers such as unique visitors are required for decision making, confidence levels should be used to make reasonable judgements about those numbers.
• If we set a consistent base-line of data at the most accurate that we can get it, then we can use this data to make accurate trend assumptions and draw conclusions about time-series analyses.

The other day I was presenting the findings of some analysis to a client. The focus of this analysis was to discover the behavioural factors affecting checkout completion rates in order to shed some light on why people drop out. For example, within this analysis I was able to say fairly basic things such as:

• Visitors who spend a lot of time on the site before their purchase are less likely to drop out of the checkout process than those who’s session is shorter
• Visitors who land on the homepage are more likely to drop out of the process than those who land on a product page

Now, the client immediately got rather excited about this and began to say, regarding the first point, “Wow, so if we can increase the time on site then we can improve our drop-out rates. Excellent, how do we increase time on site?”. Had I allowed it, this person would no doubt have been rushing back to the marketing team with a new objective to get the time on site up!

So what’s wrong with this? Well, apart from the very numerous issues with dwell-time associated with this specific example, this represents a very common misunderstanding in web analysis. Put very simply:

Your customer didn’t complete their purchase because they were on your site for a long time. They were probably on your site for a long time because they are interested in your products and your site is relevant to them which, in turn, means they are more likely to complete their purchase. Increasing dwell-time per se doesn’t make any sense in this example because it isn’t the cause.

To provide a simpler example of this: you might notice that people who dress smartly often have quite tidy hair as well. Does this mean that dressing smartly causes tidy hair? If I put a suit on, will my hair instantly become much tidier because I’m wearing a suit? No, there is some other factor (the person’s need to look smart) that is driving both of these things.

This leads to all kinds of problems in web analysis, some of which are quite subtle. Furthermore, this problem is inextricably bound up with our obsession with click-stream; if we can’t see beyond the web analysis tool then we have to find our causes within it. The biggest danger is that we stop being able to see our visitors as real people with real needs, and instead just view them as lines of data or collections of behaviours.

A couple more examples of how this can cause problems:

• You notice that direct traffic is of a higher quality than other sources. Does this mean that you should simply get more people to come to you direct? You could do this by displaying your URL as a static image in non-clickable banners, meaning that people have to physically type it into the browser. Again, no. Real direct traffic is direct because of brand familiarity and relevance, which may have nothing to do with advertising. The pure fact that it’s direct is of little relevance. (by the way, be careful – direct traffic isn’t always what it seems)
• You notice a correlation between downloads of your latest white paper and calls to your salesteam. Excellent, the white paper is a succesful acquisition tool and is driving leads! Or is it? Which way round is it really? Are people calling you because they downloaded the white paper, or did they look at your site and dowload the white paper because they called you?

Remember, correlation does not imply causation! You can avoid this by remembering that your customers are real people with needs, desires, habits and lifestyles. They are not lines of data with dwell-times, page counts and completion rates. These things are only behavioural indications of something else more complex that is happening. Look beyond the click-stream and understand how your customers think and feel.

Drowning in Data

The concept of ‘drowning in data’ cannot be understated when it comes to web analytics. Apart from the sheer quantity of information available, the situation is worsened because the tools we use are so terribly fast and effective; it has never been easier to slice, dice and peel (?) your way through such huge mountains of click-stream data. But just because it’s there and easy to access certainly doesn’t mean it’s easy to make sense of. I believe most companies that fail in this arena do so because they simply don’t know what to look at, but rather flail around in the data following endless and infinite pathways that, whilst ‘interesting’, ultimately lead nowhere fast.

This post describes how to get your head above the water and start swimming in a straight line. The answer lies in what I call a ‘Measurement and Optimisation Framework’, which might sound complicated but is, in fact, simply a strategy for: what you should be measuring; how to do it; and what you should do with the information once you get it.

Developing a Measurement & Optimisation Framework

The process of developing a measurement and optimisation framework is simply about answering 3 key questions:

1. Why does my website exist?
2. How can I measure the success of that existence?
3. What can I do to make it more successful if I find it isn’t achieving what I want it to?

For very simple websites (such as a personal blog), you could probably get away with just spending an hour or so thinking about this. For more complex business sites it could take some time! Following is a brief summary of the top-level part of the process through which I would typically take a client in order to get this up and running:

1: Define your site’s KPIs

How can you fix something or make it better if you don’t know what it was meant to do in the first place? Not setting proper objectives and goals is the most serious and fundamental mistake anyone can make, and not just in web analytics!

Most companies fail to do this because they assume that they intrinsically know what their site is for and what needs to be done to improve it. Take the example of a site selling CDs – it’s for selling CDs, right? What could be more complicated than that?

But, think about it for a moment, who is it trying to sell CDs to? Is it trying to achieve the lowest price possible or is it selling at a premium because it caters to a niche? And where does the company want to be in 5 years time, and what does that mean in terms of the brand that needs to be built? Is it important that people tell their friends about it? Oh, and how does the profitability work? Do we need to reduce the cost per sale by increasing the number of repeat buyers and therefore reducing media spend? And what about our other sales channels? Sales on the web cost much less than those that go through the call centre, so do we need to persuade some of those customers to get on-line? etc etc etc…

The point is, what your website means strategically is not necessarily all that easy to articulate. You need to get a really firm grasp on what your companies corporate goals are and work downwards. For big companies this generally means using something like a Balanced Scorecard approach. The system you use isn’t necessarily important, the point is that you align the goals of your site with the strategic goals of the company or, better still, the strategic goals of your customers!

2: Set targets

Once you have defined how to measure success (your KPIs), you then need to determine what that success IS. Again, this goes back to your corporate objectives: if your site is there to generate advertising revenue, how much revenue do your shareholders need next year? And what does that mean in terms of the number of visitors you need and the number of pages they need to look at? This is how you set targets.

Even if you can’t get anyone in your company to give you these targets, you should make them up yourself! It is incredibly difficult to optimise something to work better if you don’t know what ‘better’ means. If you are not able to prioritise which areas of the site need the most attention at any one time, you will drown – and you cannot do this without a sense of the goal for each KPI. Just do it!

At this point in time you might be able to produce something like this:

Typical Web Analytics Measurement Framework

[Please note: I doctored this a lot to protect the identity of a client, so it won't necessarily make complete intuitive sense and is provided more as a visual example]

3: Guide your analysis with a KPI dashboard

Now you know what your KPIs are and how to measure them you can produce a dashboard report showing where they are against where they need to be. This is incredibly important because it is the guiding light of your analytics and tells you exactly what to look for. If, taking the example KPIs in the chart above, I produce my weekly or monthly dashboard only to find that my unique visitors are dangerously below target but that all other KPIs are OK, then all my analysis for that week/month will be guided by a very specific question: what drives unique visitors and how can I improve the volume?

By investigating this you might find, for example, that you have saturated your search market and therefore need to optimise the site for different, non-branded keywords – or that the TV campaign you tested sent lots of high quality traffic and should be repeated. The point is that, without the KPIs, targets and the dashboard, you have nothing to focus you and, more importantly, have no solid way of telling your marketing director why they need to spend more on TV!

4: Optimise, optimise, optimise!

Remember finally that web analysis is not about understanding, its about doing. If you think your job is to report figures to someone else so that they can make sense of them, then you are not an analyst. The output of everything you do is about making changes to your site, media strategy, internal processes or whatever. Analysis and optimisation are essentially the same thing!

Empower yourself!

So what’s the benefit of all this? If it isn’t already obvious think about these two possible scenarios in which you are presenting your ‘analysis’ to your wider team:

• You hold a meeting in which you present 30 charts of data from your analytics tool, moving through geography, time on site, hour of day, browsers, screen resolution and lots of other fascinating charts. At the end everyone agrees that it was really interesting and goes back to their jobs.
• You hold a meeting in which you state that you can make the company an additional £1.5m per year in sales revenue and then proceed to present a road-map for implementing changes to make it happen, with a full ROI justification of likely costs. You get promoted and paid more!

Which one would you prefer?

Analytics direct traffic reports are often viewed as both a highly insightful metric and, in itself, as a particularly valuable stream of visitors. These are people that typed your URL directly into their browser, right? They must have seen your TV ad or just been really engaged with your brand because they remembered your address and didn’t need to use search. Who could ask for better visitors? They are motivated and focused and really intended to come here.

This kind of language continues to dominate all kinds of discussions about web analytics, including blogs, forums, and articles – and even reaches into the field of the experts; just look at the way Google Analytics defines direct traffic. It’s even more worrying when I hear the way my clients talk about it!

The fact is, this definition of direct traffic in web analysis is extremely misleading. It’s true that the direct traffic bucket does include bookmark traffic and typed URLs, but these days (unless you are very strict about your campaign tracking parameters) it can and does include all kinds of other stuff. All it really means is that the session started without a referrer being passed by the user’s browser, and this can happen for lots of reasons as defined in this rather neat list. I have done some tests on some of my clients’ sites and estimate that in some cases up to 90% of ‘direct’ traffic is infact banner ad or PPC traffic!

Here’s an exercise you can perform that will demonstrate exactly how prolific this problem is: as you’re browsing the Internet and following links from one site to the next, you can check the referrer that is passed by typing the following snippet of code into the address bar of your browser:

javascript:alert(document.referrer)

For example, if you visit a site like AOL and click on one of the advertising banners, when you arrive at the destination page replace the URL with the code – a pop-up will appear showing you the referrer (or nothing if one wasn’t passed). Try this with different types of sites, banners and links. Also try it with different browsers. As you will see, quite a lot of the time the referrer is blank. This means that your visit would have been counted as direct traffic in the analytics reports of that site!

So, it’s time stop thinking of direct traffic as people typing in your URL, this isn’t necessarily the case. ‘Other’ or ‘unknown’ would be a more accurate description.

It’s also time to realise the importance of campaign tracking on your inbound links, as Avinash Kaushik points out in his definition of analytics direct traffic. If you always ensure that your links are passing source and campaign info, then you are forcing the referrer field to be populated even if the browser doesn’t pass it. Here’s an easy way to build campaign tracking URLs in Google Analytics.