5 Pitfalls to Avoid When Reading Analytics

Recently we've been looking into the super-fun world of UX analytics (yes, it is fun!). Firstly, we looked at what UX analytics actually are and why they matter. We then looked at 5 common myths surrounding data-driven design. So if you're looking to get clued up on what UX analytics is all about, I recommend you read those first. But now it's time we prepared ourselves for collecting our first set of data.

Learning about something and actually being mentally ready to do it are two very different things, and since analytics don't always state the objective truth, we need to have the right tools and mindset if we're to unravel the mysteries of our users. This article covers everything you need to know.

You can check out all the articles in this UX Analytics series here.

1. Don't Invest in a Singular Idea

Given a number of ideas, we'll tend to lean towards the ones that are our ideas. This is known as the IKEA effect.

IKEA is known for selling furniture that you then assemble yourself. Given a ready-made item of furniture, and the exact same item of furniture that you assembled yourself, you'll naturally see more value in the latter because of the time you invested in it. To beat this cognitive bias, you'll need to let go of your ego and accept when the data speaks for itself.

We're also more inclined to favor the ideas that we find visually appealing, simply because we become subconsciously invested in beautiful things. This is bad, because ideas that look amazing on the surface aren't necessarily intuitive, and the sad truth is that fantastic user experience doesn't have to correlate with a stunning visual aesthetic (e.g. Amazon).

A/B testing can determine which idea works better.

2. Prepare to Question Everything

Analytics are plagued with cognitive biases, simply because of who we are as human beings. These are flaws in our cognitive thinking, but flaws that we can nonetheless overcome simply by being consciously aware of them. Let's start with the belief bias, which is a tendency to believe conclusions based on their plausibility. Belief bias is extremely common.

Let's take a ghost button for example (that's a button with a border, but no background or sense of depth/shadow). Numerous A/B tests have indicated their lack of effectiveness, which is suggested to be because ghost buttons go relatively unnoticed.

A badly converting CTA (call to action) on your website could be blamed on a ghost button, where in fact, it could actually be the placement of the button (or something else entirely). It could also be both. Don't rush to make assumptions because your logic seems believable. A/B test with and without the ghost button, and choose the version that converts best. After that, maybe toy with the positioning (for example, in the menu bar vs centralized in the header), because there's rarely ever a singular fix when it comes to conversions (multivariate testing can help you test multiple variations at once).

If you're keen to read more about A/B testing ghost buttons, here's a fantastic example by our very own analytics expert, Luke Hay.

Continue reading %5 Pitfalls to Avoid When Reading Analytics%

by Daniel Schwarz via SitePoint