Don’t tick your own box

How to avoid confirmation bias in your customer research

Customer research is an important tool in your strategic toolbox. It can validate gut feelings, uncover new insights, maintain a dialogue with your customers, and provide a strategic direction for your organisation to follow. This makes it easy for us to roll up our sleeves and exclaim, “Well, what are we waiting for? Let’s do some research!”

Research can and does inform our strategic direction for weeks, months, even years after the fact - giving it a lot of power and clout in informing the direction an organisation goes in. And this means it is vital to be wary of falling into the trap of Ticking Your Own Box.

Ticking Your Own Box is a term that means unwittingly constructing your research to look for only the evidence that proves your point or assumption, and not looking for anything that proves it wrong.

It is depressingly easy to fall into Ticking Your Own Box. It’s tempting to prove ourselves right. But it’s very important that we give those unexpected insights space to come through and prove us wrong, because the consequences of confirmation bias can be far reaching and devastating — especially from a business strategy perspective. It can result in chasing the wrong customers, sending the wrong message, or even building the wrong product!

Sounds like something you want to avoid? Here are the steps that we take when we’re conducting research to make sure we don’t Tick Our Own Box.

1. Know your blind spots

So, we know our company and our customers inside and out. However, deep familiarity with the surface level of our customer’s behaviour and way of being can lead to making assumptions that we don’t even realise we’re making, simply because they’ve been so essential to our business operations for so long. Of course people use our product because it makes them feel good! We might think to ourselves. But actually, why do we believe this? Is this assumption validated by current research? It’s important to ask ourselves, why do we think this particular assumption is true? And where did this assumption come from? What are we really trying to find out?

If you’re really stuck, it might be time to get someone external to your thinking involved. When you explain what you’re trying to achieve, the questions may just write themselves.

2. Ask open-ended questions; avoid leading questions

When thinking of questions to ask people, we have to start somewhere. That somewhere is usually our current assumptions and hypotheses. But our guesses, though educated, are still guesses. They still have to be properly validated - by good, solid, non Box-Ticked research.

To avoid Ticking Our Own Box, **we want to ask open ended questions that open the door to answers we can’t anticipate.

For example, if the assumption we’re starting with is that using product X makes people happy, this is how we would validate that assumption.


Instead of asking
Do you feel happy when you use product X?


How this limits your research
It narrows the spectrum of feelings someone thinks about when they use product X to one single emotion - happy. You will only learn whether or not it makes them happy - leaving you to scratch your head if they say ‘no’.

Try asking
How do you feel when you use product X?

How this improves your research
This question opens the door for other emotions to come through. People have to come up with an answer themselves, and your question doesn’t pigeonhole them before they’ve had a chance to think.


3. Use both quantitative and qualitative research methods

When conducting customer insight research, we combine these two research methods to get the full, uncensored truth of every part of the customer experience. Combining quantitative (customer surveys) and qualitative (one-on-one interviews) research gives us a full spectrum of data to work with to validate our hypotheses.

Quantitative research gives us the numbers; how many people exhibit which kinds of behaviour, how do they interact with the touch points, how much money do they spend, and when do they do things.

Qualitative research gives us deeper and more nuanced insights — things not immediately obvious to the naked eye. The ‘why’.

When you interview people one-on-one, things like the particular words they use, the expression on their face, even the way they are sitting all gives us a far more detailed view of the customer’s personal experience.

Together, these two types of data provide a rich, full view of the customer experience — what they are doing, saying, and feeling, and the reasons why they’re reacting the way they do.

4. Use multiple channels for feedback

The channels you use to get customer responses are almost as important as the way you word your questions; channels affect which chunks of your customer base you will hear from. For example, digital channels restrict feedback to those who have easy, regular access to internet,mail-ins can result in lower response rates based on perceived level of work, and conducting a street survey in Harlem will give you very different responses than a survey in the Upper East Side.

Providing multiple options (including online and offline) for feedback give you far more representative and unbiased results, allowing you to make better quality decisions.

Now you’re armed and ready to validate assumptions the right way! Collect those valuable insights blooming from your non-leading questions. Fling those surveys out on all of the channels to reach every single customer. Let the customers tick their own boxes. Go forth and collect some good, solid, unbiased, non Box-Ticked research!