Twyman’s Law states that ‘anything that is unusual or interesting is probably wrong’; luckily it relates to research rather than conference talks otherwise you may as well stop this reading now. At the Digital Gaggle conference in Bristol last month, Joe Twyman had some really interesting and unusual things to say...

Joe co-founded the market research agency DeltaPoll; you might have seen him on the telly on election night talking about exit polls and forecasting results. His data sets are hugely different to the ones that we work with at Noisy Little Monkey, but his points and principles about research are just as relevant.

They hit a nerve for two reasons:

1. On a personal level we are consumers of research. Social media marketing and the tabloid press are fuelled by surveys and research and our view of the world is shaped by these headlines. We owe it to ourselves to make sure we are informed by good research.
2. On a professional level, we commission and/or use research all the time to make business decisions. We are responsible for the clarity and correctness of this data. 

Perhaps these are old-fashioned, outdated notions, but they go to the very heart of what’s happening in the wider world right now - fake news, Brexit, Facebook data misuse – all underpinned by a manipulation of data.  As individuals that consume, commission, interpret and communicate research we make mini ethical choices all the time; these come together to create the fabric of our society. This is not small stuff.

Why do we conduct research? Fundamentally, to find out what people think. Good research ensures the data accurately represents the population surveyed and the conclusions drawn from that research are both significant and meaningful. Obvious, right? 

Obvious, but easily ignored. When looking at research on a personal or professional basis, Joe urged us to ask some critical questions:

1. Who's answering the survey questions? Twitter is not an accurate sample! 
2. How many people are answering the questions? This is about the margin of error and how much reliance you can place on the results.
3. Who is commissioning the research? The objectivity of the results can be determined by this. Often, research is commissioned by interested parties who want to further a particular agenda, so you have to question the validity of that research.
4. Who is asking the questions? Is it an agency or an organisation you can trust?
5. What questions are they asking? The phrasing of questions can have a significant impact on the results. You should also look at what options are given as answers.
6. How are they asking the questions? By phone, face-to-face, online? There are advantages and disadvantages to each method.
7. When are they asking the questions? What's happening in the world can make a big difference to people’s views, particularly political or emotionally charged issues.
8. How is the data being analysed? You can ‘spin’ research data in so many ways - it's vital to have trust in the methodology of analysis.
9. How is the data being reported? Famously correlation is not the same as causation – in other words, apply some common sense before jumping to conclusions!

What I really loved about Joe’s talk is that in a light hearted, brilliantly amusing way, he was not only giving us information on what to do when commissioning research, but also asking us to think about the research data that surrounds us in the media and ask ourselves if we can trust it.  Does it pass our own intuitive smell test?

And if you are in any doubt about a piece of research you see online, in a newspaper, magazine or on TV, just apply Twyman’s Law: anything that is unusual or interesting is probably wrong.