News and Comment

What is evidence and what role should it play in policymaking?

Wednesday 11 September 2013

Should public policy be determined by what we believe or what we know? For organisations working in the fields of research, evaluation and engagement the answer to this question is clear and unambiguous: the latter. Evidence-based policies, says this consensus, lead to better outcomes than those formed from beliefs, no matter how strongly they are held.

Not everyone however, shares this view. A recent Institute of Economic Affairs report “Quack Policy” argued that “the government and the public should be far more sceptical about policies which are purported to be ‘evidence-based’”.

Though it is fair to say the paper was not warmly received by the research community, the IEA’s report did raise some important considerations pertaining to how policies are formed and the tensions that sometimes exist between policymakers and researchers.

At its least contentious, the IEA report states that just because something is described as ‘evidence-based’ doesn’t mean it should necessarily be accepted as such. This seems sensible; research isn’t infallible and nor is it always impartial. Yet the view that researchers “exaggerate levels of confidence in their findings if it promotes actions they happen to support” is a subjective one to draw; and the depiction of experts in policy-making as “paternalists” who believe they know what’s best for everyone else is a similarly loaded conclusion.

Indeed, in practice there are many who believe quite the opposite is happening – and that it is policy-makers and politicians who value their opinions over and above relevant research. Earlier this month New Zealand’s top scientific advisor complained that policy was being decided on the basis of “received wisdom” by public servants and politicians who prefer to “to work from their own beliefs or rely on their own experience” rather than consulting evidence.

There are even those who go further and take a more cynical view that policymakers deliberately ignore evidence which conflicts with their own political goals or ideological beliefs. “Policy makers” Professor Gerry Stoker writes “are driven by perverse incentives that lead them to embrace ignorance rather than the insight…and have short-term desires to get re-elected and advance their careers, so evidence matters little save what it delivers on those fronts.”

Add to this the debate about what constitutes ‘evidence’ and whether the public have a place in decision-making, particularly on technical subjects, and it becomes ever more apparent how ambiguous and open to interpretation an evidence-based decision-making process is.

As full supporters of the Open Government agenda, which involves makinginformation and data publically available so that ordinary people are better able to understand – and contribute to – the policymaking process, the OPM Group believes that the views of members of the public have an important role to play in policy decisions. That’s not to say public views should trump academic or professional research but that policy should beinformed by both of these types of evidence, and accompanied by open and honest debate about the scope and influence of them both.

Of course asking officials who may already be reluctant to draw on reputable research when making policy, to give greater consideration to the views of ordinary citizens, is no easy task.

In his excellent blog for The Guardian, Nesta’s Stian Westlake talks about the traps which researchers often fall into when presenting their evidence to policymakers. He describes two groups, one which draws overly simplistic conclusions and presents “clear, actionable advice”, which “may well be wrong”. Or the other, who “never give an opinion on anything, except to say how complicated it is.”

But for organisations committed to improving social outcomes and opening up the exclusive world of policymaking, encouraging decision-makers to draw on the entire spectrum of available evidence and insight when making decisions is crucial: the challenge for us is to make such a process as straightforward and worthwhile as possible.