The highs and lows of policy based evidence

Well said David. Your credibility leaps up hugely in my
estimation.. Shooting the messenger is indeed an ancient crime. The
message then quickly gets lost, whilst we are diverted into the nature of
committee decision-making.

Science depends crucially on the ability to challenge long-accepted
paradigms, by producing and testing alternative hypotheses, and openly and
robustly debating their explanatory power over the 'Flat-Earthist' , or
'Ex-Cathedra' policy-makers in power.

First they duck-stool the heretics, then shoot the cranks, then they
sack the Nutts. After that they'll come for you and me, David. Before
long there will be no independent thinkers left..

And SCIENCE will be the loser.

Competing interests:
None declared

We should indeed be thankful to Professor Nutt for forcing
the government's "policy based evidence" approach into
public discourse, but the problem is broader and more
insidious than the debate on Drugs Policy.

It is fairly clear that Alan Johnson's attitude on drugs is
based on a desire to have policies appealing to a group I
will call the Daily Mail Reading Voter (DMRV). This is a
significant group that New Labour recognized it would have
to win over if it was going to get elected and reelected.
DMRVs are thought to like policies that are "tough" on
crime, drugs and public disorder.

The DMRV is particularly attracted to logically simple
policies. Making drugs illegal and telling people they are
bad should reduce consumption. Restrictive licensing hours
and higher prices should cut down alcohol abuse. Stiffer
sentences should reduce crime. To the DMRV it is obvious
that these policies will work.

The trouble with the real world is that it doesn't always
behave in a way that matches the DMRV's simple logic. While
there are large grey areas, some ideas are empirically
testable and the experiments are often not pleasing to the
DMRV. We know, for example, that cannabis consumption did
not boom when cannabis was downgraded to a class c drug; we
know that alcohol consumption in england declined when
licensing hours were relaxed; both effects the opposite of
the expected effect in DMRV logic.

Johnson's argument that government must send out strong
signals about the evils of ecstasy and cannabis to prevent
their wider use is a clear appeal to the DMRV and clearly
contrary to the evidence.

Luckily for science Nutt was never a government employee and
could talk about the evidence in public without worrying too
much about the affect on the DMRV's attitudes.

But the government does employ a lot of scientists and it
pays a lot of scientists to do studies. Unfortunately these
people are not as free or fearless as Nutt. There are two
major and insidious consequences for scientific honesty.

First the government can commission original research to
back up its policy (there isn't much incentive to actually
test policy as this might lead to results displeasing to the
DMRV).

Second, the scientists themselves may volunteer research
proposals that are likely to attract funding, rather than
those which genuinely test the prevailing policy. (or
perhaps they don't like to do work that just ends up being
suppressed. The US government managed to stop the Word
Health Organization publishing a major study that concluded
that cocaine wasn't that bad. See
http://www.tdpf.org.uk/WHOleaked.pdf That is a lot of futile
work for the contributors.)

These two effects together produce masses of poor quality
science with weak conclusions. An example would be british
policy on alcohol. The Chief Medical Officer (a government
employee) recently tried to outdo the Home Office in his
appeal to the DMRV by suggesting all sorts of new
restrictions and pricing measures on alcohol. Some of the
research he based his position on was research he had
commissioned. But it wasn't exactly starting from an
unbiased position and trying to genuinely test policy
alternatives. In fact most of the literature on the topic
starts with a point of view and seeks to prove it usually
using the weakest methodology possible. So despite the fact
that alcohol consumption has declined since the relaxation
of licensing in england, the CMO can still assert that
studies show restrictions on supply is the most important
constraint on the level of consumption.

Another example would be the extremely dodgy use of road
accident statistics to bolster the Home Office policy about
the use of speed cameras.

The large volume of poorly designed and conducted studies
(in many areas where science could impact policy) which
appear to exist for the sole purpose of reinforcing
preconceived opinions are bad for science. Spin and truth
are uncomfortable bedfellows whether the spin comes from
scientists trying to please their paymasters or governments
trying to suppress unwelcome facts.

Competing interests:
None declared