Views & Reviews Review of the Week

Trust me, I’m a scientist

BMJ 2009; 339 doi: https://doi.org/10.1136/bmj.b3658 (Published 09 September 2009) Cite this as: BMJ 2009;339:b3658

Re: Evidence is not bullying

The aberration in my first post may have distracted David Colquhoun
from an important point. Just as he cited a polemic by Goldacre as
evidence of a problem I cited an earlier award winning polemic by Goldacre
where manifestly the evidence he used regarding MMR and autism was at best
neutral rather than negative [1]: there was certainly no basis to ridicule
parents who claimed their children had been vaccine damaged.

Given the authority accorded the Cochrane review of MMR of 2005 [2]
it may be worth going back to look at what it actually said about the six
autism studies it reviewed:-

“The study demonstrates the difficulties of drawing inferences in the
absence of a non-exposed population or a clearly defined causal
hypothesis”. (Re: Taylor 1999)

“The number and possible impact of biases in this study was so high
that interpretation of the results is impossible”. (Re: Fombonne 2001)

“The retrospective person-time cohort study by Makela assessed the
association between exposure to MMR and encephalitis (EN),aseptic
meningitis (AM) and autism (AU) in a cohort of 535,544 Finnish children
(95% of the surveillance cohort); the children were aged one to seven
years at the time of vaccination.The authors compared the incidence of
outcomes in the first three months after vaccination with the incidence in
the following months and years.They concluded that there was no evidence
of association. The study was weakened by the loss of 14% of the original
birth cohort and the effects of the rather long time frame of follow up.
What the impact of either of these factors was in terms of confounder is
open to debate, however the long follow up for autism was due to the lack
of a properly constructed causal hypothesis …” (Re: Makela 2002)

"The follow up of diagnostic records ends one year (31 Dec 1999)
after the last day of admission to the cohort. Because of the length of
time from birth to diagnosis, it becomes increasingly unlikely that those
born later in the cohort could have a diagnosis" i.e. there was extensive
under-counting of autism cases in the MMR group (Re: Madsen 2002)

“The conclusion, however, implied bias in the enrollment of cases
which may not be representative of the rest of the autistic population of
the city of Atlanta, USA where the study was set.” (Re: DeStefano 2004)

“In the GPRD - based studies (Black 2003; Smeeth 2004) the precise
nature of controlled unexposed to MMR and their generalisability was
impossible to determine…The study (Smeeth 2004) appeared carefully
conducted and well reported, however, GPRD-based MMR studies had no
unexposed (to MMR)representative controls. In this study the approximately
4% to 13% seemed to be unexposed controls regarded by the authors as
representative. Such a small number may indicate some bias in the
selection of controls.” (Re: Smeeth 2004)

In fact, Cochrane did not rate any of these studies as of low risk of
bias, and this was without even considering the almost absurd conflicts of
interest that beset some of them. The studies were biased, poorly
designed, had no controls etc. It is even unclear how the met the
inclusion criteria. Other language was apparently misleading. For
instance:-

"Low risk of bias evidence did not support a causal association with
Crohn's disease, ulcerative colitis or autism."

But in the case of the autism studies there was no "low risk of bias
evidence" included in the review and even the GI study Cochrane approved
was funded and conducted by the US Centers for Disease Control [3], and
the data subsequently placed out of future scientific reach or scrutiny
[4].

The abstract conclusion of the review was remarkable:-

"The design and reporting of safety outcomes in MMR vaccine studies,
both pre- and post-marketing, are largely inadequate. The evidence of
adverse events following immunisation with MMR cannot be separated from
its role in preventing the target diseases."

The first sentence follows from what we have already seen, and the
autism studies can scarcely be exempted. The second sentence suggests that
such adverse evidence as there is should be set aside in relation to the
objective of targeting the diseases, though the review also admits it only
has anecdotal evidence that they genuinely work:-

"We were disappointed by our inability to identify effectiveness
studies with population or clinical outcomes. Given the existence of
documented elimination of targeted diseases in large population by means
of mass immunisation campaigns however, we have no reason to doubt the
effectiveness of MMR."

It is not obvious why any this should have been the basis of public
re-assurance: either for those who believed their children vaccine damaged
or those who were being asked to vaccinate their children and
grandchildren on trust. In fact, the authors seem on examination to have
drawn a complete blank - and one which reflected exceedingly ill on the
scientific culture under scrutiny.

I obviously cannot begin to review here the post Cochrane literature
but I put it to David Colquhoun that this was not a compelling body of
evidence with which to browbeat the media and the public into silence
(though it pretty much did). Meanwhile, the cavalier attitude of the
Department of Health and the National Health Service to even severe
vaccine reactions continues unabated [5].

So, I have to ask David Colquhoun - not to mention Ben Goldacre -
exactly what body of evidence they are citing, and why any of us should be
satisfied or impressed?

[1] John Stone, 'Re: Restrictions on hospitality apply to journalists
and doctors', BMJ Rapid Responses 21 September 2007,
http://www.bmj.com/cgi/eletters/335/7618/480#176662

[2] Demicheli V, Jefferson T, Rivetti A and Price D, 'Vaccines for
Measles, Mumps and Rubella in Children', The Cochrane Library, Wiley 2005,
http://mrw.interscience.wiley.com/cochrane/clsysrev/articles/CD004407/fr...

[3] Davis R et al, 'Measles-Mumps-Rubella and Other Measles-
Containing Vaccines Do Not Increase the Risk for Inflammatory Bowel
Disease: A Case-Control Study From the Vaccine Safety Datalink Project',
Arch Pediatr Adolesc Med. 2001;155:354-359, http://archpedi.ama-
assn.org/cgi/content/full/155/3/354

[4] John Stone, 'DESTRUCTION OF DATA=INVALIDATION OF IOM RESEARCH,
BMJ Rapid Responses 23 August 2005,
http://www.bmj.com/cgi/eletters/329/7463/411#71718

[5] John Stone, 'Re: Re: Re: Re: The origins of antivaccine
activism', 10 July 2009,
http://www.bmj.com/cgi/eletters/338/jun22_2/b2435#216539

Competing interests:
Autistic son

Competing interests: No competing interests

18 September 2009
John Stone
Contributing editor: Age of Autism
London N22