Trust me, I’m a scientist
BMJ 2009; 339 doi: https://doi.org/10.1136/bmj.b3658 (Published 09 September 2009) Cite this as: BMJ 2009;339:b3658
All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
So John, I think we agree. You have dissected this study and reveal
it has insufficient power, and no confidence intervals. Inevitably
therefore it cannot reject the null hypothesis and ought to be declared
'flawed' for the purposes it was being quoted. It is simply too small a
study, and falls far short of a "robust National Register".
This study was selected and quoted by the NHS Information Service.
No comment was made about study methodology or power ( but you will notice
that the NHS MMR information site makes comments about inadequate
methodolgy against those studies showing possible harms... )
I said in my "Close to agreement":-
"I do agree the need for some very robust data on the incidence of
Autism. A closely-defined National Autism Register being a major
objective.....Is [there] a real increase ( implicating a new environmental
cause), or merely an increased awareness ( detection bias)? A robust
historical incidence register, with cross-referencing capability to
vaccination status ) would go a long way toward exonerating or implicating
particular vaccines, such as MMR. An NHS-constructed postal survey of
parents , and specialist registers would be a good start. The NHS already
holds a robust vaccination register. And a close 10-year tracking follow-
up of the non-randomised UK national study of MMR-accepters versus MMR-
refusers would be well worth publication"
I would like to close this discussion by agreeing with you that we
REALLY DO NEED BETTER SCIENCE if we are to deserve people's trust.. We
need to get on and do that work.
Competing interests:
I am paid as an NHS GP if proportions of children ( regardless of parental consent ) vaccinated meet targets
Competing interests: No competing interests
With regard to the new NHS survey of adult autism mentioned by Sam
Lewis [1] I would think everyone ought to be disturbed at a study which
bases its national estimate for the rate of adult autism on just 19
potential cases detected. The 19 cases were identified in phase 2 of the
study using an ADOS inteview (Summary p. 13), but the results of an SCID
interview (Figure 3c p. 32) which should give a full DSM-IV diagnosis were
not made available. Dr Carol Stott, writing in the Age of Autism, posed
serious question about the methodology used - pointing out that there
does not appear to be a strong base for the use of ADOS in adults and that
the standardised scoring criteria were not used [2]. It is not clear that
we have fully diagnosed cases made by appropriately qualified people. This
is important since accurately profiling and diagnosing the 630 people who
took part in phase 2 - a huge task - would be essential to validity.
The 19 cases are from 7461 household which initially agreed to
cooperate, but the authors aver (p. 16)that they would have found a
further 53 cases among those not selected for phase 2. This may be in part
accounted for by the unsurprising steep drop out rate from the study -
cases which de-selected themselves - but selection for phase 2 was of
course based on psychiatric profiling in phase 1 (3.2.6 Sampling
procedures for phase 2 interviews p. 28), so they are apparently saying
that they got phase 1 wrong. But how do they know, and what basis does it
give for faith in their estimate?
All this is made more problematic by the publicity surrounding the
study. For example, the 1% rate of autism in adults which the authors
settled upon was already anticipated in pre-publicity citing a 1% rate in
children [3] and was linked to claims by NHS Information Centre that the
study cleared MMR of causing autism [4,5], despite the fact that the study
- which did not mention MMR - included young adults born in the years 1986
-1991 who would have had the vaccine anyway.
Other problems include:-
--The study made no attempt to locate lower continuum cases leaving
the existence of such cases in any numbers speculative
--Inclusion in the study was based on the ability of selected members
of the public to answer questions on the telephone in the first phase and
cooperation with a complex psychiatric assessment in the second phase. The
results are self-selecting and unrepresentative
--The 19 cases were unrepresentative of the population at large,
including no one from an ethnic minority, and a low ratio of females to
males (the usual ratio of males to females with ASD being about 4 to 1,
but here 8.5 to 1)
--On table 3.6 (p.39) the 1% result is allocated a 95% confidence
interval, but it is hard to see how you can repeat what has not eachieved
in the first place
--A recent study also suppported by Autism Research Centre of the
University of Cambridge gave a rate of autism in school-children of 1.57%
[6]
[1] Brugha et al, Autism Spectrum Disorders in adults living in
households throughout England, Report from the Adult Psychiatric Morbidity
Survey 2007,
http://www.ic.nhs.uk/webfiles/publications/mental%20health/mental%20heal...
[2] Age of Autism, UK's NHS Report Suggests Autism Increase Due to
Oversight?, 25 September 2009, http://www.ageofautism.com/2009/09/uks-nhs-
report-suggests-autism-increase-due-to-oversight.html#more
[3] 'University of Leicester to lead audit of adults with autism'
Eurekalert 9 May 2008 http://www.eurekalert.org/pub_releases/2008-05/uol-
uol050908.php
[4] Sarah Boseley, 'Autism just as common in adults, so MMR jab is
off the hook', Guardian 22 September 2009,
http://www.guardian.co.uk/society/2009/sep/22/autism-rate-mmr-vaccine
[5] Michelle Roberts 'Autism rates back MMR jab safety' BBC News 22
September 2009, http://news.bbc.co.uk/1/hi/health/8268302.stm
[6] Baron-Cohen S, Scott FJ, Allison C, Williams J, Bolton P,
Matthews FE, Brayne C, 'Prevalence of autism-spectrum conditions: UK
school-based population study', Br J Psychiatry. 2009 Jun;194(6):500-9,
http://www.ncbi.nlm.nih.gov/pubmed/19478287
Competing interests:
Autistic son
Competing interests: No competing interests
In ‘The Dawn of McScience’, the third chapter of his book ‘MMR:
Science and Fiction’, Dr Richard Horton wrote,
“Science has long been ripe for industrial colonization. The
traditional norms of disinterested inquiry and free expression of opinion
have been given up in order to harvest new and much-needed revenues.
Universities have reinvented themselves as corporations. Scientists are
coming to accept, and in many ways enjoy, their enhanced status as
entrepreneurs. But these subtle yet insidious changes to the rules of
engagement between science and commerce are causing incalculable injury to
society, as well as to science.”
Bravo Dr Horton!
And in the same chapter he references the words of journalist George
Monbiot who wrote, [1]
"The scientific establishment is rotten from top to bottom, riddled
with conflicts far graver than Dr Wakefield's. Such is the state of
science today that if, for example, there has been a genuine rise in the
incidence of autism, and if that rise is linked to an environmental
pollutant or the side-effects of a valuable drug, it's hard to see how we
would ever find out."
And in the last paragraph of that article, Monbiot says,
"So, given that undisclosed conflicts of interest in science are
everywhere, why is it only Dr Wakefield whose bloody remains are being
dragged through the streets? The obvious answer is that his alleged
cooption works against the interests of the drugs companies, while almost
everyone else's works in their favour. Why? Because in science, as in all
fields of human endeavour, you get what you pay for."
I am left wondering what GMC registered doctors think they are
getting for their money ... apart, from the vile spectacle of a bloodied
body "with the label "cheat" hung round its neck" ... being dragged
through the streets of London.
[1] George Monbiot. The corporate stooges who nobble serious science.
The MMR scandal shows a business riddled with conflicts of interest. 24
February 2004.
http://www.guardian.co.uk/education/2004/feb/24/highereducation.uk
Competing interests:
None declared
Competing interests: No competing interests
Despite a half promise to the contrary, Professor Colquhoun is still
responding disappointingly … and off topic.
The Professor does not consider the swine-flu a big danger at the
moment. That obviously being a scientifically evidenced based fact, why
has the government ordered 90 million doses of vaccine and agreed to pay
British GPs £5.25 per shot when two shots will almost certainly be
evidenced as necessary? I guess a research professor with ‘mcscientific’
credentials will know the answer to that.
And Professor Colquhoun will no doubt be aware that the GMC did take
(very) seriously the plausibility of Andrew Wakefield’s work: his trial
and that of his colleagues, Professors Walker-Smith and Murch, was finally
concluded on 9 July 2009. The trial started on Monday 16 July 2007 and the
verdicts and sentencings are still to be pronounced. The defendants have
been dragged through two years of judicial hell by the GMC, in league with
the government and pharmaceutical corporations. These hapless medical
scientists have had their professional reputations destroyed, public
discussion about vaccine damage has been stifled and of course there is
now a moratorium on relevant scientific research. All British registered
doctors may care to reflect that this hearing, lasting 144 days, the
longest in GMC history, will have cost them in excess of £5 million. I
guess the true mcscientist considers this a price worth paying …
Competing interests:
None declared
Competing interests: No competing interests
John makes many good points, and obviously does not have a closed
mind on causality, nor the urgent need for good research into autism and
vaccine safety. And I thank David for his helpful and conciliatory points
of agreement.
As the Rapid responders state, a conviction that MMR is safe has not
actually been tested by the appropriate science - a blinded prospective
RCT. Instead we are forced to cast about retrospectively looking for
second-best survey evidence, which for what it is worth shows no good
reason for alarm. It hasn't persuaded doubting parents, and in part we
are all to blame ( whether by accidentally poor communication,
authoritatrian or ridculing approaches, or flat-earthist mentalities ). So
it is highly relevant to public trust in science and doctors, that Flu-
vaccine continues to be robustly tested by blinded RCT, otherwise next
month there will be all-hell to pay !! We used to think HRT was safe,
until the RCT was done. We used to advocate Aspirin widely for primary
prevention, until the rigorous analysis of the RCTs this year !! And
I've lost count of how many drugs have been withdrawn in my lifetime as
evidence of harm emerged !!
I do agree the need for some very robust data on the incidence of
Autism. A closely-defined National Autism Register being a major
objective. In yesterday’s Daily Telegraph ( it was lying in my hotel
lobby ) reported a NHS Information Centre released study on 4000
households, reported to show no difference between autism rates between
adults and children, suggesting therefore that there has been no
discernible increase over a generation. Yet so much anecdote and
experience tells us that autism is on the increase… Is this a real
increase ( implicating a new environmental cause), or merely an increased
awareness ( detection bias) ? A robust historical incidence register,
with cross-referencing capability to vaccination status ) would go a long
way toward exonerating or implicating particular vaccines, such as MMR.
An NHS-constructed postal survey of parents , and specialist registers
would be a good start. The NHS already holds a robust vaccination
register. And a close 10-year tracking follow-up of the non-randomised
UK national study of MMR-accepters versus MMR-refusers would be well worth
publication. Perhaps I missed it ?
Time, gentleman, please .. to pursue the research ?
Competing interests:
I am paid by the NHS to vaccinate people.
Competing interests: No competing interests
It is very sad if David Colquhoun can only defend his beliefs by
attacking people, whether "air-headed celebrities" or "anti-vaxers'. This
is just a strategy for pre-empting discussion of the science, or what
happened to a great number of children under their parents' noses. I have
pointed out the gaping holes in the scientific record - the flawed studies
and the brusquely ignored children - the basis of my progressive
disillusion. And he is quite wrong if he thinks I am looking for someone
to blame - I am looking for the truth, because in the end only that will
do any good.
Competing interests:
Autsitic son
Competing interests: No competing interests
Sam Lewis
I'd go along happily with the "MMR debacle". The word "hoax" perhaps isn't ideal, but of course I was quoting Goldacre's excellent account of the debacle.
I agree too that anything that causes mistrust must be addressed. And of course the innocent mum who has been misled by some air-headed celebrity should no be ridiculed (though perhaps the celebrity should be).
Of course you are right that "Measles, Mumps and Rubella damage too many children, and too many reasonable parents do not trust us." And of course one must "perform the appropriate science". I do think though, despite the implausibility of Wakefield's work even at the time, it was taken seriously and the necessary work was done (short of RCTs which. I'd imagine, would be impossible to arrange).
Most of the people who have commented are against all sorts of vaccination and they endanger their children and other people's children. Noel Bell Thomas tries to divert attention from this by restricting himself to swine flu, but of course that isn't a big danger (at the moment). Measles, mumps, mubella, polio and diphtheria are.
Competing interests:
None declared
Competing interests: No competing interests
Sir,
Science communication, any sort of communication, is bedevilled by our
tendency to choose words with the best ( or less than best )
intentions, forgetting that our meaning may be unclear to others. “
When I use a word,” Humpty Dumpty said in a rather scornful tone, “it
means just what I choose it to mean - neither more nor less.” (1)
Professor Colquhoun complains that the discussion has been “ hijacked by
the antivaccination lobby”.
Who are they ? As the country gears up for a mass vaccination assault
on swine flu, I happened this morning on an update on the preparations.
(2) The three contributors were asked at the end, if they would be having
the vaccine. Two will be having it. The third, a member of the Cochrane
Vaccines Field and the Cochrane Acute Respiratory Infections Group, said
he will not be having the vaccine. He seemed to be suggesting that there
are other options with a stronger evidence base. Does this place him in
the antivaccination lobby ? Who can tell us ? What it does tell us , I
suggest, is that we should all exercise our humility and good humour, as
well as our hobby horses, as we prance and trample, Humpty Dumpty - like,
over all and sundry, in your hospitable columns.
1. Lewis Carroll. Through the Looking - Glass. (1872) ch. 6
2. www.learning.bmj. com
Competing interests:
voluntary vaccination devotee
Competing interests: No competing interests
Sam
As Bernardine Healy, pointed out we now have a very complex vaccine
schedule (not quite as complex here as in the US) so it is going to be
difficult to know what is going on if - as some suspect - vaccines are
influencing the autism rate. Moreover, you would have to split the shots
up - administering the components more or less simultaneously but in
separate shots does not do it (which was problem with the Honda/Rutter
study [1,2]).
One thing that shocks me about the prevailing scientific attitudes to
autism is the complacency. I was told by a friend that locally (Haringey)
we have been getting 20-30 ASD cases coming to adulthood in the past
recent years as against 1 or 2 a decade ago, and 53 forecast for next year
(including my son), while Baron-Cohen et al detected a rate of 157 in
10,000 in a Cambridgshire school population in the early part of the
decade [3]. If I understood what you were saying in your previous post you
believe this is a genetic phenomenon. If so it is an odd one - not only is
there little evidence that autism existed at such rates in the past, if it
was a genetic variation you would not expect it to be very successful or
persistent. Nor have attempts to detect an autism gene had impressive
results. So, I do think we need to know what is causing autism and we need
to find this out urgently, and without professional intimidation.
The final point I would make in relation to adverse vaccine events is
that they and their possible sequelae have to be systematically
investigated and monitored, as Wakefield tried to do. Otherwise we simply
have a hit and run situation, which is wholly unethical and undesirable.
It is not only a question of which tests but of deeply flawed culture
(as in fact you have pointed out).
John
[1] Honda H, Shimizu Y, Rutter M, 'No effect of MMR withdrawal on the
incidence of autism: a total population study', J Child Psychol
Psychiatry. 2005 Jun;46(6):572-9,
http://www3.interscience.wiley.com/journal/118735419/abstract?CRETRY=1&S...
[2] Clifford G. Miller, POWERFUL EVIDENCE MMR CAUSES AUTISM -
RECHALLENGE AT A POPULATION LEVEL, BMJ Rapid Responses 3 April 2005,
http://www.bmj.com/cgi/eletters/330/7491/558-a#102393
[3] Baron-Cohen S, Scott FJ, Allison C, Williams J, Bolton P,
Matthews FE, Brayne C, 'Prevalence of autism-spectrum conditions: UK
school-based population study', Br J Psychiatry. 2009 Jun;194(6):500-9,
http://www.ncbi.nlm.nih.gov/pubmed/19478287
Competing interests:
autistc son
Competing interests: No competing interests
What is trust?
Trust is an invisible question mark hanging over our relationships.
We ask: Can I trust you? Can I trust God? These crucial questions defy
absolute proof, and haunt us with fear and worry. What should we do? We
should also ask: Can you trust me? Can God trust me? Can I trust myself?
These additional questions are important, because relationships are a
projection of self, and trust is a self-portrait.
Competing interests: No competing interests