How to make a compelling submission to NICE: tips for sponsoring organisations
BMJ 2003; 327 doi: https://doi.org/10.1136/bmj.327.7429.1446 (Published 18 December 2003) Cite this as: BMJ 2003;327:1446
All rapid responses
I was very impressed with which knowledgeable vigour Burls et al.
critizised all the shortcomings of clinical research. So, enthusiastic to
finally learn how to conduct a good study, I did a literature search on
PubMed and looked forward to finding the most rigorous controlled trials
done by the authors themselves, let's say something like a clinically
meaningful head-to-head comparison of two active compounds in "real-life"
patients in primary care with five years follow-up on 5 continents with 10
000 participants without support by the pharmaceutical industry and
clinically relevant results over and above trivial statistical
significance in an intent-to-treat analysis. Imagine my pure shock and
horror when I learned that there was no such trial done by the authors who
instead only did systematic reviews so far. I felt disappointed, sad and
lonely in this cold world with all the bad research going on.
The footnote says that the article was written as personal therapy.
As a psychiatrist, I highly encourage this procedure ("write it all out!")
and I hope it worked. However, as sort of a complimentary therapy I would
additionally recommend that the authors should start to conduct a trial as
outlined above on their own. Trying to get funding for such a trial (Isn't
there an institution called NICE?) might be a good start. That might
increase the authors' insight into the problems of clinical research and
might further decrease their anger and desperation leading to even better
mental health. As an even more promising therapeutic approach a close look
at the Smith et al paper "Parachute use to prevent death and major trauma
related to gravitational challenge: systematic review of randomised
controlled trials" in this issue might be very helpful, too.
Competing interests:
None declared
Competing interests: No competing interests
Burls and Sandercock present an amusing view of how to improve the
prospects for a NICE submission by industry to succeed. But they overlook
a number of points. Firstly, NICE should be up to dealing with this so
that, for example, NICE will see data that is commercial in confidence,
even if others will not. Secondly, in pursuit of the pharmaceutical
industry they overlook the fact that some NICE submissions have come from
non-commercial groups and do not necessarily have an obvious commercial
focus. Lastly, they proudly tell us that they are funded, not by the
distorting hand of the private sector but by the public sector. Is this
the public sector that was responsible for the change in star rating of
the South Durham Trust? Was that assessment process wholly in the public
domain or has it only emerged through leaks?
Transparency and review are the only way of avoiding vested interests
of any kind from getting a biased result from a piece of technical
analysis. NICE submissions are more open than what went before, a process
that is difficult to describe as it was far less transparent. The authors
follow the naive line of other correspondents to your organ in pointing
the finger of bias only one way, towards the private sector. All
stakeholders have interests and these will cause pressure for bias,
pressure that must be resisted. In short, anyone who pays the piper
expects to call the tune. That is what external review by a scientific
Pop Idol Jury is all about, whoever is paying.
Peter West
Competing interests:
YHEC Ltd is owned by the University of York and carries out contract research, including NICE submissions, for the pharmaceutical industry and also research on the NHS and the impact of NICE guidance for the Department of Health, sometimes with academic colleagues at York.
Competing interests: No competing interests
Authors' response
Dear Sir
We strongly agree with the points made by Peter West. The NICE
appraisal process should be able to negate the impact of many of the
tricks employed in sponsor submissions; hence the use of independent
review teams such as ours. With reference specifically to Commercial in
Confidence material, in the interests of transparency (and in line with
WHO recommendations) the Institute has made considerable efforts to
discourage sponsors from submitting information which they are unwilling
to make public, for which we applaud them. It should be noted, however,
that NICE still does not have the power to compel sponsors to disclose
relevant information even if only to the appraisal committee.
It is true that many sponsoring organisations are not commercial in
nature. Financial motivations are not the sole cause of cynical or naïve
use of the tactics outlined in our article, as we note right at the
beginning of the piece. However, commercial organisations may perhaps have
access to rather slicker spin doctors than most.
Dr West appears to read our article as a staunch defence of
everything done in the public sector and an attack on all things
commercial; this is not our position, as a brief perusal of the other
article involving AB in the same issue of the BMJ would confirm[1].
Nevertheless, independent review teams may run into problems if they come
to rely on industry funding; bias is not necessarily a conscious process
and personal relationships may influence researchers as much as financial
or academic ones.
Christian Otte appears to believe that experience in conducting RCTs
is a pre-requisite for producing systematic reviews and developing
decision analytic models. He is presumably aware that NICE does not base
its recommendations solely, or indeed directly, upon the results of RCTs;
this is quite clear from our article.
Nevertheless we commend Dr Otte for his attempt to provide an
evidence-based (if ad hominem) critique, although he would be well advised
to brush-up on his literature searching skills before making such bold
statements as “the authors…only did systematic reviews so far”. As a quick
perusal of the BMJ website’s Medline search results would reveal, JS was
previously based at the Medical Research Council's Cancer Trials Office,
one of the world's leading trials units (now the Cancer Division of the
MRC Clinical Trials Unit). She has been involved in the design, conduct
and analysis of some of the largest RCTs ever conducted in gynaecological
cancer, most notably the International Collaborative Ovarian Neoplasm
(ICON) trials[2,3], and continues to support RCT activity within her
current department[4].
Dr Otte states that he is “enthusiastic to finally learn how to
conduct a good study”; as a preliminary measure he is welcome to attend
the clinical trials module of our MSc in Health Technology Assessment,
which is available as a short course for any interested parties[5].
Finally, if he is interested in obtaining funding to conduct an RCT he
might like to know that applying to NICE would be something of a waste of
time but we could put him in touch with a number of bodies which do fund
primary research.
Regards
Josie Sandercock & Amanda Burls
References
1. Pitches D, Burls A, Fry-Smith A. How to make a silk purse from a
sow's ear—a comprehensive review of strategies to optimise data for
corrupt managers and incompetent clinicians. BMJ 2003; 327:1436-9
2. ICON Collaborators. ICON2: a randomised trial of single agent
carboplatin against the 3-drug combination of CAP (cyclophosphamide,
doxorubicin and cisplatin) in women with ovarian cancer. Lancet 1998; 352:
1571-1576
3. The International Collaborative Ovarian Neoplasm (ICON) Group.
Paclitaxel plus carboplatin versus standard chemotherapy with either
single-agent carboplatin or cyclophosphamide, doxorubicin, and cisplatin
in women with ovarian cancer: the ICON3 randomised trial. Lancet 2002;
360: 505-15
4. Jolly K, Lip GY, Sandercock J, Greenfield SM, Raftery JP, Mant J,
Taylor R, Lane D, Lee KW, Stevens AJ.Home-based versus hospital-based
cardiac rehabilitation after myocardial infarction or revascularisation:
design and rationale of the Birmingham Rehabilitation Uptake Maximisation
Study (BRUM): a randomised controlled trial [ISRCTN72884263].
BMC Cardiovasc Disord 2003; 3: 10.
5. Courses in Health Technology Assessment and Health Services
Research. http://www.publichealth.bham.ac.uk/hta/short_courses.htm
(accessed 5 January 2004)
Competing interests:
None declared
Competing interests: No competing interests