Pitfalls of pharmacoepidemiology
BMJ 2000; 321 doi: https://doi.org/10.1136/bmj.321.7270.1171 (Published 11 November 2000) Cite this as: BMJ 2000;321:1171All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
Dear Sir,
N.V. Organon, Schering A.G. and Wyeth-Ayerst ("the three companies")
share the concern expressed by Professor Skegg in his editorial in this
journal (BMJ 2000:321: 1171-2) that two studies in the General Practice
Research Database (GPRD) come to opposing conclusions about the safety of
2nd and 3rd generation oral contraceptives (1,2). Of even more concern to
us is the growing tendency to seek to explain the differences in results
of studies by the source of funding, with the implication that studies
supported by the pharmaceutical industry are suspect (3,4).
The support of studies by Professor Farmer, by the three companies,
has been in the form of unrestricted scientific grants. The companies
have had no influence on the design, methodology, conduct, or evaluation
of the results, or on the resulting manuscripts. We are however certain
that these studies stand up to scientific scrutiny and have been conducted
with scientific rigor and the utmost integrity. The main study by Farmer
in the GPRD, which directly compared the thrombosis risk of 2nd and 3rd
generation oral contraceptives (5), was overseen by an eminent
international scientific advisory board who independently reviewed all
aspects of the conduct and analysis of the study.
We agree with Professor Skegg that the conflicting results of studies
carried out on the GPRD by Farmer and Jick require thorough investigation.
It seems impossible that both investigators are right and it seems
unlikely that, as has been suggested, "the true result lies somewhere in
the middle", or that the data can be subject to meta-analysis when there
are such divergent results and arguments over methodologies. We believe
that the only answer to resolve this divergence can be given by a truly
independent audit of design, methodology, conduct and evaluation, together
with an open and honest review of the results from the studies by Farmer
and Jick.
We are aware that our U.K. licence holders asked already in 1996, in
a formal letter to the UK Medicines Control Agency, that the data from the
studies by Jick and WHO be made available (6,7). More recently we have,
in a letter to the Chairman of the CPMP, stated that the studies sponsored
by us are open to independent audit and proposed that the same audit
should be applied to all relevant studies.
We would encourage a comparative audit, as expressed in
correspondence to the MCA and the CPMP, of all relevant studies,
especially those carried out in the GPRD. We are confident of the outcome
in the studies sponsored by us. It should be scientifically compelling for
the investigators of all other studies to open their data for audit as
well and so help to come to a resolution of the matter. Meanwhile it would
seem prudent that contributors to the discussion about the relative safety
of 2nd and 3rd generation oral contraceptives, especially when seeking to
explain the differences between results of various studies, refrain from
unsubstantiated hidden or overt allegation that the company sponsored
studies are suspect.
W de Laat,
Medical Director,
NV Organon, Oss, The Netherlands
R Raff,
Head SBU Fertility Control and Hormone Therapy,
Schering AG,
Berlin, Germany
P. Brock,
Medical Director - Europe,
Wyeth-Ayerst, Radnor P.A., USA.
Correspondence author: e-mail: brockpg@wai.wyeth.com
References
1. Farmer RDT, Williams TJ, Simpson EL, Nightingale AL. Effect of
1995 pill scare on rates of venous thromboembolism among women taking
combined oral contraceptives: analysis of General Practice Research
Database. BMJ 2000; 321: 477-9.
2. Jick H, Kaye JA, Vasilakis-Scaramozza, Jick SS. Risk of venous
thromboembolism among users of third generation oral contraceptives
compared with users of oral contraceptives with levonorgestrel before and
after 1995: cohort and case-control analysis. BMJ 2000; 321: 1190-5.
3. Skegg CG. Third generation oral contraceptives. Editorial BMJ
2000; 321: 190-1.
4. Vandenbroucke JP, Helmerhorst FM, Rosendaal FR.BMJ readers should
know whose words they read. BMJ 2000; 320: 381.
5. Farmer RDT, Lawrenson RA, Todd JC, Williams TJ, MacRae KD, Tyrer F
et al. A comparison of the risks of venous thromboembolic disease in
association with different combined oral contraceptives. Br J Clin
Pharmacol. 2000; 49: 580-90
6. WHO collaborative study of cardiovascular disease and steroid
hormone contraception. Effect of different progestogens in low oestrogen
oral contraceptives on venous thromboembolic disease. Lancet 1995;346:1582
-8
7. Jick H, Jick SS, Gurewich V, Myers MW, Vasilakis C. Risk of
idiopathic cardiovascular death and non-fatal venous thromboembolism in
women using oral contraceptives with differing progestogen components.
Lancet 1995;346:1589-93
Competing interests: No competing interests
It is rare that those who possess the power to make decisions, also
have the guts to admit when they are wrong [1]. The BMJ admission of
errors in their peer review processes is commendable [2].
The peer review process is not infallible. Errors in peer review are
recognized, not only by editors' admissions, but by readers' letters [3],
a form of external, post-publication peer review. However, the acceptance
of critical readers’ letters presents a previously unrecognized conflict
of interest situation. Recently, I have had editors decline two letters
which pointed out major mathematical errors in published articles.
A two-part approach would be useful in dealing with this conflict.
First, the editors who review the letters, must be different and separate
from the editors who review publications. Combining the two
responsibilities is analogous to business managers providing unbiased
comments on the fairness and accuracy of their financial statements.
Second, there are currently no performance measures published to
evaluate how this conflict is being handled. Relevant performance measures
could include:
A. Number of letters about major "errors" that were published
B. Reasons for not publishing any letters about major errors
C. Did the editor who accepted the paper also review the letters
about that paper?
Furthermore, it would be useful to have a survey of international
patterns of practice regarding whether there is separation of editors
reviewing letters and those reviewing papers.
Vincent Richman, MBA PhD
Research Associate
AlgoPlus Consulting Limited,
Halifax, Canada
_________________________________________________________
[1] Brunt S. Straight man in Argos' comedy. The Globe and Mail 2000
Nov 14: Sect. S:1 (col. 1).
[2] Skegg DCG. Pitfalls of pharmacoepidemiology. Oral contraceptive
studies show a need for caution with databases. BMJ. 2000; 321: 1171-2.
[3] Spodick DH. Letter pages are essential for peer review. BMJ.
1996; 312: 1611.
Competing interests: No competing interests
Rare indeed is it for a medical journal to publish, in so bold and
forthright a manner, a mea culpa such as that in the November 11 edition
of the BMJ (Pitfall of pharmacoepidemiology. BMJ 2000;321:1171-1172
Historical, acknowledgement of errors by journals creats the
impression that the correction is noted reluctantly, and in a manner and
style inversely proportionate to the prominance accorded the original
paper. In publishing Jicks' paper, which re-affirms the consensus apropos
the increased risk of VTE for third-generation pill users versus 2nd-
generation users, the BMJ has, in my opinion, established a new benchmark
for candor.
Rather than a 'shock-horror' reaction, it is to be hoped that the BMJ
will receive due plaudits for it brave acknowledge. Such an action
empowers women to make informed decisions regarding their use or otherwise
of the pill.
John Wilks B.Pharm MPS MACPP
Author, "A Consumers's Guide to the Pill and Other Drugs"
3rd ed, National Bookstore, Manilla, Philippines.
Competing interests: No competing interests
Pitfalls of pharmacoepidemiology
Editor - As independent investigators we have been distressed by
recent suggestions (innuendos perhaps) in letters and editorials
published.1 It is suggested that studies such as ours, The Transnational
Case Control Studies of Oral Contraceptives and the Health of Young Women
have been manipulated or "data-dredged" to reflect the perspectives of the
private industrial sponsors who funded the research on drug safety.2 - 5
Such allegations are untrue. We have nothing to hide and we invite
external review of our research records, including questionnaire material,
electronic databases, coding manuals, logbooks and analysis plans leading
to publication of the pertinent articles. It is acceptable that such
assessments be conducted by scientists in universities or governmental
regulatory agencies on any series of working days with reasonable notice..
The presence of a neutral observer (e.g. a journal editor) would be
desirable; the presence of one of our team members would be essential. We
would require that a written plan of inquiry be provided in advance. We
hope that colleagues covering similar concerns about the drug safety of
oral contraceptives (OCs) will also welcome the openness of detailed
external peer-review of methods and source data. We are confident that no
one has engaged in deliberate distortion or has withheld important
information. But almost everyone makes mistakes in good faith. It is best
that any such errors be brought to light if they occurred.
Society expects that pharmaceutical firms conduct rigorous safety studies
of their products as a compelling ethical priority. That is particularly
true when there is an index of suspicion in the post-marketing period.
When companies strive to implement the most objective surveillance
strategies they tend to opt for contractual investigative teams, often
from universities, rather than conducting in-house research. We are
perplexed that the external investigators' work is then regarded
suspiciously whatever precautions will have been taken to be objective and
seen to be objective (e.g., a Scientific Advisory Board and work-in-
progress audits). It deserves emphasis that if scientifically sound safety
studies are to be conducted at all, anywhere, only manufactures have the
resources to fund them Very little of justified safety surveillance would
have been done in recent years without such support for very expensive
studies.
We are perplexed by the animosity which increasingly characterises public
discussion about recent studies on OCs. Especially when changes in
frequency of serious adverse events linked to the pill have not been
detectable in populations of OC users since 1995 regardless of patterns of
use. Perhaps one way to reduce the stridor and increase clarity about all
relevant issues might be to have a public oral debate.
Walter O. Spitzer, M.D., F.R.C.P.C.
Lothar A.J. Heinemann, M.D.,
D.Sc.
1. Skegg DCG. Pitfalls of epidemiology. Oral contraceptive studies
show a need for caution with databases (Editorial). BMJ 2000;321:1771-
1772.
2. Spitzer WO, Lewis MA, Heinemann LAJ, Thorogood M, MacRae KD.
Third generation oral contraceptives and risk of venous thromboembolic
disorders. - An international case-control study. BMJ 1996; 312: 83-88.
3. Lewis MA, Spitzer WO, Heinemann LAJ, MacRae KD, Bruppacher R and
the Transnational Research Group on oral contraceptives and the health of
young women. Oral contraceptives and risk of myocardial infarction: final
results of an international case-control-study. BMJ 1997;315:679-80.
4. Heinemann LAJ, Lewis MA, Spitzer WO, Thorogood M, Guggenmoos-
Holzmann I, Bruppacher R, and the Transnational Group on Oral
Contraceptives and the Health of young Women. Thromboembolic stroke in
young women. A European case-control study on oral contraceptives.
Contraception 1998; 57: 29-37.
5. Lewis MA, MacRae KD, Kühl-Habich D, Bruppacher R, Heinemann LAJ,
Spitzer WO. The differential risk of oral contraceptives: the impact of
full exposure history. Hum Reprod 1999; 14:1493-1499
Competing interests: No competing interests