Rapid Responses to:
|
|
Rapid Responses: Rapid Responses published:
-
![[Read Rapid Response]](/icons/misc/sectionDOWN.gif)
Failing to include data from unpublished trials limits the findings of systematic reviews
- Lorna M Gibson (12 October 2008)
-
Re: Failing to include data from unpublished trials limits the findings of systematic reviews
- Tania Lourenco, Susan Wong, Adrian Grant, Robert Pickard, Cynthia Fraser, Graeme MacLennan, Luke Vale, James N'Dow and the BPE team (20 October 2008)
|
|
|||
|
Lorna M Gibson, Medical student University of Edinburgh
Send response to journal:
|
Lourenco et al. have addressed an important and relevant clinical question with their recent systematic review of trials of minimally invasive treatments for benign prostatic enlargement(1). However, their findings should be interpreted cautiously. Lourenco et al. performed a comprehensive search strategy to identify both published and unpublished reports of relevant trials which included searching conference proceedings(1). Although they used the most comprehensive report of a trial to extract data, they excluded trials which were only available in abstract form, which leads me to question why the authors searched the conference proceedings in the first place. In total, 22 randomised controlled trials were included in their review, but a further 53 were excluded because they were only reported in abstracts(1). Lourenco et al. recognise that excluding these unpublished trials may have introduced bias into their review(1), but they did not attempt to contact the authors of these numerous unpublished trials for more data. With additional data, some of these trials could have been included in the review, which would have reduced bias and improved the accuracy of the estimate of the effects of minimally invasive treatments for benign prostatic enlargement. The contact details of authors of abstracts can be found by searching Medline and Embase for the authors’ names and retrieving articles which report their contact details. An author’s institutional affiliations can be cross-referenced between the abstract and the articles to ensure that both sources refer to the same person(2). There are well documented cases in the literature which demonstrate that the addition of data from unpublished trials can change the findings of systematic reviews. Whittington et al. found that published trials of paroxetine for depression in 5-18 year olds showed a neutral to weakly positive effect. When data from unpublished trials was also included, the risks of adverse effects and suicidal ideation and suicide attempts outweighed the benefits of paroxtine in this group of patients(3). Further work which includes data from unpublished trials is needed before we can draw reliable conclusions about the effects of minimally invasive treatments for benign prostatic enlargement. References (1) Lourenco T, Pickard R, Vale L, Grant A, Fraser C, MacLennan G et al. Minimally invasive treatments for benign prostatic enlargement: systematic review of randomised controlled trials. BMJ 2008; 337:a1662. (2) Sanossian N, Ohanian AG, Saver JL, Kim LI, Ovbiagele B. Frequency and determinants of nonpublication of research in the stroke literature. Stroke 2006; 37(10):2588-2592. (3) Whittington CJ, Kendall T, Fonagy P, Cottrell D, Cotgrove A, Boddington E. Selective serotonin reuptake inhibitors in childhood depression: systematic review of published versus unpublished data. Lancet 2004; 363(9418):1341-1345. Competing interests: None declared |
|||
|
|
|||
|
Tania Lourenco, Research fellow University of Aberdeen, Health Services Research Unit, Susan Wong, Adrian Grant, Robert Pickard, Cynthia Fraser, Graeme MacLennan, Luke Vale, James N'Dow and the BPE team
Send response to journal:
|
We thank Gibson[Gibson 2008] for raising the important question about whether or not data from abstracts should be included in systematic reviews. Our paper[Lourenco 2007] reported on the clinical effectiveness of minimally invasive treatments for benign prostatic enlargement (BPE), which was part of a large Health Technology Assessment (HTA) report that aimed to evaluate the clinical and cost-effectiveness for all interventional treatments for BPE. It was agreed at the outset of the review that abstracts of conference proceedings of RCTs would be searched for because they can potentially lead to the identification of additional trials for inclusion in the systematic review. It was decided that any data available in abstracts would not be included in the review because of concern about data quality and likely overlap with some published reports. Due to time and resource limitations, no attempt would be made to obtain additional data from the authors of the abstracts. Nevertheless, when we undertook the review, we were concerned by the large number of abstracts identified and the possible publication bias that they might introduce. For this reason, we conducted a subsidiary study to investigate this. [Wong 2007] From the original literature search for the HTA report – it seemed that 67 abstracts (linked to 48 different first authors) had never been published in full as peer-reviewed journal articles. We performed a comprehensive PubMed and Google search for the contact details of these authors and contacted them by post and e-mail, and in some cases, phone calls. These authors were invited to complete a questionnaire aiming to elucidate the reasons for their abstract failing to be published as a full article, and invited them to volunteer any extra study data. A total of 47 questionnaires (linked to 47 study abstracts) were responded to; 32 out of the 47 abstracts had in fact been fully reported and all were already included in the systematic review (though we had not been able to link them from the information available in the abstract), with the exception of those abstracts that were published after our last searching date (n=7). This illustrates a deficiency of current documentation or search strategy in linking conference abstracts with subsequent full text publication that needs to be addressed although it is reassuring that most were included despite the link not being apparent. The principal reason for non-full reporting was ‘being written up’. We also conducted a sensitivity analysis, to investigate the potential impact of the addition of data from the 67 abstracts. A majority of the data (59 abstracts) reported in the abstracts lacked the necessary statistical details (standard deviations, standard errors and confidence intervals) and rendered them unsuitable for the analysis. We found that inclusion of useable abstract data had no significant impact on any of the primary analyses. These findings reassured us: firstly, exclusion of data from abstracts was justified because of duplication and quality concerns; and secondly, inclusion of the few useable data that were available did not materially alter the findings. References: Gibson L.M. Failing to include data from unpublished trials limits the findings of systematic reviews. Rapid response. BMJ.com (12th October 2008) . Lourenco T, Pickard R, Vale L, Grant A, Fraser C, MacLennan G, N’Dow J, Benign Prostatic Enlargement Team. Minimally invasive treatments for benign prostatic enlargement: systematic review of randomised controlled trials BMJ 2008, 337:a1662. Wong S.S.W. The fate of conference abstracts: A side-arm study for the systematic review and economic modelling of clinical effectiveness and cost-utility of surgical treatments for men with benign prostatic enlargement (BPE): A dissertation towards an MSc Health Services and Public Health Research. University of Aberdeen; July 2007 Competing interests: None declared |
|||