Rapid Responses to:
|
|
Rapid Responses: Rapid Responses published:
-
![[Read Rapid Response]](/icons/misc/sectionDOWN.gif)
RE: Systematic review of the accuracy of fetal fibronectin testing
- Richard F. MacLehose, Katherine Hartmann, John Thorp (5 September 2002)
-
Response to BMJ on-line letter
- Honest Honest, Lucas M. Bachmann, Jos Kleijnen, Janesh K. Gupta, Khalid S. Khan (20 November 2002)
|
|
|||
|
Richard F. MacLehose, Research Fellow Cecil G. Sheps Center for Health Services Research, Katherine Hartmann, John Thorp
Send response to journal:
|
RE: Accuracy of cervicovaginal fetal fibronectin test in predicting risk of spontaneous preterm birth: systematic review Dear Editorial Staff, We were pleased to see Honest’s 1 systematic review of studies about use of fetal fibronectin testing to predict preterm birth among women with and without symptoms of preterm labor. However, we are concerned about comparability of the test methods and multiple contributions of the same subjects to the meta-estimates. The analysis addresses “accuracy of studies using 50 ng/ml as the threshold” defining a positive test. Indeed, this is the threshold of current commercially available tests. However, some included studies do not meet this criterion: Rizzo and colleagues work reflects a threshold of 60 ng/ml.2 Further, in studies where multiple tests were performed the authors “considered any positive test as a positive result overall.” Combining sequential tests in this way results in an “aggregate test” that will have different test characteristics than any single use of the test. For instance, if a woman is tested on three occasions there are eight possible permutations of test results, seven of the eight will include one or more positive results and be counted as an aggregate test positive. Combining these “aggregate tests” with single uses of the test is mathematically inaccurate and obscures the relevance of the ordering of the test results, i.e. a positive followed by two negative tests should have a different relationship to risk of prematurity than two negatives followed by a positive, based on temporal relationship alone. The accuracy of a meta-analysis relies on each study population and each study participant from them being counted only once. It seems, based on the weights reported in the meta-analysis, that work by Peaceman and colleagues has contributed three times: line 1 of Fig. 15 represents their analysis of singleton and twin gestation, line two their analysis of only singletons, and line 25 only twin gestations. Failure to recognize that the latter two groups were nested within the former results in all 763 study subjects being included twice in the meta-estimates. Likewise for the publication by Rizzo et al., data from both the 50ng/ml and 60 ng/ml thresholds are included though they represent the same participants. Multiple listings of other publications suggest further duplication. Despite these methodologic limitations, it is unclear if the results are substantively flawed. Regardless, the challenge remains in incorporating fetal fibronectin testing and other diagnostic test modalities into proven clinical decision-making strategies that improve outcomes or sustain comparable outcomes while reducing cost of care. For the latter puposes the negative likelihood ratios and negative predictive values associated with fetal fibronectin testing have been under-reported. Richard MacLehose, Research Fellow
Katherine Hartmann, Assistant Professor
John Thorp, Professor
1. Honest H, Bachmann LM, Gupta JK, Kleijnen J, Khan KS. Accuracy of cervicovaginal fetal fibronectin test in predicting risk of spontaneous preterm birth: systematic review. BMJ 2002; 325:1.10. 2. Rizzo G, Capponi A, Arduini D, Lorido C, Romanini C. The value of fetal fibronectin in cervical and vaginal secretions and of ultrasonographic examination of the uterine cervix in predicting premature delivery for patients with preterm labor and intact membranes. Am J Obstet Gynecol1996; 175:1146.51. 3. Peaceman AM, Andrews WW, Thorp JM, Cliver SP, Lukes A, Iams JD, et al. Fetal fibronectin as a predictor of preterm birth in patients with symptoms: a multicenter trial. Am J Obstet Gynecol 1997;177:13.8. |
|||
|
|
|||
|
Honest Honest, WellBeing Research Fellow AcAcademic Dept., 2nd Floor Birmingham Womens' Hospital, Edgbaston, Birmingham B15 2TG, Lucas M. Bachmann, Jos Kleijnen, Janesh K. Gupta, Khalid S. Khan
Send response to journal:
|
The letter by MacLehose and colleagues1 raises concern about potential for mis-estimation in our meta-analysis2 due to the methods that we have employed. We have re-examined our data and analysis and can confirm that there is no risk of mis-estimation or erroneous inferences. We summarise our response as follows: The first concern relates to combination of contingency tables with two different but closely related thresholds for defining abnormality. The tables with thresholds that were vastly different from the 50 ng/ml level used in most studies were not pooled in our meta-analysis.3;4 There were just two studies, one study in asymptomatic women predicting birth before 34 weeks’ gestation,5 (Figure 1), and one in symptomatic women on predicting spontaneous preterm birth before 37 weeks’ gestation6 (Figure 2) with a threshold of 60 ng/ml. In our view, there was no substantive difference between this threshold and the majority of the rest of the studies. We repeated the meta-analysis (using random effects model) after excluding these two studies and found that the pool estimates were no different. They were as follows: For predicting preterm birth before 34 weeks’ gestation in asymptomatic women, the recalculated summary LR+ was 3.52 [95% confidence interval (CI) of 2.42 to 5.13] and summary LR- was 0.76 [95% CI 0.68 to 0.86]. These were comparable to the summary LR+ of 4.01 [95% CI 2.93 to 5.49] and summary LR- of 0.78 [95% CI 0.78 to 0.84] in our publication (Figure 7 of the unabridged PDF version). For predicting spontaneous preterm birth before 37 weeks’ gestation in symptomatic women, the recalculated summary LR+ was 3.22 [95% CI 2.64 to 3.93] and summary LR- was 0.48 [95% CI 0.41 to 0.57]. These were also comparable to the summary LR+ of 3.27 [95% CI 2.74 to 3.92] and summary LR - of 0.48 [95% CI 0.41 to 0.56] in our publication (Figure 15 of the unabridged PDF version). Thus there is no risk of mis-estimation. The second concern relates to combining sequential testing in an aggregate result. We concur with MacLehose and colleagues that pooling of such results with studies undertaking single testing may be inadequate. However, we took care to examine with meta-regression the impact of pooling serial and single results together. There were 15 studies that reported serial testing among asymptomatic women and 2 among symptomatic. These studies reported in a manner that limited our ability to examine the impact of ordering of test results. Unfortunately the kind of analysis that MacLehose and colleagues request cannot be undertaken with study level data. It requires access to individual patient data. At study level analysis, the best way to explore the impact of serial vs. single testing on the accuracy estimates is to perform a meta-regression analysis, which we reported in our original paper. This showed that estimation of accuracy was not altered. Hence we are confident that pooling in the manner that we did not bias the estimation. The last concern relates to multiple inclusions of the same data in meta-analysis. Theoretically this can both bias the estimates and erroneously improve precision.7 We were aware of this factor and at the time of original data extraction ensured that when multiple contingency tables were constructed from single studies, the same patients were not included in more than one table. The stratification of data into multiple tables wherever possible was an essential part of our approach as it allowed us to undertake advanced meta-regression analysis. We carefully re -examined the specific instance8 highlighted by MacLehose and colleagues and other studies with multiple contingency tables. We did identify some errors in data extraction. These errors were rectified and appear as Figure 3 – 6. Our updated analysis (using random effects model) does not suffer from multiple use of data from the same patients in meta-analysis and its estimates are consistent with those reported in the original paper. Hence there is no risk of mis-estimation. We hope from our response it should be clear that our results are not
substantially flawed. MacLehose and colleagues’ desire for incorporation
of diagnostic information into clinical decision making can be satisfied
by building on the approach we have outlined in the discussion section of
our original paper. It clearly highlights the value of negative test
results so if there is under-reporting about this aspect of testing, it
certainly is not from our work.
Reference List 1. MacLehose RF, Hartmann K, Thorp J. Accuracy of cervicovaginal fetal fibronectin test in predicting risk of spontaneous preterm birth: systematic review [rapid response on-line letter]. BMJ.com 5 September 2002. http://bmj.com/cgi/eletters/325/7359/301#25225. Last accessed 12 November 2002. 2. Honest H, Bachmann LM, Gupta JK, Kleijnen J, Khan KS. Accuracy of cervicovaginal fetal fibronectin test in predicting risk of spontaneous preterm birth: systematic review. BMJ.2002;325:301. 3. Gebhardt S,.Odendaal HJ. Fetal fibronectin in vaginal secretions- -a predictor of preterm delivery? [letter]. S Afr Med J.1995;85:188. 4. Luo L, Lu Q, Wei F. [Fetal fibronectin and preterm birth]. Chung Hua Fu Chan Ko Tsa Chih.1997;32:25-6. 5. Lockwood CJ, Wein R, Lapinski R, Casal D, Berkowitz G, Alvarez M et al. The presence of cervical and vaginal fetal fibronectin predicts preterm delivery in an inner-city obstetric population. Am J Obstet Gynecol.1993; 169:798-804. 6. Rizzo G, Capponi A, Arduini D, Lorido C, Romanini C. The value of fetal fibronectin in cervical and vaginal secretions and of ultrasonographic examination of the uterine cervix in predicting premature delivery for patients with preterm labor and intact membranes. Am J Obstet Gynecol.1996;175:1146-51. 7. Khan KS, Kleijnen J. Data extraction and monitoring progress (Phase 6): Conducting the review (Stage 2). From: NHS Centre for Reviews and Dissemination. Undertaking systematic reviews of research on effectiveness. 2nd edition. CRD Report No. 4. University of York: York, 2001. 8. Peaceman AM, Andrews WW, Thorp JM, Cliver SP, Lukes A, Iams JD et al. Fetal fibronectin as a predictor of preterm birth in patients with symptoms: A multicenter trial. Am J Obstet Gynecol.1997;177:13-8. Competing interests: None declared |
|||
