Rapid Responses to:
|
|
Rapid Responses: Rapid Responses published:
-
![[Read Rapid Response]](/icons/misc/sectionDOWN.gif)
What is meant by 'ruling out?'
- Ed Walker (28 September 1998)
-
Is meta-analysis of observational studies useful?
- Christopher Cates (29 September 1998)
-
Quality of primary studies should affect the inferences made from meta-analyses
- Khlaid S Khan (30 October 1998)
-
Impact of recall bias in meta-analysis of benzodiazepine use in pregnancy
- Samuel Erny, Hilal Maradit (9 January 1999)
|
|
|||
|
Ed Walker, A&E specialist Dewsbury, West Yorks, UK
Send response to journal:
|
The final line of the paper is : 'because some cases of cleft lip can be visualised by fetal ultrasound level 2 ultrasonography should be used to rule out this malformation.' I am unclear what is meant by 'rule out' in this context. Is it intended to mean that a negative finding is a reassurance to parents? Or is it suggested that a positive finding of a cleft lip would be justification for termination of the pregnancy? |
|||
|
|
|||
|
Christopher Cates, General Practitioner Bushey, Hertfordshire
Send response to journal:
|
Sir, In 1994 Shapiro suggested that Meta-analysis to combine the results of observational studies should be abandoned(1). The individual studies are open to particular bias related to their study design, and in this meta-analysis the differences found between the results of the cohort and case-controlled studies suggest that it may be bias that is being pooled rather than true effects. The authors themselves express reservations that most cases reported were derived from only 3 studies, so have the oral cleft patients been counted more than once in the pooling process? Moreover the statistical method used in this paper cannot be relied upon as the pooled results are very sensitive to the adjustment made for trials in which the number of malformations is zero. For example, the largest cohort study (Bergman, 1992) is calculated as showing an Odds ratio of 1.21 based on 0/1354 exposed patients having oral cleft, compared to 62/102,985 not exposed. This result suggests that the zeros have been adjusted to 1 for the calculation (as Odds of zero cannot be turned into an Odds ratio). However if the zero were counted as 0.5 the Odds ratio for the study would fall to 0.61. Even more bizarre is the study on epileptic patients (Robert, 1986) in which 0/4 compared to 1/144 is transformed into an Odds ratio of 10.63 on the basis of the zero transformation. Surely 0/4 is exactly the expected finding if the Odds ratio were 1 and there was no difference between the exposed and non-exposed groups! This paper seems to lend weight to Shapiro’s suggestion that meta- analysis should be confined to Controlled Clinical Trials! 1. Shaprio S. Meta-analysis/Schmeta-analysis. American Journal of Epidemiology 1994;140:771-7 |
|||
|
|
|||
|
Khlaid S Khan
Send response to journal:
|
Dolovich et al (1), in a systematic review of cohort and cose- control studies on the association of malformations with use of benzodiazepines in pregnancy, recommend the use of antenatal diagnostic tools for detecting malformations in clinical practice. For future research they feel the need for further case-control studies. The importance of the quality of primary studies has been disregarded in arriving at these conclusions. In their analysis, Dolovich et al stratify pooling of data according to study quality. Higher quality studies (cohort studies) showed lack of an association while poor quality studies (case-control studies) show an association. This is apparent from the graphic presentation of odds ratios in the paper. To emphasise this point we have performed logistic regression analysis, with fetal malformation (both major malformation and oral cleft) as the binary dependent variable. A logistic model was built to assess the impact of study quality on the strength of the association between benzodiazepines exposure and malformation. The value of the coefficient for the interaction term between "study quality" (cohort vs case-contol) and "exposure" (exposure vs non-exposure) provided an estimate of the bias in poor quality studies (2). Our analysis showed that the case-control studies exaggerated the estimate of odds ratio for the association between malformation and benzodiazepines exposure by 55% (95% confidence intervals 28%-72%, p=0.0006) compared to cohort studies. It has been well established that poorer quality literature tends to exaggerate the estimate of the strength of an assocation and in this circumstance it is more appropriate to base inferences on higher quality evidence (3). Our conclusion on the basis of meta-analysis of higher quality evidence is that there is no association between maternal benzodiazepines exposure and fetal malformation, and further research should aim at generation of high quality evidence in form of cohort studies rather than case-control studies. Khlaid S. Khan Catherine Wykes Harry Gee Lecturer Department of Obstetrics and Gynaecology Birmingham Women's Hospital, Edgbaston, Birmingham B15 References 1 Dolovich LR, Addis A, Vaillancourt JMR, Power JDB, Koren G, Einarson TR. Benzodiasapine use in preganancy and major malformation or oral cleft:meta-analysis of cohort and case-control studies. BMJ 1998;7162:839-843. 2 Khan KS, Daya S, Collins JA, Walter SD. Empirical Evidence of Bias in Infertility Research: Overestimation of Treatment Effect in Crossover Trials using Pregnancy as the Outcome Measure. Fertil Steril, 1996;65(5):939-945. 3 Khan KS, Daya S, Jadad AR. The Importance of Quality of Primary Studies in Producing Unbiased Systematic Reviews. Arch Intern Med, 1996;156(6):661-666. |
|||
|
|
|||
|
Samuel Erny, Epidemiologist / Epidemiologist Global Drug Safety, F.Hoffmann-La Roche Ltd, Basel, Switzerland, Hilal Maradit
Send response to journal:
|
EDITOR - We disagree for several reasons with the summary dismissal of the possibility of recall bias by Dolovich and collaborators in their discussion of the part of their meta-analysis that deals with oral cleft case-control studies (1). Werler and co-workers have shown that recall bias may exist when postpartum interviews are used to assess drug exposure and other factors during pregnancy. Using obstetric records as a measure of accuracy, case reports were found to be more complete than reports by controls for 5 out of 8 factors studied, with up to 7.6-fold more complete reporting by cases (2). Based mainly on a subgroup analysis of the studies that used normal versus deformed babies as controls, Dolovich and collaborators conclude that "the choice of controls did not have a large effect on study outcome". However, no large effect of bias is needed to influence results. Results may be biased even by an intergroup difference that does not reach statistical significance. We have performed a sensitivity analysis to determine the effects of various levels of assumed under- reporting by mothers of control babies on the results of the meta-analysis of the case-control studies with oral cleft as the outcome. The assumed levels of recall bias are all lower than those described by Werler. We adjusted for under-reporting only for the three studies that used normal babies as controls. The adjusted number of exposed controls was obtained by multiplication by the factor 1/(1-r) for each ratio r of under- reporting. Row totals of the cross-tabulation were kept constant by moving the adjusted number of controls from unexposed to exposed. Adjusted combined effect odds ratios (OR) were calculated using STATA (Version 5.0, 1997, Stata Corporation, College Station, Texas, USA), and results are shown in Table 1. Table 1: Impact of recall bias on combined estimates of odds ratios in meta-analysis of case-control studies with normal babies as controls
Degree of under-reporting
(%) Odds ratio (95% CI)
0 1.62 (0.88 to 2.99)
10 1.48 (0.79 to 2.80)
20 1.31 (0.70 to 2.45)
30 1.12 (0.61 to 2.03)
40 0.96 (0.52 to 1.75)
The combined OR for the three case-control studies with normal babies shows a null effect (OR=0.96; 0.52 to 1.75) with 40% under-reporting. The combined OR for all six case-control studies becomes statistically non- significant with 20% underreporting (OR=1.6; 0.99 to 2.67) and shows a null effect (OR=1.03; 0.46 to 2.31) with 70% under-reporting. When a fixed effects model is used, 40% under-reporting suffices to yield null-effect results. This sensitivity analysis shows that the results of the meta- analysis of case-control studies would be changed by a degree of differential recall that is lower than the bias previously described in similar case-control settings. Samuel Erny Epidemiologist, SAMUEL.ERNY@roche.com Hilal Maradit Epidemiologist, HILAL.MARADIT_KREMERS@roche.com Global Drug Safety, F.Hoffmann-La Roche Ltd, Basel, Switzerland References: 1. Dolovich LR, Addis A, Vaillancourt JM, Power JD, Koren G, Einarson TR. Benzodiazepine use in pregnancy and major malformations or oral cleft: meta-analysis of cohort and case-control studies. BMJ 1998;317:839-43 2. Werler MM, Pober BR, Nelson K, Holmes LB. Reporting accuracy among mothers of malformed and nonmalformed infants. Am J Epidemiol 1989;129:415 -421. |
|||