Letters

Will eradication of Helicobacter pylori improve symptoms of non-ulcer dyspepsia?

Studies included in meta-analysis had heterogenous, not homogenous, results

This meta-analysis is potentially misleading

Meta-analysis included unreliable studies

More studies should have been included in meta-analysis

Authors' reply

Studies included in meta-analysis had heterogenous, not homogenous, results

EDITORJaakkimainen et al's meta-analysis concludes that an improvement in dyspeptic symptoms occurred among patients with non-ulcer dyspepsia in whom Helicobacter pylori was eradicated.¹ Unfortunately, there is a small but crucial problem at the heart of the analysis. The authors report that the summary estimates are statistically homogenous, but this is incorrect. In the observational studies the P value of <0.001 indicates massive heterogeneity between the results of the studies included. In the therapeutic trials the P value of 0.046 also indicates heterogeneity.

Meta-analysts faced with such heterogeneity have three choices: they may ignore the heterogeneity and pool the results with a fixed effects model; they may use a random effects model, which takes the heterogeneity into account; or they may decide not to pool the results. In this instance the authors chose to use a fixed effects model despite the heterogeneity. In consequence the confidence intervals of the pooled estimates are narrow and significance is imputed. A random model might well lead to a summary estimate that does not reach significance. The data extracted from the individual trials have not been published in bmj.com so it is not possible to check this.

The authors have carried out a sensitivity analysis of the therapeutic trials and noted that the single regimen trials measured only short term outcomes and were of lower methodological quality. In contrast, the two trials of triple treatment measured outcomes at 12 months and were of higher quality; these trials showed a much smaller summary estimate of eradication, which barely reaches significance (odds ratio 1.4, 95% confidence interval 1.0 to 2.3). There is clinical as well as statistical heterogeneity between these two groups of studies, so a summary estimate that combines both is of doubtful meaning. Thus in my opinion the best option is not to pool the triple treatment and single treatment trials.

For this reason the conclusion that "eradication of H pylori was associated with an almost twofold increase in dyspeptic symptoms among patients referred to specialist clinics" is misleading because it is based on a summary estimate that makes no clinical sense and is statistically questionable. Until a systematic review is carried out with a wider search strategy and more robust statistical analysis I do not think this meta-analysis should influence guidelines or clinical practice.

Chris Cates, general practitioner. 
Manor View Practice, Bushey, Hertfordshire WD2 2NN chriscates{at}emailmsn.com

Competing interests: None declared.

This meta-analysis is potentially misleading

EDITORJaakkimainen et al carried out a meta-analysis to determine whether eradication of Helicobacter pylori will improve symptoms associated with non-ulcer dyspepsia.¹ Their conclusions differ from those that we have reached in a systematic review addressing the same question; we carried out our review for the United Kingdom health technology assessment programme. We are concerned that the authors' paper may provide a misleading impression of the effect of H pylori eradication treatment on symptoms of non-ulcer dyspepsia. Our review was conducted using a protocol peer reviewed by the Cochrane Collaboration and will be submitted to the Cochrane Library.

We have identified several problems with Jaakkimainen et al's meta-analysis. Firstly, the search strategy is substantially incomplete, with only one electronic database being searched, no text words used, no "grey literature" included (this literature is important for obtaining papers in press in a fast moving field), and non-English language papers excluded.

Secondly, we believe that there is a potential problem with the selection of trials. Single treatment trials will not have eradicated H pylori adequately, and confounding may have arisen where the treatment has benefits on dyspepsia other than H pylori eradication (for example, bismuth and erythromycin).²

Thirdly, we are concerned by the exclusion of the trial by McColl et al³ and wonder whether this was because the trial did not exclude all patients with reflux-like symptoms. A list of excluded studies, and reasons for exclusions, should be available in bmj.com.

Fourthly, as discussed by Cates in his response in bmj.com [published above], the misinterpretation of odds ratios as effect sizes, the handling of tests for heterogeneity, and the application of a fixed effects model are all potential flaws in the analysis. Even on the basis of the studies presented here it is simply not true to say that "eradication of H pylori is associated with an almost twofold improvement in dyspeptic symptoms."

Fifthly, the quantitative estimate that is required by clinicians and researchers planning trials is a measure of the likely absolute benefit (or number needed to treat) of H pylori eradication in non-ulcer dyspepsia. Nowhere in this paper are any figures quoted that could be used to determine what this might be.

The benefit from H pylori eradication may be modest, and any application in clinical practice would require careful consideration and a supportive cost effectiveness analysis in comparison with alternative treatments. We would like to extend Fischer's comments in bmj.com⁴ and emphasise how contact with the relevant Cochrane review group is helpful in ensuring quality in all phases of systematic reviews.

Brendan Delaney, senior lecturer. 
Department of Primary Care and General Practice, University of Birmingham, Birmingham B15 2TT b.c.delaney{at}bham.ac.uk

Paul Moayyedi, MRC training fellow in health services research. 
Shelly Soo, clinical research fellow. 
Centre for Digestive Diseases, University of Leeds, Leeds LS2 9JT

Jon Deeks, director. 
Systematic Review Programme, ICRF/NHS Centre for Statistics in Medicine, Oxford OX3 7LF

David Forman, coordinating editor. 
Cochrane Upper GI and Pancreatic Diseases Review Group, Leeds LS2 9LN

Competing interests: None declared.

Meta-analysis included unreliable studies

EDITORIn their two meta-analyses Jaakkimainen et al concluded, firstly, that "people infected with Helicobacter pylori are about one and a half to twice as likely to have non-ulcer dyspepsia compared to controls" and, secondly, that "eradicating H pylori results in an almost twofold improvement in dyspeptic symptoms."¹ These conclusions are based on flawed analyses.

In addition to the critique given in several responses about the paper in bmj.com we would like to raise the following points. In the first meta-analysis, of 23 epidemiological studies, Jaakkimainen et al examined the association between H pylori infection and non-ulcer dyspepsia. Unfortunately, they failed to eliminate unreliable studies (for example, those in which organic causes of dyspepsia were not excluded by endoscopy (eight studies), those in which the symptom profile of non-ulcer dyspepsia was not defined (five), and those in which the patient and control populations were not matched for age (12). Lack of age matching is particularly important because of the direct relation between advancing age and prevalence of H pylori infection.

View larger version (12K): [in this window] [in a new window]   Odds ratios and summary odds ratio for proportion of patients with complete or almost complete relief of dyspeptic symptoms 6-12 months after treatment in eradication trials

In the second meta-analysis, of five controlled clinical trials, the authors examined the effect of treatment of H pylori infection on dyspeptic symptoms. Here, several criteria should have been applied for the selection of the studies: an appropriate definition of non-ulcer dyspepsia; careful blinding; validated outcome measures of cure of the infection and relief of symptoms; adequate follow up of at least six months; and calculation of a study sample size that is adequate to detect the predefined therapeutic gain. Only two of the five studies fulfil these criteria. ^{2 3} A high quality study, published in 1998, was not included for unknown reasons.⁴ An additional negative study has recently been published.⁵

We have conducted a meta-analysis using the four studies that should be included in an up to date review (figure). We did not find significant symptomatic improvement in the group assigned to receive eradication treatment compared with the control group.^3-5 We are convinced, therefore, that both meta-analyses presented by Jaakkimainen et al are flawed and should be disregarded when doctors are deciding whether to treat H pylori infection in patients with non-ulcer dyspepsia.

D Pantoflickova, postdoctoral fellow, University of Lausanne and Geneva. 
A L Blum, head, division of gastroenterology. 
alblum{at}hola.hospvd.ch Centre Hospitalier Universitaire Vaudois, CH-1011 Lausanne, Switzerland

N J Talley, head, division of gastroenterology. 
Department of Medicine, University of Sydney, Clinical Sciences Building, Nepan Hospital, PO Box 63, Penrith, NSW 2751, Australia

H R Koelz, head. 
Division of Gastroenterology, Triemli Hospital, CH-8063 Zurich, Switzerland

Competing interests: None declared.

More studies should have been included in meta-analysis

EDITORStudies such as Jaakkimainen et al's meta-analysis on Helicobacter pylori infection and non ulcer dyspepsia¹ will help to clarify currently controversial issues. Unfortunately, the authors' conclusions regarding the efficacy of H pylori eradication for non-ulcer dyspepsia may be questionable as only five treatment studies were included-a source of bias recognised by the authors in their discussion.

Several key studies were omitted despite apparently meeting the inclusion criterianamely, being randomised control trials with accepted definitions of dyspepsia and non-ulcer dyspepsia, using accepted and effective eradication regimens, and having symptoms of dyspepsia as a defined outcome measure.

McColl et al randomised 160 patients to omeprazole, amoxycillin, and metronidazole and 158 to placebo. The authors used a validated Glasgow dyspepsia severity score and at one year found a significant benefit in resolution of symptoms for those who had become H pylori negative (21% v 7% for those who remained infected).² Similarly, an earlier study by Gilvarry et al reported a significant reduction in symptoms in patients successfully treated with bismuth, tetracycline, and metronidazole compared with bismuth and placebo (symptom score 14.2 and 9.2 at inclusion and at one year follow up respectively).³

A contradictory, and equally valid, study by Talley et al was also not included for analysis. In that study, 278 subjects were randomised to triple treatment that included a proton pump inhibitor or to placebo; symptom scores at one year did not differ between the groups, but an improvement in symptoms with resolution of chronic gastritis was reported.⁴

Although these studies are not perfect with regard to assessment of compliance, description of the randomisation process, and even presentation (as referred to in the meta-analysis), their design is adequate and their findings significant. Their lack of inclusion in the meta-analysis could affect its findings and conclusions. The jury in the trial of H pylori infection is still out; the evidence put forward should include all relevant information.

Deirdre McNamara, specialist registrar. 
M Buckley, consultant gastroenterologist. 
C O'Morain, consultant gastroenterologist. 
Department of Gastroenterology, Adelaide and Meath Hospitals, Tallaght, Dublin 24 Annemarie.Murphy{at}AMNCH.ie

Competing interests: None declared.

Authors' reply

EDITORAs two letters point out here, heterogeneity exists with the summary estimate of the association studies in our meta-analysis. For this reason we undertook sensitivity analyses to explain the robustness of the estimate and the reasons for statistical heterogeneity. Heterogeneity improved in the subgroup analyses (study design, control of confounders, and study quality), and the summary estimate remained stable. Pantoflickova et al think that studies should be eliminated if dyspepsia is not defined or organic causes not excluded by endoscopy. When we pooled 11 studies meeting these criteria the summary odds ratio was 2.0 (95% confidence interval 1.6 to 2.4).

The summary estimate for the five eradication trials is not generalisable. We base this argument mainly on the lack of robustness of the estimate when we performed the sensitivity analysis. Even though a random effect model may produce a more conservative estimate,¹ the appropriateness of its use has been debated.²

Inclusion of more studies will indeed produce a more stable estimate. What is not obvious in our paper is that we reviewed more studies than are referenced, including McColl et al's trial.³ We calculated a similar estimate (not published) to that of Pantoflickova et al when we compared improvement in symptoms in groups receiving eradication treatment with that in a control group. Our pooled estimate compared groups in which Helicobacter pylori had and had not been eradicated. For this reason, the inclusion criteria for our paper were limited to studies that provided data allowing calculation of an odds ratio in relation to eradication, not just treatment.

The literature search was conducted through March 1999 and did not include studies published after this. The search began with quite broad criteria for including studies, including observational studies and non-randomised trials. We did not include the qualitative review of all studies in our paper, or all the summary estimates. We mentioned in our discussion the point that Delaney et al make about our search strategy. Studies obtained from the "grey literature" rely on the cooperation of editors. This may introduce another selection bias.

Changing clinical practice for a common condition with multiple therapeutic strategies requires a broad look at the literature and a full understanding of the consequences of treatment. For non-ulcer dyspepsia, this includes fully examining acid suppression treatment, motility agents, and lifestyle changes along with eradication of H pylori. We do not know why Delaney et al would prefer presenting number needed to treat as the pooled measure. We hesitate to provide this number, given concerns about pooling studies with variations in the background level of risk related to different entry criteria and clinical settings.⁴

Liisa Jaakkimainen, assistant professor. 
Department of Family and Community Medicine, University of Toronto, Sunnybrook and Women's College Health Sciences Centre, Toronto, Ontario, Canada liisa.jaakkimainen{at}ices.on.ca

Eleanor Boyle, doctorate candidate. 
Inner City Health Research, St Michael's Hospital, Toronto

Fred Tudiver, director. 
Department of Family Medicine, Center for Evidence-Based Practice, State University of New York Health Science Center at Syracuse, Syracuse, NY, USA

1.	Berlin JA, Laird NM, Sacks HS, Chalmers TC. A comparison of statistical methods for combining event rates from clinical trials. Stat Med 1989; 8: 141-151[Medline].
2.	Petitti DB. Meta-analysis, decision analysis and cost-effectiveness analysis. Methods for quantitative synthesis in medicine. Oxford: Oxford University Press, 1994:90-114.
3.	McColl K, Murray L, El-Omar E, Dickson A, El-Nujumi A, Wirz A, et al. Symptomatic benefit from eradicating Helicobacter pylori in patients with nonulcer dyspepsia. N Engl J Med 1998; 339: 1869-1874.
4.	Smeeth L, Haines A, Ebrahim S. Numbers needed to treat derived from meta-analysessometimes informative, usually misleading. BMJ 1999; 318: 1548-1551[Free Full Text].

Competing interests: None declared.