When meta-analysis and common sense collide
In their reply to my letter , Howden and colleagues  spend considerable time indulging in a straw man fallacy. At no time did I say or imply that I interpreted their data as proving that PPIs increase the mortality in patients with bleeding peptic ulcers. However, I still believe that their comment that “… there may have been too few patients in our pooled analysis of mortality to enable us to detect a difference”  is curious. At best, it creates unnecessary doubt and, at worst, it raises the spectre of bias. The authors should ask themselves why, after analysing the results of 18 studies with a total of 2,774 patients and producing an odds ratio of 1.11 (CIs 0.79-1.57), they chose to make a comment that suggested that proton pump inhibitors may, after all, reduce mortality.
Howden et al. suggest that I have misinterpreted the statement in their paper that “It is, however, a remarkably consistent observation that such treatment reduces rates of rebleeding and, in general, the need for surgical intervention.” They say that the consistency refers to the subgroup analysis yet, in the context of the statement, there is nothing to indicate that this is the case. It is the responsibility of the authors to ensure that such misinterpretations are avoided and this could have been achieved simply by qualifying the phrase “consistent observation”. They should re-read the final paragraph in the published paper and ask themselves whether or not it might have been made more explicit.
Howden et al. judge the study by Hasselgren et al.  to be an outlier and, by referring to the low mortality rate in the placebo group, implicitly raise questions about the validity of its results. Although the selection of the study by Hasselgren et al. for particular scrutiny may be justified on statistical grounds, it may also be viewed as a one-sided reaction to unwelcome results.
Throughout their reply, Howden et al. repeatedly imply that I did not read the complete on-line version of their paper. This is simply untrue. Indeed, it was the contents of Table I – only available on-line – that drew my attention to the possible relevance of the different location of studies included in their meta-analysis.
Howden et al. do not agree with my exclusion of comparative studies between PPIs and H2-receptor antagonists. I focussed on the results the seven placebo-controlled randomised trials because the matter in question is not whether PPIs are superior to H2-receptor antagonists but rather whether PPIs are of benefit to patients with acute peptic ulcer haemorrhage in terms of reducing mortality, re-bleeding or surgery. As it happens, these seven studies also turn out to be the largest in the meta- analysis. Concentrating on the placebo-controlled studies, the striking finding was the difference in the efficacy of PPIs in terms of re-bleeding between the three studies from Europe and the four from Asia. None of the European studies reported any reduction in re-bleeding with PPIs whereas three of the four Asian trials showed benefit. Regardless of the reasons for this discrepancy – and, as outlined in my letter , there may be other factors apart from the differences in the populations studied – this finding is crucial to any recommendations regarding the management of patients. Clearly, if we are concerned about the external validity of the data, then we should base recommendations on those studies which recruited patients who most closely reflect the target population. Thus, in the United Kingdom, we should give preference to data from studies performed here and in the rest of Europe. Accordingly, we should conclude that the relevant data do not show that PPIs reduce mortality, re-bleeding or surgery in patients with peptic ulcer haemorrhage.
No doubt, this interpretation of the data will be anathema to the advocates of meta-analysis. But, before they rush to criticise, they may, perhaps, stop and wonder why it is that this methodology may so readily lead to a clash with common sense.
 Penston J. An alternative interpretation of the data. BMJ. Rapid responses, 14th march 2005.
 Howden CW, Leontiadis GI, Sharma VK. An alternative interpretation of the data. BMJ. Rapid responses, 22nd March 2005.
 Leontiadis GI, Sharma VK, Howden CW. Systematic review and meta- analysis of proton pump inhibitor therapy in peptic ulcer bleeding. BMJ 2005;330;568-70.
 Hasselgren G, Lind T, Lundell L, Aadland E, Efskind P, et al. Continuous intravenous infusion of omeprazole in elderly patients with peptic ulcer bleeding. Results of a placebo-controlled multicentre study.Scand J Gastroenterol 1997;32;328-33.
Competing interests: None declared
Competing interests: No competing interests