Rapid Responses to:
|
|
Rapid Responses: Rapid Responses published:
-
![[Read Rapid Response]](/icons/misc/sectionDOWN.gif)
All or none studies
- Olivier Steichen (16 February 2007)
-
The irresistible urge to sprinkle statistics
- James Penston (19 February 2007)
-
Beware of the Texas sharp shooter in calculating rate ratios of progression
- Anna C Goodman (20 February 2007)
-
Evidence from case series
- Mounia N Hocine, Heather J. Whitaker, Paddy Farrington (20 February 2007)
-
Mother's Kiss
- Daniel Polowetzky (24 February 2007)
-
Response from the Authors
- Paul P Glasziou (5 March 2007)
-
75 years ago: 24. December 1932. Streptococci kill, antibiotics help to survive!
- Friedrich Flachsbart (26 December 2007)
|
|
|||
|
Olivier Steichen, Associate Specialist Internal Medicine, Tenon Hospital, F-75020 Paris, France
Send response to journal:
|
Aronson and Hauben recently described circumstances where anecdotal reports can prove definitive adverse reactions without further formal verification [1]. Glasziou and colleagues now outline circumstances where even methodologically weak study designs can produce strong evidence supporting a clinical intervention. They refine and extend the concept of "all or none studies", introduced in the reference book on evidence-based medicine [2]. All or none studies are not explicitly mentioned in this paper but most historical references to treatments with dramatic effects relate to conditions with rapidly fatal outcome, like diabetic ketoacidosis. All patients died before the treatment, insulin, became available and some then survived on it, proving its effectiveness. The authors remind us that quality of evidence is not an absolute notion solely based on methodological criteria. Its assessment must be adjusted with clinical knowledge of the condition under study. 1. Aronson JK, Hauben M. Anecdotes that provide definitive evidence. BMJ 2006;333:1267-1269. 2. Straus SE, Richardson WS, Glasziou P. Evidence-Based Medicine: How To Practice And Teach EBM. 3rd ed. Churchill Livingstone, 2005 Competing interests: None declared |
|||
|
|
|||
|
James Penston, Consultant Physician/Gastroenterologist Scunthorpe General Hospital, North Lincolnshire DN15 7BH
Send response to journal:
|
Sir, Reading the article by Glasziou et al.[1], we might be forgiven for believing that they had discovered some hitherto unknown method of causal inference. Instead, of course, they have merely stumbled across the way in which causes have been identified in everyday life and science throughout history. [2] The “Mother’s kiss” technique for removing a bead lodged in a nostril is known to be an effective treatment not only because it has been shown to work in case reports but also because it is grounded in elementary principles of physics familiar to every child who has played with a pea- shooter. It does not need statistical analysis. Yet, the authors - unable to free themselves of the urge to season the data with a sprinkle of relative risks or P-values - neglect the fact that the many examples they provide of treatments with clearly observable effects are widely accepted without the need for statistical tricks. The obsession with both randomised controlled trials and the statistical approach to causation has clouded the thinking of a generation or more of medical researchers. So much so, that the common sense notion of causation has been relegated to little more than an afterthought. And this accounts for the dismissive approach to any data not derived from randomised trials. Perhaps, after their damascene conversion, Glasziou et al. will campaign for a change in the hierarchy of evidence in favour of data from non-randomised sources. References [1] Glasziou P, Chalmers I, Rawlins M, McCulloch P. When are randomised trials unnecessary? Picking signal from noise. BMJ 2007;334;349 -351. [2] Penston J. Fiction and fantasy in medical research: the large- scale randomised trial. The London Press. London, 2003. Competing interests: None declared |
|||
|
|
|||
|
Anna C Goodman, PhD student LSHTM, Keppel Street, London, WC1E 7HT
Send response to journal:
|
Glasziou et al’s method of calculating rate ratios of progression (stable unchanging condition before vs. change shortly after the intervention) is appealing, but in applying it we need to be wary of a ‘Texas sharp shooter’ effect. This effect is usually associated in epidemiology with the problem of interpreting apparent ‘clusters’ of disease in space, where the geographical unit of analysis - a town, a borough, a few streets, the top half of one street etc - may have been chosen post hoc in such a way as to maximise the apparent density of cases (the sharp shooter metaphor comes from a joke about a Texan firing bullets into the wall of a barn and then drawing the targets around the bullet holes as a demonstration of his shooting prowess). There is the potential for an analogous problem when trying to calculate rate ratios in the manner described in this article, although here the sharp shooting is in time not in space. To take the authors’ example of the mother’s kiss, the time period used is 10 seconds which gives a rate ratio of progression of 1440. Perhaps, however, the bead dislodged after only 8 seconds – if a somewhat sharper shooter had used 8 seconds as a time frame, this would have given a rate ratio of 1440/0.8 = 1800. Alternatively, if the bead had taken 15 seconds to dislodge, the GP, nurse and mother might still reasonably have felt that they should take the credit for the this happy outcome. The authors’ might correspondingly have stretched their time interval to 15 seconds (associated rate ratio 960) or included two 10 second intervals, just as they included 3 (rather than one) month in their portwine stain example (associated rate ratio 720). The point is that one needs to make an a priori decision about the post-intervention time frame you will use – presumably based on the maximum length of time after the event during which, if improvement occurs, you are prepared to attribute it to your intervention. Competing interests: None declared |
|||
|
|
|||
|
Mounia N Hocine, research fellow The Open University, Milton Keynes MK76AA, UK, Heather J. Whitaker, Paddy Farrington
Send response to journal:
|
We very much agree with Glaziou et al [1] that in some situations evidence from case series or non-randomized cohorts render randomised trials unnecessary. The authors discuss a rate ratio obtained by comparing the rate of progression between treated and untreated periods for a single case. This is similar to the relative incidence estimated from case series using the self-controlled case series approach [2]. The authors recommend their approach when effects are large, but acknowledge that their approach may yield biased estimates in the presence of time trends. Indeed such methods have been used inappropriately, as in investigating a possible association between DTP immunisation and sudden infant death syndrome (SIDS) (see Example 1 in [3]). The self-controlled case series method allows for full adjustment for underlying trends, most flexibly using a semi-parametric approach [4]. Furthermore, it controls implicitly for all fixed confounders and random individual effects, whether measured or not. Thus, in this respect, the method achieves a degree of confounder control usually only available in randomized trials. Under suitable conditions, the method can be used to evaluate treatment effects of small and moderate size for both acute and progressive conditions. It has been applied in a wide range of situations in pharmaco-epidemiology, including MMR vaccine and autism and antidepressants and hip fracture (see [5] for details), and in other contexts such as bacterial resistance to antibiotics [6]. References 1. Glasziou P., Chalmers I., Rawlins M. and McCulloch P. When are randomized trials necessary? Picking signal from noise. British Medical Journal 334: 349-351. 2. Farrington, C.P., 1995. Relative incidence estimation from case series for vaccine safety evaluation. Biometrics, 51 228-235. 3. Farrington, C.P., 2004. Control without separate controls: Evaluation of vaccine safety using case-only methods. Vaccine, 22 2064- 2070. 4. Farrington, C.P. and Whitaker, H.J., 2006. Semiparametric analysis of case series data (with Discussion). Journal of Royal Statistical Society, Series C, 55 553-594. 5. Whitaker, H.J., Farrington, C.P., Spiessens, B. and Musonda, P., 2006. Tutorial in Biostatistics: The self-controlled case series method. Statistics in Medicine, 25 1768-1797. 6. Hocine M, Guillemot D, Tubert-Bitter P, Moreau T. 2005. Testing independence between two Poisson-generated multinomial variables in case- series and cohort studies. Statistics in Medicine, 24 4035-4044. Competing interests: None declared |
|||
|
|
|||
|
Daniel Polowetzky, RN Mount Sinai School of Medicine, One Gustave L. Levy Place, New York, NY 10029 USA
Send response to journal:
|
Although efficacy of the "Mother's Kiss" technique for removal of a foreign body lodged in a nostril may be demonstrated sufficiently by a report of a case series in which it was successful in 15 out of 19 cases,and thereby, precludes futher randomized trials for demonstration of treatment effect, no mention of safety is made. The authors in fact go so far as to recommend the technique: A search yields only one report of a case series, in which the mother's kiss was successful in 15 out of 19 children. We think this is sufficient evidence to recommend use in practice without randomised trials. On first blush, the technique does not seem obviously safe, as compared to visualizing the object and removing it with forceps. It sounds like performing the pulmonary portion of CPR on a living, breathing person. Is it not possible that the increased airway pressure may not be such a good idea? Competing interests: None declared |
|||
|
|
|||
|
Paul P Glasziou, Director Centre for Evidence-Based Medicine, Oxford OX3 7LF
Send response to journal:
|
We are glad to see the general agreement with the principles and examples we set out. However we agree that further work and refinement are needed if we are to avoid erroneous conclusions about non-randomised evidence. Further careful examination of examples and the development of methods will help us tease out which methods apply when. While some may find the examples simple and plain “common-sense”, they are also important bridges to more complex areas of evidence. We need to find the harmony between uncommon sense and common sense (the sense that tells us the world is flat, but also keeps us alive). In this regard, the work of Aronson and the development of case- crossover designs (mentioned by Hocine) are both important and clearly closely related to the methods we set out. However as Goodman suggests, though a case or case series or other non-randomised evidence may sometimes be sufficient, careful Methods are still important if we are to avoid being caught by Texas sharpshooters and other biases. Finally like Polowetzky, a number of folk have expressed concerns about the possible safety of the Mother's Kiss method. To date we have found no problems reported in the case series, the case reports, and the experience of folk we have spoken to who have used it, but cleary there is a need for sufficient cumulative experience. This is analogous to the safety element of phase IV pharmaco-epidemiological studies after randomised trials. We plan to study this further, but would appreciate hearing of anyone's experience, adverse or not, with the Mother's Kiss method. Competing interests: None declared |
|||
|
|
|||
|
Friedrich Flachsbart, General Medicine Praxis 37085 Göttingen-Geismar
Send response to journal:
|
Dear Sir, only 75 years ago a miracle occurred. "On 24th December, 1932, it was found that in an experiment begun on 20th December, 1932, all the controls had died, whereas all the mice which had been given Prontosil were alive and well. This was the basis of the discovery which was destined to bring undreamed-of advances in chemotherapy." (1) Now we even forgot the furor epidemicus of these streptococcal diseases. Everybody talks against antibiotics. It is a shame. Merry Christmas Yours Friedrich Flachsbart 1. N. Svartz: Presentation Speech. The Nobel Prize in Physiology or Medicine 1939. http://nobelprize.org/nobel_prizes/medicine/laureates/1939/press.html Competing interests: None declared |
|||