Believability of relative risks and odds ratios in abstracts: cross sectional studyBMJ 2006; 333 doi: https://doi.org/10.1136/bmj.38895.410451.79 (Published 27 July 2006) Cite this as: BMJ 2006;333:231
w1.Tsang JC, Morin JF, Tchervenkov CI, Platt RW, Sampalis J, Shum-Tim D. Single aortic clamp versus partial occluding clamp technique for cerebral protection during coronary artery bypass: a randomized prospective trial. J Card Surg 2003;18:158-63.
w2. Bishai D, Qureshi A, Cantu N, Parks C. Contracting with children and helmet distribution in the emergency department to improve bicycle helmet use. Acad Emerg Med 2003;10:1371-7.
w3. Lykke N, Lerud PJ, Stromsoe K, Thorngren KG. Fixation of fractures of the femoral neck. A prospective, randomised trial of three Ullevaal hip screws versus two Hansson hook-pins. J Bone Joint Surg Br 2003;85:426-30.
w4. Thies F, Garry JM, Yaqoob P, Rerkasem K, Williams J, Shearman CP et al. Association of n-3 polyunsaturated fatty acids with stability of atherosclerotic plaques: a randomised controlled trial. Lancet 2003;361:477-85.
w5. Sato Y, Metoki N, Iwamoto J, Satoh K. Amelioration of osteoporosis and hypovitaminosis D by sunlight exposure in stroke patients. Neurology 2003;61:338-42.
w6. Gaster AL, Slothuus Skjoldborg U, Larsen J, Korsholm L, von Birgelen C, Jensen S et al. Continued improvement of clinical outcome and cost effectiveness following intravascular ultrasound guided PCI: insights from a prospective, randomised study. Heart 2003;89:1043-9.
w7. Sheu BS, Huang JJ, Yang HB, Huang AH, Wu JJ. The selection of triple therapy for Helicobacter pylori eradication in chronic renal insufficiency. Aliment Pharmacol Ther 2003;17:1283-90.
w8. Arshad SH, Bateman B, Matthews SM. Primary prevention of asthma and atopy during childhood by allergen avoidance in infancy: a randomised controlled study. Thorax 2003;58:489-93.
w9. Hughes JR, Novy P, Hatsukami DK, Jensen J, Callas PW. Efficacy of nicotine patch in smokers with a history of alcoholism. Alcohol Clin Exp Res 2003;27:946-54.
w10. Leon C, Alvarez-Lerma F, Ruiz-Santana S, Gonzalez V, de la Torre MV, Sierra R et al. Antiseptic chamber-containing hub reduces central venous catheter-related infection: a prospective, randomized study. Crit Care Med 2003;31:1318-24.
w11. Trivedi DP, Doll R, Khaw KT. Effect of four monthly oral vitamin D3 (cholecalciferol) supplementation on fractures and mortality in men and women living in the community: randomised double blind controlled trial. BMJ 2003;326:469.
w12. George TP, Vessicchio JC, Termine A, Jatlow PI, Kosten TR, O'Malley SS. A preliminary placebo-controlled trial of selegiline hydrochloride for smoking cessation. Biol Psychiatry 2003;53:136-43.
w13. Mion LC, Palmer RM, Meldon SW, Bass DM, Singer ME, Payne SM et al. Case finding and referral model for emergency department elders: a randomized clinical trial. Ann Emerg Med 2003;41:57-68.
w14. Basnyat B, Gertsch JH, Johnson EW, Castro-Marin F, Inoue Y, Yeh C. Efficacy of low-dose acetazolamide (125 mg BID) for the prophylaxis of acute mountain sickness: a prospective, double-blind, randomized, placebo-controlled trial. High Alt Med Biol 2003;4:45-52.
w15. Neumann FJ, Kastrati A, Pogatsa-Murray G, Mehilli J, Bollwein H, Bestehorn HP et al. Evaluation of prolonged antithrombotic pretreatment ("cooling-off" strategy) before intervention in patients with unstable coronary syndromes: a randomized controlled trial. JAMA 2003;290:1593-9.
w16. Fjaertoft H, Indredavik B, Lydersen S. Stroke unit care combined with early supported discharge: long-term follow-up of a randomized controlled trial. Stroke 2003;34:2687-91.
w17. Wood E, Kerr T, Spittal PM et al. An external evaluation of a peer-run "unsanctioned" syringe exchange program. J Urban Health 2003;80:455-64.
w18. Suter NM, Malone KE, Daling JR, Doody DR, Ostrander EA. Androgen receptor (CAG)n and (GGC)n polymorphisms and breast cancer risk in a population-based case-control study of young women. Cancer Epidemiol Biomarkers Prev 2003;12:127-35.
w19. Brassard P, Bourgault C, Brophy J, Kezouh A, Rainville B, Xhignesse M et al. Antibiotics in primary prevention of myocardial infarction among elderly patients with hypertension. Am Heart J 2003;145(5):E20 (given as E20 on PubMed but as E25 in the paper).
Web Table A. Results of recalcuation of P values in the interval 0.04≤P<0.05 in abstracts of randomised clinical trials.
POSTED AS SUPPLIED BY AUTHOR
Data in the abstract
Wrong. 2/135 vs 0/133. The main text notes RR=2.0 (1.77-2.26) (which gives P<0.0001). With Fisher, I got P=0.50, and with Review Manager, that adds 0.5 to the cells when any cell is empty, RR=4.93 (0.24-102). No reply from author.
Wrong. Rates in per cent for completers; these correspond to 8/53 vs 17/57. Authors report P=0.0273. With chi-square I got P=0.065 without and P=0.11 with continuity correction. OR also wrong, and CI too narrow. I got OR=0.42 (0.16-1.07) or 0.44 (0.18-1.06), depending on method, and even if all patients had been included, it would be wrong. No reply from author.
P=0421 [sic], OR=6.1
Wrong. 1/109 vs 6/108. The main text notes P=0.0421, I got P=0.065 with Fisher. Author’s comment: acknowledges the error.
Wrong. Rates in per cent; these correspond to 42/54 vs 32/54. I got P=0.061 with Fisher. Author’s comment: confirmed rates but forgot to mention under Methods that the P-value resulted from a logistic regression analysis.
P<0.05, RR=0.128 (0.016-0.979)
Chi-square, with Pearson’s correction, two-sided.
Doubtful. 2/44 vs 8/44 gives P=0.093 with adjusted chi-square. The authors excluded one of the two cases which gives P<0.05, but this post hoc exclusion seems unwarranted. No reply from author.
Chi-square or Fisher; also logistic regression.
Doubtful. 8/58 vs 17/62, authors reported P=0.08. They then adjusted for 10 factors and got P=0.04. Not clear why these 10 factors were chosen, e.g. one of them occurred for 13 vs 15 patients, although imbalance was given as the reason for the adjustments. Author’s comment: confounders also included.
Statistical methods not described in Methods, noted as "regression analyses" in Results.
Doubtful. 61 vs 54 patients. There was no difference in the unadjusted analysis the authors reported, OR 1.4 (0.6-3.5), P=0.45. Number of factors used for adjustment described as both 4 and 6. Two factors were chosen because of significant baseline difference, and 2 others because they "appeared to differ". Author’s comment: the correct number is 6; controlling for additonal factors did not change the OR much.
P<0.049, RR=4.14 (0.8-19)
Chi-square or Fisher.
Doubtful. 2/116 vs 8/114, I got P=0.058 with Fisher. The authors used one-sided alpha in the sample size calculation, and I got P=0.048 with one-sided Fisher. No reply from author.
P=0.04, RR=0.78 (0.61-0.99)
"crude rates and then, after adjustment for age, with the Cox regression method".
Doubtful. As there were 2686 patients, and the mean age differed by only 0.1 years, there was no need to adjust for age. Without adjustment, I got P=0.058. Author’s comment: adjusted because age is a strong risk factor. My comment: although this may have been perfectly reasonable in this particular case, the more general issue is that it is not likely that authors are equally inclined to adjust for a very small age difference in a large sample if the adjustment changes the P value in the opposite direction, from 0.04 to 0.058.
P<0.05, OR=4.64 (1.02-21.00)
Could be discussed. 9/20 vs 3/20. The authors did not correct for small numbers and found a chi-square of 4.29 (which gives P=0.038). I got P=0.085 with continuity correction and P=0.082 with Fisher. Author’s comment: agrees that they should have used an adjusted chi-square or Fisher.
Chi-square likelihood ratio tests, two-sided.
Could be discussed. 2/326 vs 9/324. The authors reported in a table that they used chi-square or Fisher as appropriate. I got P=0.037 with Fisher, but an odds ratio that overlapped 1: OR=0.22 (0.05-1.01). Author’s comment: by mistake, we did not use intention-to-treat for this analysis which explains the slight difference in odds ratios, but Fisher was significant in both cases.
20/81 vs 9/74, "statistically significant reduction" of 50.6% in RR
Fisher or likelihood ratio tests.
Could be discussed. The authors reported P=0.043, I got P=0.063 with Fisher, but also RR=2.03 (1.01-4.08) in support of the authors’ abstract. No reply from author.
Logistic regression adjusted for one factor.
Could be discussed. 25/38 vs 13/31. The main text notes OR=2.66 (0.90-7.95) (which gives P=0.078). Author’s comment: used two methods; the CI for OR should have been 1.000544 to 7.086235 (computer print-out with correct event rates).
RR: relative risk; 95% confidence interval (CI) in brackets; Fisher: Fisher’s exact test; Author’s comment: personal communication.
- This Week In The BMJ Published: 27 July 2006; BMJ 333 doi:10.1136/bmj.333.7561.0-d
- Education And DebateSifting the evidence—what's wrong with significance tests?Another comment on the role of statistical methodsPublished: 27 January 2001; BMJ 322 doi:10.1136/bmj.322.7280.226
- Research Methods & ReportingCONSORT 2010 Explanation and Elaboration: updated guidelines for reporting parallel group randomised trialsPublished: 24 March 2010; BMJ 340 doi:10.1136/bmj.c869
- PAPERS Published: 06 October 2004; BMJ doi:10.1136/bmj.38250.571088.55
- Paper Published: 14 October 2004; BMJ 329 doi:10.1136/bmj.38250.571088.55
- Editorial Published: 21 December 2002; BMJ 325 doi:10.1136/bmj.325.7378.1437
- Research Methods & ReportingThe PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaborationPublished: 21 July 2009; BMJ 339 doi:10.1136/bmj.b2700
- Analysis Published: 20 December 2012; BMJ 345 doi:10.1136/bmj.e8522
- PAPERSIdentifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authorsPublished: 28 January 2005; BMJ doi:10.1136/bmj.38356.424606.8F
- PaperIdentifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authorsPublished: 31 March 2005; BMJ 330 doi:10.1136/bmj.38356.424606.8F
- Education And Debate Published: 04 May 1996; BMJ 312 doi:10.1136/bmj.312.7039.1153
- ResearchDefinition, reporting, and interpretation of composite outcomes in clinical trials: systematic reviewPublished: 18 August 2010; BMJ 341 doi:10.1136/bmj.c3920
- Research Published: 14 June 2010; BMJ 340 doi:10.1136/bmj.c2697
- River blindness in East Africa: gains and lossesBMJ January 21, 2020, 368 m155; DOI: https://doi.org/10.1136/bmj.m155
- Association of ideal cardiovascular health at age 50 with incidence of dementia: 25 year follow-up of Whitehall II cohort studyBMJ August 07, 2019, 366 l4414; DOI: https://doi.org/10.1136/bmj.l4414
- Genetic predisposition to increased serum calcium, bone mineral density, and fracture risk in individuals with normal calcium levels: mendelian randomisation studyBMJ August 01, 2019, 366 l4410; DOI: https://doi.org/10.1136/bmj.l4410
- Dietary fats and mortality among patients with type 2 diabetes: analysis in two population based cohort studiesBMJ July 02, 2019, 366 l4009; DOI: https://doi.org/10.1136/bmj.l4009
- Physical activity trajectories and mortality: population based cohort studyBMJ June 26, 2019, 365 l2323; DOI: https://doi.org/10.1136/bmj.l2323
- Deficiencies in proposed new EU regulation of clinical trials
- Definition, reporting, and interpretation of composite outcomes in clinical trials: systematic review
- Modified intention to treat reporting in randomised controlled trials: systematic review
- CONSORT 2010 Explanation and Elaboration: updated guidelines for reporting parallel group randomised trials
- The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration