BMJ 1995;310:170 (21 January)

Statistics notes

Multiple significance tests: the Bonferroni method

J Martin Bland, reader in medical statistics,^a Douglas G Altman, head ^b

^a Department of Public Health Sciences, St George's Hospital Medical School, London SW17 0RE, ^b Medical Statistics Laboratory, Imperial Cancer Research Fund, London WC2A 3PX

Many published papers include large numbers of significance tests. These may be difficult to interpret because if we go on testing long enough we will inevitably find something which is "significant." We must beware of attaching too much importance to a lone significant result among a mass of non-significant ones. It may be the one in 20 which we expect by chance alone.

Lee et al simulated a clinical trial of the treatment of coronary artery disease by allocating 1073 patient records from past cases into two "treatment" groups at random.¹ They then analysed the outcome as if it were a genuine trial of two treatments. The analysis was quite detailed and thorough. As we would expect, it failed to show any significant difference in survival between those patients allocated to the two treatments. Patients were then subdivided by two variables which affect prognosis, the number of diseased coronary vessels and whether the left ventricular contraction pattern was normal or abnormal. A significant difference in survival between the two "treatment" groups was found in those patients with three diseased vessels (the maximum) and abnormal ventricular contraction. As this would be the subset of patients with the worst prognosis, the finding would be easy to account for by saying that the superior "treatment" had its greatest advantage in the most severely ill patients! This approach to the comparison of subgroups is clearly flawed.

Why does this happen? If we test a null hypothesis which is in fact true, using 0.05 as the critical significance level, we have a probability of 0.95 of coming to a not significant--that is, correct--conclusion. If we test two independent true null hypotheses, the probability that neither test will be significant is 0.95x0.95=0.90. If we test 20 such hypotheses the probability that none will be significant is 0.95²⁰=0.36. This gives a probability of 1-0.36=0.64 of getting at least one significant result--we are more likely to get one than not. The expected number of spurious significant results is 20x0.05=1. In general, if we have (kappa) independent significant tests at the (alpha) level of null hypotheses which are all true, the probability that we will get no significant differences is (1-(alpha))^(kappa). If we make (alpha) small enough we can make the probability that none of the separate tests is significant equal to 0.95. Then if any of the (kappa) tests has a P value less than (alpha) we will have a significant difference between the treatments at the 0.05 level. Since (alpha) will be very small, it can be shown that (1-(alpha))^(kappa)(about)1-(kappa) (alpha). If we put (kappa)(alpha)=0.05, so (alpha)=0.05/(kappa), we will have probability 0.05 that one of the (kappa) tests will have a P value less than (alpha) if the null hypotheses are true. Thus, if in a clinical trial we compare two treatments within five subsets of patients the treatments will be significantly different at the 0.05 level if there is a P value less than 0.01 within any of the subsets. This is the Bonferroni method. Note that they are not significant at the 0.01 level, but at only the 0.05 level.

We can do the same thing by multiplying the observed P value from the significance tests by the number of tests, (kappa), any (kappa)P which exceeds one being ignored. Then if any (kappa)P is less than 0.05 the two treatments are significant at the 0.05 level.

Williams et al randomly allocated elderly patients discharged from hospital to two groups.² There were no significant differences overall between the intervention and control groups, but among women aged 75-79 living alone the control group showed significantly greater deterioration in physical score than did the intervention group (P=0.04), and among men aged over 80 the control group showed significantly greater deterioration in disability score than did the intervention group (P=0.03). Subjects were cross classified by age groups, whether living alone, and sex, so there were at least eight subgroups and three different measurement scales. Even if we considered the scales separately the corrected P values are 8x0.04=0.32 and 8x0.03=0.24.

A similar problem arises if we have multiple outcome measurements, where the tests will not in general be independent. Newnham et al randomised pregnant women to receive either standard care or a series of Doppler ultrasound blood flow measurements.³ They found a significantly higher proportion of birth weights in the Doppler group below the 10th and 3rd centiles (P=0.006 and P=0.02). Birth weight was not the primary outcome variable for the trial. These were only two of many comparisons and one suspects that there might be some spurious significant differences among so many. At least 35 tests were reported in the paper. These tests are not independent because they are all on the same subjects, using variables which may not be independent. The proportions of birth weights below the 10th and 3rd centiles are clearly not independent, for example. The probability that two correlated variables both give non-significant differences when the null hypothesis is true is now greater than (1-(alpha))², because if the first test is not significant the second has a probability greater than 1-(alpha) of also being not significant. A P value less than (alpha) for any variable, or (kappa)P<0.05, would still mean that the treatments were significantly different. The overall P value is actually smaller than the nominal 0.05--by an unknown amount which depends on the lack of independence between the tests. The power of the test, its ability to detect true differences in the population, is correspondingly diminished. In statistical terms, the test is conservative.

For the example, we have (alpha)=0.05/35=0.0014, and so by the Bonferroni criterion the treatment groups are not significantly different. Alternatively, the P values could be adjusted to give 35x0.006=0.21 and 35x0.02=0.70.

Other multiple testing problems arise when we have more than two groups of subjects and wish to compare each pair of groups; when we have a series of observations over time, such as blood pressure every 15 minutes after administration of a drug, where there may be a temptation to test each time point separately; and when we have relations between many variables to examine, as in a survey. For all these problems the multiple tests are highly correlated and the Bonferroni method is inappropriate, as it will be highly conservative and may miss real differences. We shall deal with these types of analysis in separate notes.

1. Lee KL, McNeer JF, Starmer FC, Harris PJ, Rosati RA. Clinical judgements and statistics: lessons from a simulated randomized trial in coronary artery disease. Circulation 1980;61:508-15. [Medline]
2. Williams EI, Greenwell J, Groom LM. The care of people over 75 years old after discharge from hospital: an evaluation of timetabled visiting by health visitor assistants. J Pub Hlth Med 1992;14:138-44.
3. Newnham JP, Evans SF, Con AM, Stanley F J, Landau LI. Effects of frequent ultrasound during pregnancy: a randomized controlled trial. Lancet 1993;342:887-91. [Medline]

Related Article

Multiple significance tests

Simon Voss and Steve George
BMJ 1995 310: 1073. [Extract] [Full Text]

This article has been cited by other articles:

• Brasher, P. M.A., Brant, R. F. (2008). Problems of multiplicity/Problemes de multiplicite. Canadian J. Anesthesia 55: 259-264 [Full text]  
• Osier, F. H. A., Fegan, G., Polley, S. D., Murungi, L., Verra, F., Tetteh, K. K. A., Lowe, B., Mwangi, T., Bull, P. C., Thomas, A. W., Cavanagh, D. R., McBride, J. S., Lanar, D. E., Mackinnon, M. J., Conway, D. J., Marsh, K. (2008). Breadth and Magnitude of Antibody Responses to Multiple Plasmodium falciparum Merozoite Antigens Are Associated with Protection from Clinical Malaria. Infect. Immun. 76: 2240-2248 [Abstract] [Full text]  
• Lim, L. S., Fink, H. A., Kuskowski, M. A., Taylor, B. C., Schousboe, J. T., Ensrud, K. E., for the Osteoporotic Fractures in Men (MrOS) Study, (2008). Loop Diuretic Use and Increased Rates of Hip Bone Loss in Older Men: The Osteoporotic Fractures in Men Study. Arch Intern Med 168: 735-740 [Abstract] [Full text]  
• Ng, J. S., Bearse, M. A. Jr, Schneck, M. E., Barez, S., Adams, A. J. (2008). Local Diabetic Retinopathy Prediction by Multifocal ERG Delays over 3 Years. IOVS 49: 1622-1628 [Abstract] [Full text]  
• Okazaki, T., Jiao, L., Chang, P., Evans, D. B., Abbruzzese, J. L., Li, D. (2008). Single-Nucleotide Polymorphisms of DNA Damage Response Genes Are Associated with Overall Survival in Patients with Pancreatic Cancer. Clin. Cancer Res. 14: 2042-2048 [Abstract] [Full text]  
• Collins, C. E., Young, A. F, Hodge, A. (2008). Diet Quality Is Associated with Higher Nutrient Intake and Self-Rated Health in Mid-Aged Women. J. Am. Coll. Nutr. 27: 146-157 [Abstract] [Full text]  
• Shen, G.-Q., Li, L., Rao, S., Abdullah, K. G., Ban, J. M., Lee, B.-S., Park, J. E., Wang, Q. K. (2008). Four SNPs on Chromosome 9p21 in a South Korean Population Implicate a Genetic Locus That Confers High Cross-Race Risk for Development of Coronary Artery Disease. Arterioscler. Thromb. Vasc. Bio. 28: 360-365 [Abstract] [Full text]  
• Mackay, A. M., Bradnam, M. S., Hamilton, R., Elliot, A. T., Dutton, G. N. (2008). Real-Time Rapid Acuity Assessment Using VEPs: Development and Validation of the Step VEP Technique. IOVS 49: 438-441 [Abstract] [Full text]  
• Thomas, F. H, Ismail, R. S, Jiang, J.-Y., Vanderhyden, B. C (2008). Kit Ligand 2 Promotes Murine Oocyte Growth In Vitro. Biol. Reprod. 78: 167-175 [Abstract] [Full text]  
• Girard, O, Eicher, F, Fourchet, F, Micallef, J P, Millet, G P (2007). Effects of the playing surface on plantar pressures and potential injuries in tennis. Br. J. Sports. Med. 41: 733-738 [Abstract] [Full text]  
• Mahsoudi, B, Li, A, O'Neill, C (2007). Assessment of the Long-Term and Transgenerational Consequences of Perturbing Preimplantation Embryo Development in Mice. Biol. Reprod. 77: 889-896 [Abstract] [Full text]  
• Ashley, E. A., Spin, J. M., Tabibiazar, R., Quertermous, T. (2007). Frontiers in Nephrology: Genomic Approaches to Understanding the Molecular Basis of Atherosclerosis. J. Am. Soc. Nephrol. 18: 2853-2862 [Abstract] [Full text]  
• Thoirs, K., Williams, M. A., Phillips, M. (2007). Systematic Review of Sonographic Measurements of the Ulnar Nerve at the Elbow. Journal of Diagnostic Medical Sonography 23: 255-262 [Abstract]  
• Witjes-Ane, M.-N. W., Mertens, B., van Vugt, J. P.P., Bachoud-Levi, A.-C., van Ommen, G.-J. B., Roos, R. A.C. (2007). Longitudinal Evaluation of "Presymptomatic" Carriers of Huntington's Disease. J. Neuropsychiatry Clin. Neurosi. 19: 310-317 [Abstract] [Full text]  
• Machens, A., Hofmann, C., Hauptmann, S., Dralle, H. (2007). Locoregional recurrence and death from medullary thyroid carcinoma in a contemporaneous series: 5-year results. Eur J Endocrinol 157: 85-93 [Abstract] [Full text]  
• Ng, C. Y., Ballantyne, J. A., Brenkel, I. J. (2007). Quality of life and functional outcome after primary total hip replacement: A FIVE-YEAR FOLLOW-UP. J Bone Joint Surg Br 89-B: 868-873 [Abstract] [Full text]  
• van der Bol, J. M., Mathijssen, R. H.J., Loos, W. J., Friberg, L. E., van Schaik, R. H.N., de Jonge, M. J.A., Planting, A. S.Th., Verweij, J., Sparreboom, A., de Jong, F. A. (2007). Cigarette Smoking and Irinotecan Treatment: Pharmacokinetic Interaction and Effects on Neutropenia. JCO 25: 2719-2726 [Abstract] [Full text]  
• Klunk, W. E., Price, J. C., Mathis, C. A., Tsopelas, N. D., Lopresti, B. J., Ziolko, S. K., Bi, W., Hoge, J. A., Cohen, A. D., Ikonomovic, M. D., Saxton, J. A., Snitz, B. E., Pollen, D. A., Moonis, M., Lippa, C. F., Swearer, J. M., Johnson, K. A., Rentz, D. M., Fischman, A. J., Aizenstein, H. J., DeKosky, S. T. (2007). Amyloid Deposition Begins in the Striatum of Presenilin-1 Mutation Carriers from Two Unrelated Pedigrees. J. Neurosci. 27: 6174-6184 [Abstract] [Full text]  
• Ketel, I. J G, Volman, M. N M, Seidell, J. C, Stehouwer, C. D A, Twisk, J. W, Lambalk, C. B (2007). Superiority of skinfold measurements and waist over waist-to-hip ratio for determination of body fat distribution in a population-based cohort of Caucasian Dutch adults. Eur J Endocrinol 156: 655-661 [Abstract] [Full text]  
• Liu, X., Yuan, L., Shen, F., Zhu, Z., Jiang, H., Guo, S.-W. (2007). Patterns of and Risk Factors for Recurrence in Women With Ovarian Endometriomas. Obstet Gynecol 109: 1411-1420 [Abstract] [Full text]  
• Kalavrouziotis, D., Buth, K. J., Ali, I. S. (2007). The Impact of New-Onset Atrial Fibrillation on In-hospital Mortality Following Cardiac Surgery. Chest 131: 833-839 [Abstract] [Full text]  
• Wong, P.-l., Chamari, K., Chaouachi, A., Mao, D. W., Wisloff, U., Hong, Y. (2007). Difference in plantar pressure between the preferred and non-preferred feet in four soccer-related movements. Br. J. Sports. Med. 41: 84-92 [Abstract] [Full text]  
• Wong, P.-l., Chamari, K., Mao, D. W., Wisloff, U., Hong, Y. (2007). Higher plantar pressure on the medial side in four soccer-related movements. Br. J. Sports. Med. 41: 93-100 [Abstract] [Full text]  
• Poolman, R. W., Sierevelt, I. N., Farrokhyar, F., Mazel, J. A., Blankevoort, L., Bhandari, M. (2007). Perceptions and Competence in Evidence-Based Medicine: Are Surgeons Getting Better? A Questionnaire Survey of Members of the Dutch Orthopaedic Association. JBJS 89: 206-215 [Abstract] [Full text]  
• Stevens, N., Westerhof, G. J. (2006). Partners and others: Social provisions and loneliness among married Dutch men and women in the second half of life. Journal of Social and Personal Relationships 23: 921-941 [Abstract]  
• Sun, T., Gao, Y., Tan, W., Ma, S., Zhang, X., Wang, Y., Zhang, Q., Guo, Y., Zhao, D., Zeng, C., Lin, D. (2006). Haplotypes in Matrix Metalloproteinase Gene Cluster on Chromosome 11q22 Contribute to the Risk of Lung Cancer Development and Progression. Clin. Cancer Res. 12: 7009-7017 [Abstract] [Full text]  
• Huffard, C. L. (2006). Locomotion by Abdopus aculeatus (Cephalopoda: Octopodidae): walking the line between primary and secondary defenses. J. Exp. Biol. 209: 3697-3707 [Abstract] [Full text]  
• Tsukimoto, M., Maehata, M., Harada, H., Ikari, A., Takagi, K., Degawa, M. (2006). P2X7 Receptor-Dependent Cell Death Is Modulated during Murine T Cell Maturation and Mediated by Dual Signaling Pathways.. J. Immunol. 177: 2842-2850 [Abstract] [Full text]  
• Forsberg, F., Liu, J.-B., Shi, W. T., Ro, R., Lipcan, K. J., Deng, X., Hall, A. L. (2006). In vivo perfusion estimation using subharmonic contrast microbubble signals.. J Ultrasound Med 25: 15-21 [Abstract] [Full text]  
• Kahles, H., Ramos-Lopez, E., Lange, B., Zwermann, O., Reincke, M., Badenhoop, K. (2005). Sex-specific association of PTPN22 1858T with type 1 diabetes but not with Hashimoto's thyroiditis or Addison's disease in the German population. Eur J Endocrinol 153: 895-899 [Abstract] [Full text]  
• Camp, S. J., Stevenson, V. L., Thompson, A. J., Ingle, G. T., Miller, D. H., Borras, C., Brochet, B., Dousset, V., Falautano, M., Filippi, M., Kalkers, N. F., Montalban, X., Polman, C. H., Langdon, D. W. (2005). A longitudinal study of cognition in primary progressive multiple sclerosis. Brain 128: 2891-2898 [Abstract] [Full text]  
• Eisler, I., Dare, C., Hodes, M., Russell, G., Dodge, E., Le Grange, D. (2005). Family Therapy for Adolescent Anorexia Nervosa: The Results of a Controlled Comparison of Two Family Interventions. Focus 3: 629-640 [Abstract] [Full text]  
• Awomoyi, A. A., Charurat, M., Marchant, A., Miller, E. N., Blackwell, J. M., McAdam, K. P.W.J., Newport, M. J. (2005). Polymorphism in IL1B: IL1B--511 association with tuberculosis and decreased lipopolysaccharide-induced IL-1{beta} in IFN-{gamma} primed ex-vivo whole blood assay. Innate Immunity 11: 281-286 [Abstract]  
• Al-Jenaidi, F. A., Wakim-Ghorayeb, S. F., Al-Abbasi, A., Arekat, M. R., Irani-Hakime, N., Najm, P., Al-Ola, K., Motala, A. A., Almawi, W. Y. (2005). Contribution of Selective HLA-DRB1/DQB1 Alleles and Haplotypes to the Genetic Susceptibility of Type 1 Diabetes among Lebanese and Bahraini Arabs. J. Clin. Endocrinol. Metab. 90: 5104-5109 [Abstract] [Full text]  
• Greenough, A, Limb, E, Marston, L, Marlow, N, Calvert, S, Peacock, J (2005). Risk factors for respiratory morbidity in infancy after very premature birth. Arch. Dis. Child. Fetal Neonatal Ed. 90: F320-f323 [Abstract] [Full text]  
• Anderson, T. J. C., Nair, S., Qin, H., Singlam, S., Brockman, A., Paiphun, L., Nosten, F. (2005). Are Transporter Genes Other than the Chloroquine Resistance Locus (pfcrt) and Multidrug Resistance Gene (pfmdr) Associated with Antimalarial Drug Resistance?. Antimicrob. Agents Chemother. 49: 2180-2188 [Abstract] [Full text]  
• Powers, J. H., Lin, D., Ross, D., Wunderink, R., Kollef, M., Rello, J. (2005). FDA Evaluation of Antimicrobials: Subgroup Analysis. Chest 127: 2298-2301 [Full text]  
• Oudard, S., Banu, E., Beuzeboc, P., Voog, E., Dourthe, L. M., Hardy-Bessard, A. C., Linassier, C., Scotte, F., Banu, A., Coscas, Y., Guinet, F., Poupon, M.-F., Andrieu, J.-M. (2005). Multicenter Randomized Phase II Study of Two Schedules of Docetaxel, Estramustine, and Prednisone Versus Mitoxantrone Plus Prednisone in Patients With Metastatic Hormone-Refractory Prostate Cancer. JCO 23: 3343-3351 [Abstract] [Full text]  
• Hecht, S. S., Murphy, S. E., Carmella, S. G., Li, S., Jensen, J., Le, C., Joseph, A. M., Hatsukami, D. K. (2005). Similar Uptake of Lung Carcinogens by Smokers of Regular, Light, and Ultralight Cigarettes. Cancer Epidemiol. Biomarkers Prev. 14: 693-698 [Abstract] [Full text]  
• Sorokin, R., Riggio, J. M., Hwang, C. (2005). Attitudes About Patient Safety: A Survey of Physicians-in-Training. American Journal of Medical Quality 20: 70-77 [Abstract]  
• Diprose, P., Herbertson, M. J., O'Shaughnessy, D., Deakin, C. D., Gill, R. S. (2005). Reducing allogeneic transfusion in cardiac surgery: a randomized double-blind placebo-controlled trial of antifibrinolytic therapies used in addition to intra-operative cell salvage. Br J Anaesth 94: 271-278 [Abstract] [Full text]  
• Wemeau, J. L., Caron, P., Beckers, A., Rohmer, V., Orgiazzi, J., Borson-Chazot, F., Nocaudie, M., Perimenis, P., Bisot-Locard, S., Bourdeix, I., Dejager, S. (2005). Octreotide (Long-Acting Release Formulation) Treatment in Patients with Graves' Orbitopathy: Clinical Results of a Four-Month, Randomized, Placebo-Controlled, Double-Blind Study. J. Clin. Endocrinol. Metab. 90: 841-848 [Abstract] [Full text]  
• van der Mee-Marquet, N., Domelier, A.-S., Girard, N., Quentin, R., the Bloodstream Infection Study Group of the Relai, (2004). Epidemiology and Typing of Staphylococcus aureus Strains Isolated from Bloodstream Infections. J. Clin. Microbiol. 42: 5650-5657 [Abstract] [Full text]  
• Almawi, W. Y., Busson, M., Tamim, H., Al-Harbi, E. M., Finan, R. R., Wakim-Ghorayeb, S. F., Motala, A. A. (2004). HLA Class II Profile and Distribution of HLA-DRB1 and HLA-DQB1 Alleles and Haplotypes among Lebanese and Bahraini Arabs. CVI 11: 770-774 [Abstract] [Full text]  
• Powers, J. H., Ross, D. B., Lin, D., Soreth, J., Wunderink, R. G., Kollef, M., Rello, J. (2004). Linezolid and Vancomycin for Methicillin-Resistant Staphylococcus aureus Nosocomial Pneumonia: The Subtleties of Subgroup Analyses. Chest 126: 314-316 [Full text]  
• Hoorn, E. J., Geary, D., Robb, M., Halperin, M. L., Bohn, D. (2004). Acute Hyponatremia Related to Intravenous Fluid Administration in Hospitalized Children: An Observational Study. Pediatrics 113: 1279-1284 [Abstract] [Full text]  
• Thieblemont, C., Nasser, V., Felman, P., Leroy, K., Gazzo, S., Callet-Bauchu, E., Loriod, B., Granjeaud, S., Gaulard, P., Haioun, C., Traverse-Glehen, A., Baseggio, L., Bertucci, F., Birnbaum, D., Magrangeas, F., Minvielle, S., Avet-Loiseau, H., Salles, G., Coiffier, B., Berger, F., Houlgatte, R. (2004). Small lymphocytic lymphoma, marginal zone B-cell lymphoma, and mantle cell lymphoma exhibit distinct gene-expression profiles allowing molecular diagnosis. Blood 103: 2727-2737 [Abstract] [Full text]  
• Reimer, J, Grabowski, M, Lindvall, O, Hagell, P (2004). Use and interpretation of on/off diaries in Parkinson's disease. J. Neurol. Neurosurg. Psychiatry 75: 396-400 [Abstract] [Full text]  
• Seplaki, C. L., Goldman, N., Weinstein, M., Lin, Y.-H. (2004). How Are Biomarkers Related to Physical and Mental Well-Being?. J. Gerontol. A Biol. Sci. Med. Sci. 59: B201-B217 [Abstract] [Full text]  
• Hecht, S. S., Chen, M., Yagi, H., Jerina, D. M., Carmella, S. G. (2003). r-1,t-2,3,c-4-Tetrahydroxy-1,2,3,4-tetrahydrophenanthrene in Human Urine: A Potential Biomarker for Assessing Polycyclic Aromatic Hydrocarbon Metabolic Activation. Cancer Epidemiol. Biomarkers Prev. 12: 1501-1508 [Abstract] [Full text]  
• Skinner, J., Weinstein, J. N., Sporer, S. M., Wennberg, J. E. (2003). Racial, Ethnic, and Geographic Disparities in Rates of Knee Arthroplasty among Medicare Patients. NEJM 349: 1350-1359 [Abstract] [Full text]  
• Blanck, H. M., Pfeiffer, C. M., Caudill, S. P., Reyes, M., Gunter, E. W., Imperatore, G., van Assendelft, O. W., Strider, S., Dearth, T. (2003). Serum Iron and Iron-binding Capacity: A Round-Robin Interlaboratory Comparison Study. Clin. Chem. 49: 1672-1675 [Full text]  
• Hawton, K., Houston, K., Haw, C., Townsend, E., Harriss, L. (2003). Comorbidity of Axis I and Axis II Disorders in Patients Who Attempted Suicide. Am. J. Psychiatry 160: 1494-1500 [Abstract] [Full text]  
• CEDERLUND, R., IWARSSON, S., LUNDBORG, G. (2003). Hand Function Tests and Questions on Hand Symptoms as Related to the Stockholm Workshop Scales for Diagnosis of Hand-Arm Vibration Syndrome. J Hand Surg Eur Vol 28: 165-171 [Abstract] [Full text]  
• Witjes-Ane, M.-N. W., Vegter-van der Vlis, M., van Vugt, J. P.P., Lanser, J. B.K., Hermans, J., Zwinderman, A. H., van Ommen, G.-J. B., Roos, R. A.C. (2003). Cognitive and Motor Functioning in Gene Carriers for Huntington's Disease: A Baseline Study. J. Neuropsychiatry Clin. Neurosi. 15: 7-16 [Abstract] [Full text]  
• Pasipoularides, A., Shu, M., Shah, A., Silvestry, S., Glower, D. D. (2002). Right ventricular diastolic function in canine models of pressure overload, volume overload, and ischemia. Am. J. Physiol. Heart Circ. Physiol. 283: H2140-H2150 [Abstract] [Full text]  
• Johnson, A. H., Peacock, J. L., Greenough, A., Marlow, N., Limb, E. S., Marston, L., Calvert, S. A., the United Kingdom Oscillation Study Group, (2002). High-Frequency Oscillatory Ventilation for the Prevention of Chronic Lung Disease of Prematurity. NEJM 347: 633-642 [Abstract] [Full text]  
• Saraux, A, Maillefert, J F, Fautrel, B, Flipo, R M, Kaye, O, Lafforgue, P, Guillemin, F, Botton, E (2002). Laboratory and imaging studies used by French rheumatologists to determine the cause of recent onset polyarthritis without extra-articular manifestations. Ann Rheum Dis 61: 626-629 [Abstract] [Full text]  
• Weets, I., Siraux, V., Daubresse, J.-C., De Leeuw, I. H., Fery, F., Keymeulen, B., Krzentowski, G., Letiexhe, M., Mathieu, C., Nobels, F., Rottiers, R., Scheen, A., Van Gaal, L., Schuit, F. C., Van Der Auwera, B., Rui, M., De Pauw, P., Kaufman, L., Gorus, F. K. (2002). Relation between Disease Phenotype and HLA-DQ Genotype in Diabetic Patients Diagnosed in Early Adulthood. J. Clin. Endocrinol. Metab. 87: 2597-2605 [Abstract] [Full text]  
• Allison, D. B., Coffey, C. S. (2002). Two-Stage Testing in Microarray Analysis: What Is Gained?. J. Gerontol. A Biol. Sci. Med. Sci. 57: B189-192 [Abstract] [Full text]  
• O'Hara, L., Cadbury, H., De Souza, L., Ide, L. (2002). Evaluation of the effectiveness of professionally guided self-care for people with multiple sclerosis living in the community: A randomized controlled trial. Clin Rehabil 16: 119-128 [Abstract]  
• Balkestein, E. J., Staessen, J. A., Wang, J.-G., van der Heijden-Spek, J. J., Van Bortel, L. M., Barlassina, C., Bianchi, G., Brand, E., Herrmann, S.-M., Struijker-Boudier, H. A. (2001). Carotid and Femoral Artery Stiffness in Relation to Three Candidate Genes in a White Population. Hypertension 38: 1190-1197 [Abstract] [Full text]  
• Flaherty, J. H., Takahashi, R., Teoh, , Kim, J.-I., Habib, S., Ito, M., Matsushita, S. (2001). Use of Alternative Therapies in Older Outpatients in the United States and Japan: Prevalence, Reporting Patterns, and Perceived Effectiveness. J. Gerontol. A Biol. Sci. Med. Sci. 56: M650-655 [Abstract] [Full text]  
• Walker-Batson, D., Curtis, S., Natarajan, R., Ford, J., Dronkers, N., Salmeron, E., Lai, J., Unwin, D. H., Feeney, D. M. (2001). A Double-Blind, Placebo-Controlled Study of the Use of Amphetamine in the Treatment of Aphasia Editorial Comment. Stroke 32: 2093-2098 [Abstract] [Full text]  
• Camp, S J, Thompson, A J, Langdon, D W (2001). A new test of memory for multiple sclerosis 1: Format development and stimuli design. Mult Scler 7: 255-262 [Abstract]  
• O'Dwyer, P. J., Manola, J., Valone, F. H., Ryan, L. M., Hines, J. D., Wadler, S., Haller, D. G., Arbuck, S. G., Weiner, L. M., Mayer, R. J., Benson, A. B. III (2001). Fluorouracil Modulation in Colorectal Cancer: Lack of Improvement With N -Phosphonoacetyl- l -Aspartic Acid or Oral Leucovorin or Interferon, But Enhanced Therapeutic Index With Weekly 24-Hour Infusion Schedule--An Eastern Cooperative Oncology Group/Cancer and Leukemia Group B Study. JCO 19: 2413-2421 [Abstract] [Full text]  
• Fried, P. W., Wechsler, A. S. (2001). How to get your paper published. J. Thorac. Cardiovasc. Surg. 121: S3-7 [Abstract] [Full text]  
• Tyrberg, B., Ustinov, J., Otonkoski, T., Andersson, A. (2001). Stimulated Endocrine Cell Proliferation and Differentiation in Transplanted Human Pancreatic Islets: Effects of the ob Gene and Compensatory Growth of the Implantation Organ. Diabetes 50: 301-307 [Abstract] [Full text]  
• Sperling, R. A., Guttmann, C. R. G., Hohol, M. J., Warfield, S. K., Jakab, M., Parente, M., Diamond, E. L., Daffner, K. R., Olek, M. J., Orav, E. J., Kikinis, R., Jolesz, F. A., Weiner, H. L. (2001). Regional Magnetic Resonance Imaging Lesion Burden and Cognitive Function in Multiple Sclerosis: A Longitudinal Study. Arch Neurol 58: 115-121 [Abstract] [Full text]  
• Jeffery, K. J. M., Siddiqui, A. A., Bunce, M., Lloyd, A. L., Vine, A. M., Witkover, A. D., Izumo, S., Usuku, K., Welsh, K. I., Osame, M., Bangham, C. R. M. (2000). The Influence of HLA Class I Alleles and Heterozygosity on the Outcome of Human T Cell Lymphotropic Virus Type I Infection. J. Immunol. 165: 7278-7284 [Abstract] [Full text]  
• Signoretti, S., Montironi, R., Manola, J., Altimari, A., Tam, C., Bubley, G., Balk, S., Thomas, G., Kaplan, I., Hlatky, L., Hahnfeldt, P., Kantoff, P., Loda, M. (2000). Her-2-neu Expression and Progression Toward Androgen Independence in Human Prostate Cancer. JNCI J Natl Cancer Inst 92: 1918-1925 [Abstract] [Full text]  
• CUESTA, M. J., PERALTA, V., ZARZUELA, A. (2000). Reappraising insight in psychosis: Multi-scale longitudinal study. Br. J. Psychiatry 177: 233-240 [Abstract] [Full text]  
• Kirk, C. W, Graham, C. A, McDonnell, G. V, Hawkins, S. A (2000). Chromosome 19 locus apolipoprotein C-II association with multiple sclerosis. Mult Scler 6: 291-292  
• Jung, H., Wensing, M, de Wilt, A, Olesen, F, Grol, R (2000). Comparison of patients' preferences and evaluations regarding aspects of general practice care. Fam Pract 17: 236-242 [Abstract] [Full text]  
• Butland, B. K, Fehily, A. M, Elwood, P. C (2000). Diet, lung function, and lung function decline in a cohort of 2512 middle aged men. Thorax 55: 102-108 [Abstract] [Full text]  
• Herman, N. L., Choi, K. C., Affleck, P. J., Calicott, R., Brackin, R., Singhal, A., Andreasen, A., Gadalla, F., Fong, J., Gomillion, M. C., Hartman, J. K., Koff, H. D., Lee, S. H. R., Decar, T. K. V. (1999). Analgesia, Pruritus, and Ventilation Exhibit a Dose-Response Relationship in Parturients Receiving Intrathecal Fentanyl During Labor. Anesth. Analg. 89: 378-378 [Abstract] [Full text]  
• Camp, S. J., Stevenson, V. L., Thompson, A. J., Miller, D. H., Borras, C., Auriacombe, S., Brochet, B., Falautano, M., Filippi, M., Herisse-Dulo, L., Montalban, X., Parrcira, E., Polman, C. H., De Sa, J., Langdon, D. W. (1999). Cognitive function in primary progressive and transitional progressive multiple sclerosis: A controlled study with MRI correlates. Brain 122: 1341-1348 [Abstract] [Full text]  
• Viswanathan, A. C., McNaught, A. I., Poinoosawmy, D., Fontana, L., Crabb, D. P., Fitzke, F. W., Hitchings, R. A. (1999). Severity and Stability of Glaucoma: Patient Perception Compared With Objective Measurement. Arch Ophthalmol 117: 450-454 [Abstract] [Full text]  
• Jeffery, K. J. M., Usuku, K., Hall, S. E., Matsumoto, W., Taylor, G. P., Procter, J., Bunce, M., Ogg, G. S., Welsh, K. I., Weber, J. N., Lloyd, A. L., Nowak, M. A., Nagai, M., Kodama, D., Izumo, S., Osame, M., Bangham, C. R. M. (1999). HLA alleles determine human T-lymphotropic virus-I (HTLV-I) proviral load and the risk of HTLV-I-associated myelopathy. Proc. Natl. Acad. Sci. USA 96: 3848-3853 [Abstract] [Full text]  
• Fossum, E., Hoieggen, A., Moan, A., Rostrup, M., Kjeldsen, S. E. (1999). Insulin Sensitivity Is Related to Physical Fitness and Exercise Blood Pressure to Structural Vascular Properties in Young Men. Hypertension 33: 781-786 [Abstract] [Full text]  
• Bender, R., Lange, S. (1999). Multiple test procedures other than Bonferroni's deserve wider use. BMJ 318: 600a-600 [Full text]  
• Fossum, E., Hoieggen, A., Moan, A., Rostrup, M., Nordby, G., Kjeldsen, S. E. (1998). Relationship Between Insulin Sensitivity and Maximal Forearm Blood Flow in Young Men. Hypertension 32: 838-843 [Abstract] [Full text]  
• Nussbaum, A. K., Dick, T. P., Keilholz, W., Schirle, M., Dietz, K., Heinemeyer, W., Groll, M., Wolf, D. H., Huber, R., Rammensee, H.-G., Schild, H. (1998). Cleavage motifs of the yeast 20S proteasome beta  subunits deduced from digests of enolase 1. Proc. Natl. Acad. Sci. USA 95: 12504-12509 [Abstract] [Full text]  
• Agarwal, M. R., Sharma, V. K., K.V., K. K., Lowe, D. (1998). Non-Compliance With Treatment in Patients Suffering From Schizophrenia: a Study To Evaluate Possible Contributing Factors. International Journal of Social Psychiatry 44: 92-106 [Abstract]  
• Perneger, T. V (1998). What's wrong with Bonferroni adjustments. BMJ 316: 1236-1238 [Full text]  
• Zaman, M. M., Ikemoto, S., Yoshiike, N., Date, C., Yokoyama, T., Tanaka, H. (1997). Association of Apolipoprotein Genetic Polymorphisms With Plasma Cholesterol in a Japanese Rural Population : The Shibata Study. Arterioscler. Thromb. Vasc. Bio. 17: 3495-3504 [Abstract] [Full text]  
• Robinson, T., Potter, J. (1997). Cardiopulmonary and Arterial Baroreflex-Mediated Control of Forearm Vasomotor Tone Is Impaired After Acute Stroke. Stroke 28: 2357-2362 [Abstract] [Full text]  
• Ye, J., Yang, L., Del Bigio, M. R., Summers, R., Jackson, D., Somorjai, R. L., Salerno, T. A., Deslauriers, R. (1997). RETROGRADE CEREBRAL PERFUSION PROVIDES LIMITED DISTRIBUTION OF BLOOD TO THE BRAIN: A STUDY IN PIGS. J. Thorac. Cardiovasc. Surg. 114: 660-665 [Abstract] [Full text]  
• Jacobson, L., Zurakowski, D., Majzoub, J. A. (1997). Protein Malnutrition Increases Plasma Adrenocorticotropin and Anterior Pituitary Proopiomelanocortin Messenger Ribonucleic Acid in the Rat. Endocrinology 138: 1048-1057 [Abstract] [Full text]  
• Altman, D. G, Matthews, J. N S (1996). Statistics Notes: Interaction 1: heterogeneity of effects. BMJ 313: 486-486 [Full text]  
• Altman, D. G, Bland, J M. (1996). Statistics Notes: Comparing several groups using analysis of variance. BMJ 312: 1472-1473 [Full text]  
• Voss, S., George, S. (1995). Multiple significance tests. BMJ 310: 1073-1073 [Full text]