BMJ 1998;316:1236-1238 ( 18 April )

Education and debate

What's wrong with Bonferroni adjustments

Thomas V Perneger, medical epidemiologist. 

Institute of Social and Preventive Medicine, University of Geneva, CH-1211 Geneva 4, Switzerland

Correspondence to: Dr Perneger perneger{at}cmu.unige.ch

When more than one statistical test is performed in analysing the data from a clinical study, some statisticians and journal editors demand that a more stringent criterion be used for "statistical significance" than the conventional P<0.05.¹ Many well meaning researchers, eager for methodological rigour, comply without fully grasping what is at stake. Recently, adjustments for multiple tests (or Bonferroni adjustments) have found their way into introductory texts on medical statistics, which has increased their apparent legitimacy. This paper advances the view, widely held by epidemiologists, that Bonferroni adjustments are, at best, unnecessary and, at worst, deleterious to sound statistical inference.

Summary points Adjusting statistical significance for the number of tests that have been performed on study datathe Bonferroni methodcreates more problems than it solves The Bonferroni method is concerned with the general null hypothesis (that all null hypotheses are true simultaneously), which is rarely of interest or use to researchers The main weakness is that the interpretation of a finding depends on the number of other tests performed The likelihood of type II errors is also increased, so that truly important differences are deemed non-significant Simply describing what tests of significance have been performed, and why, is generally the best way of dealing with multiple comparisons

	Adjustment for multiple tests

Bonferroni adjustments are based on the following reasoning.^1-3 If a null hypothesis is true (for instance, two treatment groups in a randomised trial do not differ in terms of cure rates), a significant difference (P<0.05) will be observed by chance once in 20 trials. This is the type I error, or . When 20 independent tests are performed (for example, study groups are compared with regard to 20 unrelated variables) and the null hypothesis holds for all 20 comparisons, the chance of at least one test being significant is no longer 0.05, but 0.64. The formula for the error rate across the study is 1(1)ⁿ, where n is the number of tests performed. However, the Bonferroni adjustment deflates the  applied to each, so the study-wide error rate remains at 0.05. The adjusted significance level is 1(1)^1/n (in this case 0.00256), often approximated by /n (here 0.0025). What is wrong with this statistical approach?

	Problems

Irrelevant null hypothesis
The first problem is that Bonferroni adjustments are concerned with the wrong hypothesis.^4-6 The study- wide error rate applies only to the hypothesis that the two groups are identical on all 20 variables (the universal null hypothesis). If one or more of the 20 P values is less than 0.00256, the universal null hypothesis is rejected. We can say that the two groups are not equal for all 20 variables, but we cannot say which, or even how many, variables differ. Such information is usually of no interest to the researcher, who wants to assess each variable in its own right. A clinical equivalent would be the case of a doctor who orders 20 different laboratory tests for a patient, only to be told that some are abnormal, without further detail. Thus, Bonferroni adjustments provide a correct answer to a largely irrelevant question.

Inference defies common sense
Bonferroni adjustments imply that a given comparison will be interpreted differently according to how many other tests were performed. For example, the difference in remission rates between two chemotherapeutic treatments could be interpreted as statistically significant or not depending on whether or not survival rates, quality of life scores, and complication rates were also tested. In a clinical setting, a patient's packed cell volume might be abnormally low, except if the doctor also ordered a platelet count, in which case it could be deemed normal. Surely this is absurd, at least within the current scientific paradigm. Evidence in data is what the data sayother considerations, such as how many other tests were performed, are irrelevant.

Increase in type II errors
Type I errors cannot decrease (the whole point of Bonferroni adjustments) without inflating type II errors (the probability of accepting the null hypothesis when the alternative is true).⁴ And type II errors are no less false than type I errors. In clinical practice, if a high concentration of creatine kinase were considered compatible with "no myocardial infarction" by virtue of a Bonferroni adjustment, the patient would be denied appropriate care. In research, an effective treatment may be deemed no better than placebo. Thus, contrary to what some researchers believe, Bonferroni adjustments do not guarantee a "prudent" interpretation of results.

What tests should be included?
Most proponents of the Bonferroni method would count at least all the statistical tests in a given report as a basis for adjusting P values. But how about tests that were performed, but not published, or tests published in other papers based on the same study? If several papers are planned, should future ones be accounted for in the first publication? Should we worry about error rates related to an investigatortaking the number of tests he or she has done in their lifetime into consideration⁶or error rates related to journals? Should confidence intervals, which are not statistical tests, but are often interpreted as such (the confidence interval includes 0, hence the groups do not differ) be counted? No statistical theory provides answers for these practical issues.

View larger version (60K): [in this window] [in a new window]

	A futuristic scenario

What would happen to biomedical research if Bonferroni adjustments became routine? Cynical researchers would slice their results like salami, publishing one P value at a time to escape the wrath of the statistical reviewer. Idealists would conduct studies to examine only one association at a timewasting time, energy, and public money. Meta-analysts would go out of business, since a pooled analysis would invalidate retrospectively all original findings by adding more tests to be adjusted for. Journals would have to create a new section entitled "P value updates," in which P values of previously published papers would be corrected for newly published tests based on the same study. And so on ....

	Back to the Neyman-Pearson theory

These objections seem so compelling that the reader may wonder why adjustments for multiple tests were developed at all. The answer is that such adjustments are correct in the original framework of statistical test theory, proposed by Neyman and Pearson in the 1920s.⁷ This theory was intended to aid decisions in repetitive situations. Imagine that your factory produces light bulbs in lots of 1000, and that testing each bulb before shipment would be impractical. You can decide to test only a sample in each lot, and to reject (literally) any lots in which more than a predefined number (x) of bulbs in the sample are defective. Of course, your decision might be wrong for any particular lot, but the Neyman-Pearson theory provides a decision rule (the number x), so that over many trials your error rates (type I and type II) will be minimised. Now, if for some reason you took 20 samples out of a given lot instead of one, and decided that you would reject the lot if the number of defective bulbs exceeded x in only one sample, you would be much too likely to reject a good lot in error, and a Bonferroni adjustment would restore the original optimal error rates.

The catch is that Neyman and Pearson developed their statistical tests to aid decision making, not to assess evidence in data. The latter practice may be objected to for several reasons (this topic would deserve a discussion of its own), and alternative approaches to statistical inference, such as estimation procedures, use of likelihood ratios, and Bayesian methods, have been proposed.^8-11 Bonferroni adjustments follow the original logic of statistical tests as supports of repeated decisions, but they are of little help in determining what the data say in one particular study.

	Should Bonferroni adjustments ever be used?

Statistical adjustment for multiple tests make sense in a few situations. Firstly, the universal null hypothesis is occasionally of interest. For instance, to verify that a disease is not associated with an HLA phenotype, we may compare available HLA antigens (perhaps 40) in a group of cases and controls. If no association existed, at least one test would be significant with a probability of 0.87, and Bonferroni adjustments would protect against making excessive claims. A clinical equivalent is the case of a healthy person undergoing several laboratory tests as part of a general health check. Secondly, adjustments are appropriate when the same test is repeated in many subsamples, such as when stratified analyses (by age group, sex, income status, etc) are conducted without an a priori hypothesis that the primary association should differ between these subgroups. Note that this is the scenario, reminiscent of repeated sampling of the same lot, that Tukey and Bland and Altman use in their justifications of multiple test adjustments. Sequential testing of trial results also falls in this category. A final situation in which Bonferroni adjustments may be acceptable is when searching for significant associations without pre-established hypotheses.

	The best approach

However, even in these situations, simply describing what was done and why, and discussing the possible interpretations of each result, should enable the reader to reach a reasonable conclusion without the help of Bonferroni adjustments. There is an important difference between what the data say and what the researcher (or the reader) believes to be true.⁸ The latter depends not only on the data at hand but also on considerations such as whether a finding is biologically plausible or whether the significant test was a serendipitous finding in a fishing expedition. The integration of prior beliefs with evidence is best achieved by Bayesian methods, not by Bonferroni adjustments. In summary, Bonferroni adjustments have, at best, limited applications in biomedical research, and should not be used when assessing evidence about specific hypotheses.

	Acknowledgments

I thank Dr Richard M Royall, Department of Biostatistics, Johns Hopkins University, for helpful comments on the manuscript.

Funding: Swiss National Science Foundation (PROSPER 3233-32609.91).

Conflict of interest: None.

	References

Top References

1. Tukey JW. Some thoughts on clinical trials, especially problems of multiplicity. Science 1977; 198: 679-684.
2. Greenhalgh T. Statistics for the non-statistician. l. Different types of data need different statistical tests. BMJ 1997; 315: 364-366[Free Full Text].
3. Bland JM, Altman DG. Multiple significance tests: the Bonferroni method. BMJ 1995; 310: 170[Free Full Text].
4. Rothman KJ. No adjustments are needed for multiple comparisons. Epidemiology 1990; 1: 43-46[Medline].
5. Savitz DA, Olshan AF. Multiple comparisons and related issues in the interpretation of epidemiologic data. Am J Epidemiol 1995; 142: 904-908[Free Full Text].
6. Thomas DC, Siemiatycki J, Dewar R, Robins J, Goldberg M, Armstrong RG. The problem of multiple inference in studies designed to generate hypotheses. Am J Epidemiol 1985; 122: 1080-1095[Abstract/Free Full Text].
7. Neyman J, Pearson ES. On the use and interpretation of certain test criteria for purposes of statistical inference. Biometrika 1928; 20A: 175-240, 263-97. 
8. Royall RM. Statistical inference: a likelihood paradigm. New York: Chapman and Hall , 1997.
9. Berkson J. Tests of significance considered as evidence. J Am Stat Assoc 1942; 37: 325-335.
10. Goodman SN, Royall R. Evidence and scientific research. Am J Public Health 1988; 78: 1568-1574[Abstract/Free Full Text].
11. Oakes M. Statistical inference. Boston: Epidemiologic Resources , 1990.
12. Jones DR, Rushton L. Simultaneous inference in epidemiologic studies. Int J Epidemiol 1982; 11: 276-282[Abstract/Free Full Text].

(Accepted 16 January 1998)

---

© BMJ 1998
CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

Related Articles

Racial variation in the association between gestational age and perinatal mortality: prospective study

Imelda Balchin, John C Whittaker, Roshni R Patel, Ronald F Lamont, and Philip J Steer
BMJ 2007 334: 833. [Abstract] [Full Text] [PDF]

Multiple test procedures other than Bonferroni's deserve wider use

Ralf Bender and Stefan Lange
BMJ 1999 318: 600. [Extract] [Full Text]

Other method for adjustment of multiple testing exists

Mikel Aickin
BMJ 1999 318: 127. [Extract] [Full Text]

This article has been cited by other articles:

• Burke, V., Beilin, L. J., Cutt, H. E., Mansour, J., Mori, T. A. (2008). Moderators and mediators of behaviour change in a lifestyle program for treated hypertensives: a randomized controlled trial (ADAPT). Health Educ Res 23: 583-591 [Abstract] [Full text]  
• Kantarci, K., Knopman, D. S., Dickson, D. W., Parisi, J. E., Whitwell, J. L., Weigand, S. D., Josephs, K. A., Boeve, B. F., Petersen, R. C., Jack, C. R. Jr (2008). Alzheimer Disease: Postmortem Neuropathologic Correlates of Antemortem 1H MR Spectroscopy Metabolite Measurements. Radiology 248: 210-220 [Abstract] [Full text]  
• Lucchinetti, C. F., Gavrilova, R. H., Metz, I., Parisi, J. E., Scheithauer, B. W., Weigand, S., Thomsen, K., Mandrekar, J., Altintas, A., Erickson, B. J., Konig, F., Giannini, C., Lassmann, H., Linbo, L., Pittock, S. J., Bruck, W. (2008). Clinical and radiographic spectrum of pathologically confirmed tumefactive multiple sclerosis. Brain 131: 1759-1775 [Abstract] [Full text]  
• Sookoian, S, Gianotti, T F, Pirola, C J (2008). Role of the C-344T aldosterone synthase gene variant in left ventricular mass and left ventricular structure-related phenotypes. Heart 94: 903-910 [Abstract] [Full text]  
• Terracciano, A., Lockenhoff, C. E., Zonderman, A. B., Ferrucci, L., Costa, P. T. Jr (2008). Personality Predictors of Longevity: Activity, Emotional Stability, and Conscientiousness. Psychosom. Med. 70: 621-627 [Abstract] [Full text]  
• Jones, P. M., Armstrong, K. P., Armstrong, P. M., Cherry, R. A., Harle, C. C., Hoogstra, J., Turkstra, T. P. (2008). A Comparison of GlideScope(R) Videolaryngoscopy to Direct Laryngoscopy for Nasotracheal Intubation. Anesth. Analg. 107: 144-148 [Abstract] [Full text]  
• McWilliams, R. R., Bamlet, W. R., Cunningham, J. M., Goode, E. L., de Andrade, M., Boardman, L. A., Petersen, G. M. (2008). Polymorphisms in DNA Repair Genes, Smoking, and Pancreatic Adenocarcinoma Risk. Cancer Res. 68: 4928-4935 [Abstract] [Full text]  
• Ingle, L, Cleland, J G F, Clark, A L (2008). Perception of symptoms is out of proportion to cardiac pathology in patients with "diastolic heart failure". Heart 94: 748-753 [Abstract] [Full text]  
• Flores, G., Tomany-Korman, S. C. (2008). The Language Spoken at Home and Disparities in Medical and Dental Health, Access to Care, and Use of Services in US Children. Pediatrics 121: e1703-e1714 [Abstract] [Full text]  
• Josephs, K. A., Whitwell, J. L., Knopman, D. S., Hu, W. T., Stroh, D. A., Baker, M., Rademakers, R., Boeve, B. F., Parisi, J. E., Smith, G. E., Ivnik, R. J., Petersen, R. C., Jack, C. R. Jr, Dickson, D. W. (2008). Abnormal TDP-43 immunoreactivity in AD modifies clinicopathologic and radiologic phenotype. Neurology 70: 1850-1857 [Abstract] [Full text]  
• Johnson, E. J, Chung, H.-Y., Caldarella, S. M, Snodderly, D M. (2008). The influence of supplemental lutein and docosahexaenoic acid on serum, lipoproteins, and macular pigmentation. Am. J. Clin. Nutr. 87: 1521-1529 [Abstract] [Full text]  
• Krishnan, V., Collop, N. A., Scherr, S. C. (2008). An Evaluation of a Titration Strategy for Prescription of Oral Appliances for Obstructive Sleep Apnea. Chest 133: 1135-1141 [Abstract] [Full text]  
• Thomas, C., Hypponen, E., Power, C. (2008). Obesity and Type 2 Diabetes Risk in Midadult Life: The Role of Childhood Adversity. Pediatrics 121: e1240-e1249 [Abstract] [Full text]  
• Tam, L.-S., Tomlinson, B., Chu, T. T.-W., Li, M., Leung, Y.-Y., Kwok, L.-W., Li, T. K., Yu, T., Zhu, Y.-E., Wong, K.-C., Kun, E. W.-L., Li, E. K. (2008). Cardiovascular risk profile of patients with psoriatic arthritis compared to controls--the role of inflammation. Rheumatology (Oxford) 47: 718-723 [Abstract] [Full text]  
• Dotson, V. M., Resnick, S. M., Zonderman, A. B. (2008). Differential Association of Concurrent, Baseline, and Average Depressive Symptoms With Cognitive Decline in Older Adults. AJGP 16: 318-330 [Abstract] [Full text]  
• Bebbington, A., Anderson, A., Ravine, D., Fyfe, S., Pineda, M., de Klerk, N., Ben-Zeev, B., Yatawara, N., Percy, A., Kaufmann, W. E., Leonard, H. (2008). Investigating genotype-phenotype relationships in Rett syndrome using an international data set. Neurology 70: 868-875 [Abstract] [Full text]  
• Balchin, I., Whittaker, J. C., Lamont, R. F., Steer, P. J. (2008). Timing of Planned Cesarean Delivery by Racial Group. Obstet Gynecol 111: 659-666 [Abstract] [Full text]  
• McKeever, T M, Lewis, S A, Cassano, P A, Ocke, M, Burney, P, Britton, J, Smit, H A (2008). The relation between dietary intake of individual fatty acids, FEV1 and respiratory disease in Dutch adults. Thorax 63: 208-214 [Abstract] [Full text]  
• Wittink, M. N., Bogner, H. R. (2008). Primary Care Physicians' Assessments of Older Patients' Health and Psychological Status and Recommendation of Mammography. J Am Board Fam Med 21: 17-23 [Abstract] [Full text]  
• Lundby, C., Boushel, R., Robach, P., Moller, K., Saltin, B., Calbet, J. A. L. (2008). During hypoxic exercise some vasoconstriction is needed to match O2 delivery with O2 demand at the microcirculatory level. J. Physiol. 586: 123-130 [Abstract] [Full text]  
• Haase, D., Germing, U., Schanz, J., Pfeilstocker, M., Nosslinger, T., Hildebrandt, B., Kundgen, A., Lubbert, M., Kunzmann, R., Giagounidis, A. A. N., Aul, C., Trumper, L., Krieger, O., Stauder, R., Muller, T. H., Wimazal, F., Valent, P., Fonatsch, C., Steidl, C. (2007). New insights into the prognostic impact of the karyotype in MDS and correlation with subtypes: evidence from a core dataset of 2124 patients. Blood 110: 4385-4395 [Abstract] [Full text]  
• Conrad, R., Weber, N. F., Lehnert, M., Holz, F. G., Liedtke, R., Eter, N. (2007). Alexithymia and Emotional Distress in Patients With Central Serous Chorioretinopathy. Psychosomatics 48: 489-495 [Abstract] [Full text]  
• Hillenkamp, J., Kraus, J., Framme, C., Jackson, T. L, Roider, J., Gabel, V.-P., Sachs, H. G (2007). Retreatment of full-thickness macular hole: predictive value of optical coherence tomography. Br. J. Ophthalmol. 91: 1445-1449 [Abstract] [Full text]  
• Matakidou, A., el Galta, R., Webb, E. L., Rudd, M. F., Bridle, H., the GELCAPS Consortium, , Eisen, T., Houlston, R. S. (2007). Genetic variation in the DNA repair genes is predictive of outcome in lung cancer. Hum Mol Genet 16: 2333-2340 [Abstract] [Full text]  
• Khaleeli, Z, Sastre-Garriga, J, Ciccarelli, O, Miller, D H, Thompson, A J (2007). Magnetisation transfer ratio in the normal appearing white matter predicts progression of disability over 1 year in early primary progressive multiple sclerosis. J. Neurol. Neurosurg. Psychiatry 78: 1076-1082 [Abstract] [Full text]  
• Hwang, R., Shinkai, T., De Luca, V., Xingqun Ni, , Potkin, S. G., Lieberman, J. A., MeLtzer, H. Y., Kennedy, J. L. (2007). Association study of four dopamine D1 receptor gene polymorphisms and clozapine treatment response. J Psychopharmacol 21: 718-727 [Abstract]  
• Huntley, E. D., Campo, J. V., Dahl, R. E., Lewin, D. S. (2007). Sleep Characteristics of Youth with Functional Abdominal Pain and a Healthy Comparison Group. J Pediatr Psychol 32: 938-949 [Abstract] [Full text]  
• Liu, Y., Zhang, H., Zhou, K., Chen, L., Xu, Z., Zhong, Y., Liu, H., Li, R., Shugart, Y. Y., Wei, Q., Jin, L., Huang, F., Lu, D., Zhou, L. (2007). Tagging SNPs in non-homologous end-joining pathway genes and risk of glioma. Carcinogenesis 28: 1906-1913 [Abstract] [Full text]  
• Rao, F., Zhang, L., Wessel, J., Zhang, K., Wen, G., Kennedy, B. P., Rana, B. K., Das, M., Rodriguez-Flores, J. L., Smith, D. W., Cadman, P. E., Salem, R. M., Mahata, S. K., Schork, N. J., Taupenot, L., Ziegler, M. G., O'Connor, D. T. (2007). Tyrosine Hydroxylase, the Rate-Limiting Enzyme in Catecholamine Biosynthesis: Discovery of Common Human Genetic Variants Governing Transcription, Autonomic Activity, and Blood Pressure In Vivo. Circulation 116: 993-1006 [Abstract] [Full text]  
• Lucas, R., Ponsonby, A-L., McMichael, A., van der Mei, I., Chapman, C., Coulthard, A., Dear, K., Dwyer, T., Kilpatrick, T., Pender, M., Taylor, B., Valery, P., Williams, D. (2007). Observational analytic studies in multiple sclerosis: controlling bias through study design and conduct. The Australian Multicentre Study of Environment and Immune Function. Mult Scler 13: 827-839 [Abstract]  
• McMorris, B. J., Hemphill, S. A., Toumbourou, J. W., Catalano, R. F., Patton, G. C. (2007). Prevalence of Substance Use and Delinquent Behavior in Adolescents From Victoria, Australia and Washington State, United States. Health Educ Behav 34: 634-650 [Abstract]  
• Oberlander, T. F., Warburton, W., Misri, S., Aghajanian, J., Hertzman, C. (2007). Neonatal Risks of Maternal Treatment With Mood Stabilizers--Reply. Arch Gen Psychiatry 64: 867-868 [Full text]  
• Barlassina, C., Dal Fiume, C., Lanzani, C., Manunta, P., Guffanti, G., Ruello, A., Bianchi, G., Del Vecchio, L., Macciardi, F., Cusi, D. (2007). Common genetic variants and haplotypes in renal CLCNKA gene are associated to salt-sensitive hypertension. Hum Mol Genet 16: 1630-1638 [Abstract] [Full text]  
• Cechaviciute, I., Kenny, D. T. (2007). The Relationship Between Neutralizations and Perceived Delinquent Labeling on Criminal History in Young Offenders Serving Community Orders. Criminal Justice and Behavior 34: 816-829 [Abstract]  
• Covington, M. A., Riedel, W. J., Brown, C., He, C., Morris, E., Weinstein, S., SempLe, J., Brown, J. (2007). Does ketamine mimic aspects of schizophrenic speech?. J Psychopharmacol 21: 338-346 [Abstract]  
• Morgan, J. F (2007). p Value fetishism and use of the Bonferroni adjustment. Evid. Based Ment. Health 10: 34-35 [Full text]  
• Balchin, I., Whittaker, J. C, Patel, R. R, Lamont, R. F, Steer, P. J (2007). Racial variation in the association between gestational age and perinatal mortality: prospective study. BMJ 334: 833-833 [Abstract] [Full text]  
• Rudi, K., Zimonja, M., Kvenshagen, B., Rugtveit, J., Midtvedt, T., Eggesbo, M. (2007). Alignment-Independent Comparisons of Human Gastrointestinal Tract Microbial Communities in a Multidimensional 16S rRNA Gene Evolutionary Space. Appl. Environ. Microbiol. 73: 2727-2734 [Abstract] [Full text]  
• Tomovic, A., Oakeley, E. J. (2007). Position dependencies in transcription factor binding sites. Bioinformatics 23: 933-941 [Abstract] [Full text]  
• Book, A. S., Quinsey, V. L., Langford, D. (2007). Psychopathy and the Perception of Affect and Vulnerability. Criminal Justice and Behavior 34: 531-544 [Abstract]  
• Ingle, L., Rigby, A. S., Carroll, S., Butterly, R., King, R. F., Cooke, C. B., Cleland, J. G.J.F., Clark, A. L. (2007). Prognostic value of the 6 min walk test and self-perceived symptom severity in older patients with chronic heart failure. Eur Heart J 28: 560-568 [Abstract] [Full text]  
• Harding, D R, Humphries, S E, Whitelaw, A, Marlow, N, Montgomery, H E (2007). Cognitive outcome and cyclo-oxygenase-2 gene (-765 G/C) variation in the preterm infant. Arch. Dis. Child. Fetal Neonatal Ed. 92: F108-F112 [Abstract] [Full text]  
• Albini, T. A., Abd-El-Barr, M. M., Carvounis, P. E., Iyer, M. N., Lakhanpal, R. R., Pennesi, M. E., Chevez-Barrios, P., Wu, S. M., Holz, E. R. (2007). Long-term Retinal Toxicity of Intravitreal Commercially Available Preserved Triamcinolone Acetonide (Kenalog) in Rabbit Eyes. IOVS 48: 390-395 [Abstract] [Full text]  
• Calisher, C. H., Wagoner, K. D., Amman, B. R., Root, J. J., Douglass, R. J., Kuenzi, A. J., Abbott, K. D., Parmenter, C., Yates, T. L., Ksiazek, T. G., Beaty, B. J., Mills, J. N. (2007). DEMOGRAPHIC FACTORS ASSOCIATED WITH PREVALENCE OF ANTIBODY TO SIN NOMBRE VIRUS IN DEER MICE IN THE WESTERN UNITED STATES. J Wildl Dis 43: 1-11 [Abstract] [Full text]  
• Chow, S.-M., Yau, S., Kwan, C.-K., Poon, P. C M, Law, S. C K (2006). Local and regional control in patients with papillary thyroid carcinoma: specific indications of external radiotherapy and radioactive iodine according to T and N categories in AJCC 6th edition. Endocr Relat Cancer 13: 1159-1172 [Abstract] [Full text]  
• Sjaheim, T., Kongerud, J., Bjortuft, O., Drablos, P. A., Malterud, D., Halstensen, T. S. (2006). Reduced bronchial CD4+ T-cell density in smokers with occupational asthma. Eur Respir J 28: 1138-1144 [Abstract] [Full text]  
• Ariza, M., Matarin, M. d., Junque, C., Mataro, M., Clemente, I., Moral, P., Antonia Poca, M., Garnacho, A., Sahuquillo, J. (2006). Influence of Angiotensin-converting enzyme polymorphism on neuropsychological subacute performance in moderate and severe traumatic brain injury.. J. Neuropsychiatry Clin. Neurosi. 18: 39-44 [Abstract] [Full text]  
• Shanley, S., McReynolds, K., Ardern-Jones, A., Ahern, R., Fernando, I., Yarnold, J., Evans, G., Eccles, D., Hodgson, S., Ashley, S., Ashcroft, L., Tutt, A., Bancroft, E., Short, S., Gui, G., The Breast Unit of the Royal Marsden NHS Foundatio, , Barr, L., Baildam, A., Howell, A., Royle, G., Pierce, L., Easton, D., Eeles, R. (2006). Late Toxicity Is Not Increased in BRCA1/BRCA2 Mutation Carriers Undergoing Breast Radiotherapy in the United Kingdom. Clin. Cancer Res. 12: 7025-7032 [Abstract] [Full text]  
• Shanley, S., McReynolds, K., Ardern-Jones, A., Ahern, R., Fernando, I., Yarnold, J., Evans, G., Eccles, D., Hodgson, S., Ashley, S., Ashcroft, L., Tutt, A., Bancroft, E., Short, S., Smith, I., Gui, G., The Breast Unit of the Royal Marsden NHS Foundatio, , Barr, L., Baildam, A., Howell, A., Royle, G., Pierce, L., Easton, D., Eeles, R. (2006). Acute Chemotherapy-Related Toxicity Is Not Increased in BRCA1 and BRCA2 Mutation Carriers Treated for Breast Cancer in the United Kingdom. Clin. Cancer Res. 12: 7033-7038 [Abstract] [Full text]  
• Jawad, M., Seedhouse, C. H., Russell, N., Plumb, M. (2006). Polymorphisms in human homeobox HLX1 and DNA repair RAD51 genes increase the risk of therapy-related acute myeloid leukemia. Blood 108: 3916-3918 [Abstract] [Full text]  
• Lui, K., Abdel-Latif, M. E., Allgood, C. L., Bajuk, B., Oei, J., Berry, A., Henderson-Smart, D., and the New South Wales and Australian Capital Ter, (2006). Improved Outcomes of Extremely Premature Outborn Infants: Effects of Strategic Changes in Perinatal and Retrieval Services. Pediatrics 118: 2076-2083 [Abstract] [Full text]  
• LeBlanc, J. C., Pless, I. B., King, W. J., Bawden, H., Bernard-Bonnin, A.-C., Klassen, T., Tenenbein, M. (2006). Home safety measures and the risk of unintentional injury among young children: a multicentre case-control study. CMAJ 175: 883-887 [Abstract] [Full text]  
• Kasner, S. E., Lynn, M. J., Chimowitz, M. I., Frankel, M. R., Howlett-Smith, H., Hertzberg, V. S., Chaturvedi, S., Levine, S. R., Stern, B. J., Benesch, C. G., Jovin, T. G., Sila, C. A., Romano, J. G., for the Warfarin Aspirin Symptomatic Intracranial, (2006). Warfarin vs aspirin for symptomatic intracranial stenosis: Subgroup analyses from WASID.. Neurology 67: 1275-1278 [Abstract] [Full text]  
• Shelton, R. J., Clark, A. L., Goode, K., Rigby, A. S., Cleland, J. G.F. (2006). The diagnostic utility of N-terminal pro-B-type natriuretic peptide for the detection of major structural heart disease in patients with atrial fibrillation. Eur Heart J 27: 2353-2361 [Abstract] [Full text]  
• Benzinou, M., Walley, A., Lobbens, S., Charles, M.-A., Jouret, B., Fumeron, F., Balkau, B., Meyre, D., Froguel, P. (2006). Bardet-biedl syndrome gene variants are associated with both childhood and adult common obesity in French Caucasians.. Diabetes 55: 2876-2882 [Abstract] [Full text]  
• Witte, K K A, Nikitin, N P, Cleland, J G F, Clark, A L (2006). Excessive breathlessness in patients with diastolic heart failure. Heart 92: 1425-1429 [Abstract] [Full text]  
• Laurvick, C. L., Msall, M. E., Silburn, S., Bower, C., Klerk, N. d., Leonard, H. (2006). Physical and Mental Health of Mothers Caring for a Child With Rett Syndrome. Pediatrics 118: e1152-e1164 [Abstract] [Full text]  
• Dijk, J. M., van der Graaf, Y., Bots, M. L., Grobbee, D. E., Algra, A., on behalf of the SMART study group, (2006). Carotid intima-media thickness and the risk of new vascular events in patients with manifest atherosclerotic disease: the SMART study. Eur Heart J 27: 1971-1978 [Abstract] [Full text]  
• Calbet, J. A. L., Lundby, C., Sander, M., Robach, P., Saltin, B., Boushel, R. (2006). Effects of ATP-induced leg vasodilation on VO2 peak and leg O2 extraction during maximal exercise in humans. Am. J. Physiol. Regul. Integr. Comp. Physiol. 291: R447-R453 [Abstract] [Full text]  
• Dickson, J M, Wilkinson, I D, Howell, S J L, Griffiths, P D, Grunewald, R A (2006). Idiopathic generalised epilepsy: a pilot study of memory and neuronal dysfunction in the temporal lobes, assessed by magnetic resonance spectroscopy. J. Neurol. Neurosurg. Psychiatry 77: 834-840 [Abstract] [Full text]  
• Garamszegi, L. Z. (2006). Comparing effect sizes across variables: generalization without the need for Bonferroni correction. Behav Ecol 17: 682-687 [Full text]  
• Abdel-Latif, M E, Bajuk, B, Oei, J, Vincent, T, Sutton, L, Lui, K, on behalf of the Neonatal Intensive Care Units Gro, (2006). Does rural or urban residence make a difference to neonatal outcome in premature birth? A regional study in Australia. Arch. Dis. Child. Fetal Neonatal Ed. 91: F251-F256 [Abstract] [Full text]  
• Abdel-Latif, M. E., Pinner, J., Clews, S., Cooke, F., Lui, K., Oei, J. (2006). Effects of breast milk on the severity and outcome of neonatal abstinence syndrome among infants of drug-dependent mothers.. Pediatrics 117: e1163-e1169 [Abstract] [Full text]  
• Garamszegi, L. Z., Merino, S., Torok, J., Eens, M., Martinez, J. (2006). Indicators of physiological stress and the elaboration of sexual traits in the collared flycatcher. Behav Ecol 17: 399-404 [Abstract] [Full text]  
• Fekkes, M., Pijpers, F. I.M., Fredriks, A. M., Vogels, T., Verloove-Vanhorick, S. P. (2006). Do Bullied Children Get Ill, or Do Ill Children Get Bullied? A Prospective Cohort Study on the Relationship Between Bullying and Health-Related Symptoms. Pediatrics 117: 1568-1574 [Abstract] [Full text]  
• Abdel-Latif, M. E., Bajuk, B., Oei, J., Lui, K., for the New South Wales and the Australian Capital, (2006). Mortality and Morbidities Among Very Premature Infants Admitted After Hours in an Australian Neonatal Intensive Care Unit Network. Pediatrics 117: 1632-1639 [Abstract] [Full text]  
• de Silva, R., Nikitin, N. P., Witte, K. K.A., Rigby, A. S., Goode, K., Bhandari, S., Clark, A. L., Cleland, J. G.F. (2006). Incidence of renal dysfunction over 6 months in patients with chronic heart failure due to left ventricular systolic dysfunction: contributing factors and relationship to prognosis. Eur Heart J 27: 569-581 [Abstract] [Full text]  
• Masuki, S., Eisenach, J. H., Dinenno, F. A., Joyner, M. J. (2006). Reduced forearm {alpha}1-adrenergic vasoconstriction is associated with enhanced heart rate fluctuations in humans. J. Appl. Physiol. 100: 792-799 [Abstract] [Full text]  
• Freburger, J. K, Carey, T. S, Holmes, G. M (2006). Effectiveness of Physical Therapy for the Management of Chronic Spine Disorders: A Propensity Score Approach. ptjournal 86: 381-394 [Abstract] [Full text]  
• Ong, K. K., Emmett, P. M., Noble, S., Ness, A., Dunger, D. B., and the ALSPAC Study Team, (2006). Dietary Energy Intake at the Age of 4 Months Predicts Postnatal Weight Gain and Childhood Body Mass Index. Pediatrics 117: e503-e508 [Abstract] [Full text]  
• Hakkanen, H., Laajasalo, T. (2006). Homicide Crime Scene Behaviors in a Finnish Sample of Mentally Ill Offenders. Homicide Studies 10: 33-54 [Abstract]  
• Kallmann, B A, Fackelmann, S, Toyka, K V, Rieckmann, P, Reiners, K (2006). Early abnormalities of evoked potentials and future disability in patients with multiple sclerosis. Mult Scler 12: 58-65 [Abstract]  
• Forsberg, F., Liu, J.-B., Shi, W. T., Ro, R., Lipcan, K. J., Deng, X., Hall, A. L. (2006). In vivo perfusion estimation using subharmonic contrast microbubble signals.. J Ultrasound Med 25: 15-21 [Abstract] [Full text]  
• Westfall, P., Young, S. S. (2005). Contradictions in Highly Cited Clinical Research. JAMA 294: 2695-2696 [Full text]  
• Buechel, E. R. V., Dave, H. H., Kellenberger, C. J., Dodge-Khatami, A., Pretre, R., Berger, F., Bauersfeld, U. (2005). Remodelling of the right ventricle after early pulmonary valve replacement in children with repaired tetralogy of Fallot: assessment by cardiovascular magnetic resonance. Eur Heart J 26: 2721-2727 [Abstract] [Full text]  
• Booth, P. J, Humpherson, P. G, Watson, T. J, Leese, H. J (2005). Amino acid depletion and appearance during porcine preimplantation embryo development in vitro. Reproduction 130: 655-668 [Abstract] [Full text]  
• Witte, K. K.A., Nikitin, N. P., Parker, A. C., von Haehling, S., Volk, H.-D., Anker, S. D., Clark, A. L., Cleland, J. G.F. (2005). The effect of micronutrient supplementation on quality-of-life and left ventricular function in elderly patients with chronic heart failure. Eur Heart J 26: 2238-2244 [Abstract] [Full text]  
• Racette, L., Boden, C., Kleinhandler, S. L., Girkin, C. A., Liebmann, J. M., Zangwill, L. M., Medeiros, F. A., Bowd, C., Weinreb, R. N., Wilson, M. R., Sample, P. A. (2005). Differences in Visual Function and Optic Nerve Structure Between Healthy Eyes of Blacks and Whites. Arch Ophthalmol 123: 1547-1553 [Abstract] [Full text]  
• Van Vliet, M., Spruit, M. A., Verleden, G., Kasran, A., Van Herck, E., Pitta, F., Bouillon, R., Decramer, M. (2005). Hypogonadism, Quadriceps Weakness, and Exercise Intolerance in Chronic Obstructive Pulmonary Disease. Am. J. Respir. Crit. Care Med. 172: 1105-1111 [Abstract] [Full text]  
• Bell, C. G., Meyre, D., Samson, C., Boyle, C., Lecoeur, C., Tauber, M., Jouret, B., Jaquet, D., Levy-Marchal, C., Charles, M. A., Weill, J., Gibson, F., Mein, C. A., Froguel, P., Walley, A. J. (2005). Association of Melanin-Concentrating Hormone Receptor 1 5' Polymorphism With Early-Onset Extreme Obesity. Diabetes 54: 3049-3055 [Abstract] [Full text]  
• Ingle, L., Shelton, R. J., Rigby, A. S., Nabb, S., Clark, A. L., Cleland, J. G.F. (2005). The reproducibility and sensitivity of the 6-min walk test in elderly patients with chronic heart failure. Eur Heart J 26: 1742-1751 [Abstract] [Full text]  
• Vernon, S. A., Hawker, M. J., Ainsworth, G., Hillman, J. G., MacNab, H. K., Dua, H. S. (2005). Laser Scanning Tomography of the Optic Nerve Head in a Normal Elderly Population: The Bridlington Eye Assessment Project. IOVS 46: 2823-2828 [Abstract] [Full text]  
• Terra-Bustamante, V C, Coimbra, E R, Rezek, K O, Escorsi-Rosset, S R, Guarnieri, R, Dalmagro, C L, Inuzuka, L M, Bianchin, M M, Wichert-Ana, L, Alexandre, V, Takayanagui, O M, Araujo, D, dos Santos, A C, Carlotti, C G, Walz, R, Markowitsch, H J, Sakamoto, A C (2005). Cognitive performance of patients with mesial temporal lobe epilepsy and incidental calcified neurocysticercosis. J. Neurol. Neurosurg. Psychiatry 76: 1080-1083 [Abstract] [Full text]  
• Wade, T. J., Bielitz, S. (2005). The Differential Effect of Skin Color on Attractiveness, Personality Evaluations, and Perceived Life Success of African Americans. Journal of Black Psychology 31: 215-236 [Abstract]  
• Joo, E. Y., Hong, S. B., Han, H. J., Tae, W. S., Kim, J. H., Han, S. J., Seo, D. W., Lee, K.-H., Hong, S.-C., Lee, M., Kim, S., Kim, B. T. (2005). Postoperative alteration of cerebral glucose metabolism in mesial temporal lobe epilepsy. Brain 128: 1802-1810 [Abstract] [Full text]  
• Gilboa, S. M., Mendola, P., Olshan, A. F., Langlois, P. H., Savitz, D. A., Loomis, D., Herring, A. H., Fixler, D. E. (2005). Relation between Ambient Air Quality and Selected Birth Defects, Seven County Study, Texas, 1997-2000. Am J Epidemiol 162: 238-252 [Abstract] [Full text]  
• Parker, K. P., Bliwise, D. L., Bailey, J. L., Rye, D. B. (2005). Polysomnographic measures of nocturnal sleep in patients on chronic, intermittent daytime haemodialysis vs those with chronic kidney disease. Nephrol Dial Transplant 20: 1422-1428 [Abstract] [Full text]  
• Rutherford, J. J., Clutton-Brock, T. H., Parkes, M. J. (2005). Hypocapnia reduces the T wave of the electrocardiogram in normal human subjects. Am. J. Physiol. Regul. Integr. Comp. Physiol. 289: R148-R155 [Abstract] [Full text]  
• Greenough, A, Limb, E, Marston, L, Marlow, N, Calvert, S, Peacock, J (2005). Risk factors for respiratory morbidity in infancy after very premature birth. Arch. Dis. Child. Fetal Neonatal Ed. 90: F320-f323 [Abstract] [Full text]  
• Lederle, F. A. (2005). Endovascular Repair of Abdominal Aortic Aneurysm -- Round Two. NEJM 352: 2443-2445 [Full text]  
• Anderson, T. J. C., Nair, S., Qin, H., Singlam, S., Brockman, A., Paiphun, L., Nosten, F. (2005). Are Transporter Genes Other than the Chloroquine Resistance Locus (pfcrt) and Multidrug Resistance Gene (pfmdr) Associated with Antimalarial Drug Resistance?. Antimicrob. Agents Chemother. 49: 2180-2188 [Abstract] [Full text]  
• Biederman, J., Kwon, A., Aleardi, M., Chouinard, V.-A., Marino, T., Cole, H., Mick, E., Faraone, S. V. (2005). Absence of Gender Effects on Attention Deficit Hyperactivity Disorder: Findings in Nonreferred Subjects. Am. J. Psychiatry 162: 1083-1089 [Abstract] [Full text]  
• Sastre-Garriga, J., Ingle, G. T., Chard, D. T., Cercignani, M., Ramio-Torrenta, L., Miller, D. H., Thompson, A. J. (2005). Grey and white matter volume changes in early primary progressive multiple sclerosis: a longitudinal study. Brain 128: 1454-1460 [Abstract] [Full text]  
• Gattellari, M., Donnelly, N., Taylor, N., Meerkin, M., Hirst, G., Ward, J. E (2005). Does 'peer coaching' increase GP capacity to promote informed decision making about PSA screening? A cluster randomised trial. Fam Pract 22: 253-265 [Abstract] [Full text]  

Rapid Responses:

Read all Rapid Responses

What's wrong with arguments against multiplicity adjustments

Ralf Bender
bmj.com, 28 Aug 1998 [Full text]