Systematic reviews in health care: Investigating and dealing with publication and other biases in meta-analysis

doi:10.1136/bmj.323.7304.101

BMJ 2001;323:101-105 ( 14 July )

Education and debate

Systematic reviews in health care

Investigating and dealing with publication and other biases in meta-analysis

This is the second in a series of four articles

Jonathan A C Sterne, senior lecturer in medical statistics, Matthias Egger, senior lecturer in epidemiology and public health medicine, George Davey Smith, professor of clinical epidemiology.

Medical Research Council Health Services Research Collaboration, Department of Social Medicine, University of Bristol, Bristol BS8 2PR

Correspondence to: J A C Sterne jonathan.sterne@bristol.ac.uk

The first 150 words of the full text of this article appear below.

Studies that show a significant effect of treatment are more likely to be published, be published in English, be cited byother authors, and produce multiple publications than other studies.^1-8Such studies are therefore also more likely to be identified andincluded in systematic reviews, which may introduce bias.⁹Low methodological quality of studies included in a systematicreview is another important source of bias.¹⁰

All these biases are more likely to affect small studies than large ones. The smaller a study the larger the treatment effectnecessary for the results to be significant. The greater investmentof time and money in larger studies means that they are more likelyto be of high methodological quality and published even if theirresults are negative. Bias in a systematic review may thereforebecome evident through an association between the size of thetreatment effect and study size $---$ such associations may be . . . [Full text of this article]

CiteULike

Complore

Connotea

Del.icio.us Add to Digg

Digg

StumbleUpon Add to Technorati

Technorati What's this?

Relevant Articles

Income inequality, mortality, and self rated health: meta-analysis of multilevel studies: Naoki Kondo, Grace Sembajwe, Ichiro Kawachi, Rob M van Dam, S V Subramanian, and Zentaro Yamagata
BMJ 2009 339: b4471. [Abstract] [Full Text] [PDF]

Novel methods to deal with publication biases: secondary analysis of antidepressant trials in the FDA trial registry database and related journal publications: Santiago G Moreno, Alex J Sutton, Erick H Turner, Keith R Abrams, Nicola J Cooper, Tom M Palmer, and A E Ades
BMJ 2009 339: b2981. [Abstract] [Full Text] [PDF]

Effects of antithyroid drugs on radioiodine treatment: systematic review and meta-analysis of randomised controlled trials: Martin A Walter, Matthias Briel, Mirjam Christ-Crain, Steen J Bonnema, John Connell, David S Cooper, Heiner C Bucher, Jan Müller-Brand, and Beat Müller
BMJ 2007 334: 514. [Abstract] [Full Text] [PDF]

Is society losing control of the medical research agenda?: Brendan Delaney
BMJ 2006 332: 1063-1064. [Full Text] [PDF]

Website of the week: Systematic reviews and the Cochrane Collaboration: Marcus Müllner
BMJ 2001 323: 116. [Full Text]

This article has been cited by other articles:

Kondo, N., Sembajwe, G., Kawachi, I., van Dam, R. M, Subramanian, S V, Yamagata, Z. (2009). Income inequality, mortality, and self rated health: meta-analysis of multilevel studies. BMJ 339: b4471-b4471 [Abstract] [Full text]
Etminan, M., Sadatsafavi, M., Jafari, S., Doyle-Waters, M., Aminzadeh, K., FitzGerald, J. M. (2009). Acetaminophen Use and the Risk of Asthma in Children and Adults: A Systematic Review and Metaanalysis. Chest 136: 1316-1323 [Abstract] [Full text]
Yazdanyar, S., Weischer, M., Nordestgaard, B. G. (2009). Genotyping for NOD2 Genetic Variants and Crohn Disease: a Metaanalysis. Clin. Chem. 55: 1950-1957 [Abstract] [Full text]
Ho-Pham, L. T, Nguyen, N. D, Nguyen, T. V (2009). Effect of vegetarian diets on bone mineral density: a Bayesian meta-analysis. Am. J. Clin. Nutr. 90: 943-950 [Abstract] [Full text]
Moreno, S. G, Sutton, A. J, Turner, E. H, Abrams, K. R, Cooper, N. J, Palmer, T. M, Ades, A E (2009). Novel methods to deal with publication biases: secondary analysis of antidepressant trials in the FDA trial registry database and related journal publications. BMJ 339: b2981-b2981 [Abstract] [Full text]
Steel, Z., Chey, T., Silove, D., Marnane, C., Bryant, R. A., van Ommeren, M. (2009). Association of Torture and Other Potentially Traumatic Events With Mental Health Outcomes Among Populations Exposed to Mass Conflict and Displacement: A Systematic Review and Meta-analysis. JAMA 302: 537-549 [Abstract] [Full text]
Pavia, M., Bianco, A., Nobile, C. G. A., Marinelli, P., Angelillo, I. F. (2009). Efficacy of Pneumococcal Vaccination in Children Younger Than 24 Months: A Meta-Analysis. Pediatrics 123: e1103-e1110 [Abstract] [Full text]
Dentali, F., Squizzato, A., Brivio, L., Appio, L., Campiotti, L., Crowther, M., Grandi, A. M., Ageno, W. (2009). JAK2V617F mutation for the early diagnosis of Ph- myeloproliferative neoplasms in patients with venous thromboembolism: a meta-analysis. Blood 113: 5617-5623 [Abstract] [Full text]
Panoulas, V F, Nikas, S N, Smith, J P, Douglas, K M J, Nightingale, P, Milionis, H J, Treharne, G J, Toms, T E, Kita, M D, Kitas, G D (2008). Lymphotoxin 252A>G polymorphism is common and associates with myocardial infarction in patients with rheumatoid arthritis. Ann Rheum Dis 67: 1550-1556 [Abstract] [Full text]
Islami, F., Kamangar, F. (2008). Helicobacter pylori and Esophageal Cancer Risk: A Meta-analysis. Cancer Prevention Research 1: 329-338 [Abstract] [Full text]
Mostafa, M. M. (2008). Through the Eye of a Child: A Meta-analytic Study of Children's Understanding of Advertising Intent. Global Business Review 9: 243-255 [Abstract]
Shuldham, C., Fleming, S., Yorke, J. (2008). Undertaking a systematic review: the road to successful completion. Journal of Research in Nursing 13: 282-298 [Abstract]
Zambon, M., Vincent, J.-L. (2008). Mortality Rates for Patients With Acute Lung Injury/ARDS Have Decreased Over Time. Chest 133: 1120-1127 [Abstract] [Full text]
Garcia, A. M, Sisternas, A., Hoyos, S. P. (2008). Occupational exposure to extremely low frequency electric and magnetic fields and Alzheimer disease: a meta-analysis. Int J Epidemiol 37: 329-340 [Abstract] [Full text]
Dartnell, J., Hemming, M., Collier, J., Ollenschlaeger, G. (2008). Putting evidence into context: some advice for guideline writers. Evid. Based Nurs. 11: 6-8 [Full text]
Garg, A. X., Hackam, D., Tonelli, M. (2008). Systematic Review and Meta-analysis: When One Study Is Just not Enough. CJASN 3: 253-260 [Full text]
van der Jagt, M., Koudstaal, P. J., Dippel, D. W.J., Habbema, J. D. F. (2008). Methodological Quality and Publication Bias in Observational Studies on Risk of Rupture of Unruptured Intracranial Aneurysms. Stroke 39: e11-e11 [Full text]
van der Worp, H. B., Sena, E. S., Donnan, G. A., Howells, D. W., Macleod, M. R. (2007). Hypothermia in animal models of acute ischaemic stroke: a systematic review and meta-analysis. Brain 130: 3063-3074 [Abstract] [Full text]
Chou, R., Huffman, L. H. (2007). Nonpharmacologic Therapies for Acute and Chronic Low Back Pain: A Review of the Evidence for an American Pain Society/American College of Physicians Clinical Practice Guideline. ANN INTERN MED 147: 492-504 [Abstract] [Full text]
Chou, R., Huffman, L. H. (2007). Medications for Acute and Chronic Low Back Pain: A Review of the Evidence for an American Pain Society/American College of Physicians Clinical Practice Guideline. ANN INTERN MED 147: 505-514 [Abstract] [Full text]
Dartnell, J., Hemming, M., Collier, J., Ollenschlaeger, G. (2007). Putting evidence into context: some advice for guideline writers. Evid. Based Med. 12: 130-132 [Full text]
Shang, A., Huwiler, K., Nartey, L., Juni, P., Egger, M. (2007). Placebo-controlled trials of Chinese herbal medicine and conventional medicine comparative study. Int J Epidemiol 36: 1086-1092 [Abstract] [Full text]
SINGH, S. P., GREENWOOD, N., WHITE, S., CHURCHILL, R. (2007). Ethnicity and the Mental Health Act 1983. Br. J. Psychiatry 191: 99-105 [Abstract] [Full text]
Vanhoenacker, P. K., Heijenbrok-Kal, M. H., Van Heste, R., Decramer, I., Van Hoe, L. R., Wijns, W., Hunink, M. G. M. (2007). Diagnostic Performance of Multidetector CT Angiography for Assessment of Coronary Artery Disease: Meta-analysis. Radiology 244: 419-428 [Abstract] [Full text]
Walter, M. A, Briel, M., Christ-Crain, M., Bonnema, S. J, Connell, J., Cooper, D. S, Bucher, H. C, Muller-Brand, J., Muller, B. (2007). Effects of antithyroid drugs on radioiodine treatment: systematic review and meta-analysis of randomised controlled trials. BMJ 334: 514-514 [Abstract] [Full text]
Dentali, F., Douketis, J. D., Gianni, M., Lim, W., Crowther, M. A. (2007). Meta-analysis: Anticoagulant Prophylaxis to Prevent Symptomatic Venous Thromboembolism in Hospitalized Medical Patients. ANN INTERN MED 146: 278-288 [Abstract] [Full text]
Reider, B. (2007). Announcing a Prize Competition: Reviewing Systematically. Am J Sports Med 35: 187-188 [Full text]
Sena, E., Wheble, P., Sandercock, P., Macleod, M. (2007). Systematic Review and Meta-Analysis of the Efficacy of Tirilazad in Experimental Stroke. Stroke 38: 388-394 [Abstract] [Full text]
Krupnick, A. (2007). Mortality-risk Valuation and Age: Stated Preference Evidence. Rev Environ Econ Policy 1: 261-282 [Abstract] [Full text]
Appleton, K. M, Hayward, R. C, Gunnell, D., Peters, T. J, Rogers, P. J, Kessler, D., Ness, A. R (2006). Effects of n-3 long-chain polyunsaturated fatty acids on depressed mood: systematic review of published trials. Am. J. Clin. Nutr. 84: 1308-1316 [Abstract] [Full text]
Nordmann, A. J., Briel, M., Bucher, H. C. (2006). Mortality in randomized controlled trials comparing drug-eluting vs. bare metal stents in coronary artery disease: a meta-analysis. Eur Heart J 27: 2784-2814 [Abstract] [Full text]
Bolton, C. M., Myles, P. S., Nolan, T., Sterne, J. A. (2006). Prophylaxis of postoperative vomiting in children undergoing tonsillectomy: a systematic review and meta-analysis. Br J Anaesth 97: 593-604 [Abstract] [Full text]
He, F. J., MacGregor, G. A. (2006). Importance of Salt in Determining Blood Pressure in Children: Meta-Analysis of Controlled Trials. Hypertension 48: 861-869 [Abstract] [Full text]
Etter, J.-F., Stapleton, J. A (2006). Nicotine replacement therapy for long-term smoking cessation: a meta-analysis.. Tobacco Control 15: 280-285 [Abstract] [Full text]
Latour-Perez, J., Coves-Orts, F. J., Abad-Terrado, C., Abraira, V., Zamora, J. (2006). Accuracy of B-type natriuretic peptide levels in the diagnosis of left ventricular dysfunction and heart failure: A systematic review. Eur J Heart Fail 8: 390-399 [Abstract] [Full text]
Delaney, B. (2006). Is society losing control of the medical research agenda?. BMJ 332: 1063-1064 [Full text]
Briel, M., Schwartz, G. G., Thompson, P. L., de Lemos, J. A., Blazing, M. A., van Es, G.-A., Kayikcioglu, M., Arntz, H.-R., den Hartog, F. R., Veeger, N. J. G. M., Colivicchi, F., Dupuis, J., Okazaki, S., Wright, R. S., Bucher, H. C., Nordmann, A. J. (2006). Effects of early treatment with statins on short-term clinical outcomes in acute coronary syndromes: a meta-analysis of randomized controlled trials.. JAMA 295: 2046-2056 [Abstract] [Full text]
Pavia, M., Pileggi, C., Nobile, C. G., Angelillo, I. F (2006). Association between fruit and vegetable consumption and oral cancer: a meta-analysis of observational studies. Am. J. Clin. Nutr. 83: 1126-1134 [Abstract] [Full text]
Dentali, F., Crowther, M., Ageno, W. (2006). Thrombophilic abnormalities, oral contraceptives, and risk of cerebral vein thrombosis: a meta-analysis. Blood 107: 2766-2773 [Abstract] [Full text]
Delgado-Rodriguez, M. (2006). Systematic reviews of meta-analyses: applications and limitations. J. Epidemiol. Community Health 60: 90-92 [Full text]
Minelli, C., Thompson, J. R, Abrams, K. R, Thakkinstian, A., Attia, J. (2005). The choice of a genetic model in the meta-analysis of molecular association studies. Int J Epidemiol 34: 1319-1328 [Abstract] [Full text]
Zintzaras, E., Voulgarelis, M., Moutsopoulos, H. M. (2005). The Risk of Lymphoma Development in Autoimmune Diseases: A Meta-analysis. Arch Intern Med 165: 2337-2344 [Abstract] [Full text]
Herrington, D. (2005). Eliminating the Improbable: Sherlock Holmes and Standards of Evidence in the Genomic Age. Circulation 112: 2081-2084 [Full text]
Guo, S.-W. (2005). Glutathione S-transferases M1/T1 gene polymorphisms and endometriosis: a meta-analysis of genetic association studies. Mol Hum Reprod 11: 729-743 [Abstract] [Full text]
Terasawa, T., Blackmore, C. C. (2005). Computed Tomography and Ultrasonography To Detect Appendicitis. ANN INTERN MED 142: 800-800 [Full text]
Hemels, M. E., Einarson, A., Koren, G., Lanctot, K. L, Einarson, T. R (2005). Antidepressant Use During Pregnancy and the Rates of Spontaneous Abortions: A Meta-Analysis. The Annals of Pharmacotherapy 39: 803-809 [Abstract] [Full text]
Arrich, J., Piribauer, F., Mad, P., Schmid, D., Klaushofer, K., Mullner, M. (2005). Intra-articular hyaluronic acid for the treatment of osteoarthritis of the knee: systematic review and meta-analysis. CMAJ 172: 1039-1043 [Abstract] [Full text]
Hansen, M., Bower, C., Milne, E., de Klerk, N., J.Kurinczuk, J. (2005). Assisted reproductive technologies and the risk of birth defects--a systematic review. Hum Reprod 20: 328-338 [Abstract] [Full text]
LINDE, K., BERNER, M., EGGER, M., MULROW, C. (2005). St John's wort for depression: Meta-analysis of randomised controlled trials. Br. J. Psychiatry 186: 99-107 [Abstract] [Full text]
Minelli, C., Thompson, J. R., Tobin, M. D., Abrams, K. R. (2004). An Integrated Approach to the Meta-Analysis of Genetic Association Studies using Mendelian Randomization. Am J Epidemiol 160: 445-452 [Abstract] [Full text]
Arterburn, D. E., Crane, P. K., Veenstra, D. L. (2004). The Efficacy and Safety of Sibutramine for Weight Loss: A Systematic Review. Arch Intern Med 164: 994-1003 [Abstract] [Full text]
Rogers, W A (2004). Evidence based medicine and justice: a framework for looking at the impact of EBM upon vulnerable or disadvantaged groups. J. Med. Ethics 30: 141-145 [Abstract] [Full text]
Lawes, C. M.M., Bennett, D. A., Feigin, V. L., Rodgers, A. (2004). Blood Pressure and Stroke: An Overview of Published Reviews. Stroke 35: 1024-1033 [Abstract] [Full text]
Lawes, C. M.M., Bennett, D. A., Feigin, V. L., Rodgers, A. (2004). Blood Pressure and Stroke: An Overview of Published Reviews. Stroke 35: 776-785 [Abstract] [Full text]
Yazdanpanah, Y., Sissoko, D., Egger, M., Mouton, Y., Zwahlen, M., Chene, G. (2004). Clinical efficacy of antiretroviral combination therapy based on protease inhibitors or non-nucleoside analogue reverse transcriptase inhibitors: indirect comparison of controlled trials. BMJ 328: 249- [Abstract] [Full text]
Bottomley, A., Efficace, F., Thomas, R., Vanvoorden, V., Ahmedzai, S. H. (2003). Health-Related Quality of Life in Non-Small-Cell Lung Cancer: Methodologic Issues in Randomized Controlled Trials. JCO 21: 2982-2992 [Abstract] [Full text]
Bigby, M., Williams, H. (2003). Appraising Systematic Reviews and Meta-analyses. Arch Dermatol 139: 795-798 [Abstract] [Full text]
Elwy, A. R., Hart, G. J., Hawkes, S., Petticrew, M. (2002). Effectiveness of Interventions to Prevent Sexually Transmitted Infections and Human Immunodeficiency Virus in Heterosexual Men: A Systematic Review. Arch Intern Med 162: 1818-1830 [Abstract] [Full text]
Vandenbroucke, J. P. (2002). Alternative treatments in reproductive medicine: The vexing problem of `seemingly impeccable trials....'. Hum Reprod 17: 2228-2229 [Abstract] [Full text]
Egger, M., Ebrahim, S., Smith, G. D. (2002). Where now for meta-analysis?. Int J Epidemiol 31: 1-5 [Full text]