Correcting the scientific record on abortion and mental health outcomes
BMJ 2024; 384 doi: https://doi.org/10.1136/bmj-2023-076518 (Published 27 February 2024) Cite this as: BMJ 2024;384:e076518
- Julia H Littell, professor emerita1,
- Kathryn M Abel, professor2,
- M Antonia Biggs, associate professor3,
- Robert W Blum, professor4,
- Diana Greene Foster, professor3,
- Lisa B Haddad, medical director5,
- Brenda Major, distinguished professor6,
- Trine Munk-Olsen, professor78,
- Chelsea B Polis, senior scientist59,
- Gail Erlick Robinson, professor10,
- Corinne H Rocca, professor3,
- Nancy Felipe Russo, regents professor emerita11,
- Julia R Steinberg, associate professor12,
- Donna E Stewart, university professor1314,
- Nada Logan Stotland, professor15,
- Ushma D Upadhyay, professor3,
- Jenneke van Ditzhuijzen, assistant professor1617
- 1Graduate School of Social Work and Social Research, Bryn Mawr College, Bryn Mawr, PA, USA
- 2Centre for Women’s Mental Health, Faculty of Biology, Medicine and Health Sciences, University of Manchester, Manchester, UK
- 3Advancing New Standards in Reproductive Health, Department of Obstetrics, Gynecology, and Reproductive Sciences, School of Medicine, University of California San Francisco, Oakland, CA, USA
- 4Department of Population, Family and Reproductive Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA
- 5Center for Biomedical Research, Population Council, New York, NY, USA
- 6Department of Psychological and Brain Sciences, University of California Santa Barbara, USA
- 7Department of Clinical Research (Research Unit for Child and Adolescent Psychiatry), University of Southern Denmark, Odense, Denmark
- 8The National Center for Register-based Research, Aarhus University, Aarhus, Denmark
- 9Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA
- 10Department of Psychiatry, University of Toronto, Toronto, Canada
- 11Department of Psychology, Arizona State University, Tempe, AZ, USA
- 12Department of Family Science, School of Public Health, University of Maryland, College Park, MD, USA
- 13Departments of Psychiatry, Obstetrics and Gynecology, Medicine, Surgery, Anesthesiology, Family and Community Medicine, University of Toronto, Toronto, Canada
- 14Toronto General Hospital Research Institute, Toronto, Canada
- 15Department of Psychiatry and Behavioral Sciences, Rush Medical Center, Rush University, Chicago, IL, USA
- 16Interdisciplinary Social Science: Social Policy and Public Health, Utrecht University, Utrecht, Netherlands
- 17Department of Obstetrics and Gynaecology, Amsterdam University Medical Center, Amsterdam, Netherlands
- Correspondence to: J H Littell jlittell{at}brynmawr.edu
The scientific community has contended with unreliable research for decades.12 Ideally, fatally flawed studies will be detected by peer reviewers and rejected by journal editors, but these processes are subjective, varied, and susceptible to error.34 Post-publication critiques can identify important flaws, and public debate can shed light on the nature of these problems, but these actions do not correct the scientific record. Thus, when authors are unwilling or unable to make sufficient corrections, published papers occasionally have to be retracted to correct the scientific record.
Standards for correction and retraction have been established by the Committee on Publication Ethics (COPE),56 the International Committee of Medical Journal Editors (ICMJE),7 and editors’ professional associations, including the Council of Science Editors, World Association of Medical Editors, and European Association of Science Editors. These organisations provide guidance on best practices for science editors and journals. However, problematic science continues to remain uncorrected. The importance of correcting the scientific record has been highlighted by the use of four unreliable studies on the mental health outcomes of abortion in US court cases that have restricted access to abortion.
Four abortion studies
In 2002, The BMJ published a cohort study concluding that the risk of depression was higher in women who had an abortion compared with those who continued an unwanted first pregnancy.8 Independent re-analysis of the same data found that this study incorrectly identified unwanted first pregnancies and did not control for pre-pregnancy levels of depression, and after correction of these errors abortion was not associated with increased risk of subsequent maternal depression.9The BMJ published criticism of the original article10 and partial corrections,11 but serious methodological problems remain uncorrected (including failure to control for previous depression).
In 2005, the Journal of Anxiety Disorders published a cohort study concluding that abortion was associated with higher rates of generalised anxiety than giving birth.12 An independent re-analysis showed that pre-existing levels of anxiety and exposure to domestic violence fully accounted for the apparent association between abortion and subsequent anxiety.13 The original paper has not been corrected or retracted.
In 2009, the Journal of Psychiatric Research published an analysis of the US national comorbidity survey concluding that abortion increased the likelihood of various mental health problems and substance use disorders.14 The article’s results could not be replicated because it contained incorrect statements about its methods1516; instead of using post-pregnancy outcome measures, as described in the methods section, authors used lifetime measures of mental health problems.1617 Associations between lifetime measures of abortion and lifetime measures of mental health disorders were incorrectly interpreted as evidence that abortion independently contributed to mental health problems.1418 Substantial criticisms of this article were published in the journal and elsewhere.1516 A corrigendum by the authors corrected sample weights18 but did not address more fundamental analytical problems.1516 Although the study was “decisively debunked”19 and the editor stated that the authors’ conclusions were not supported by their analyses,1920 the journal rejected a call for retraction and invited additional published debate instead (E Hollander, personal communication).
The errors in these papers were accumulated in 2011 when they were included in a systematic review and meta-analysis published in the British Journal of Psychiatry. This synthesis of data from 22 studies (on 14 separate datasets) concluded that abortion accounts for a substantial increase in the risk of adverse mental health outcomes.21 Eleven of the 22 included studies were authored or co-authored by the sole author of the meta-analysis. The review failed to meet any published methodological criteria for systematic reviews and meta-analyses.22232425262728 It cited but failed to follow recommendations for avoiding statistical dependencies, which can result in over-representation of some studies and biased estimates in meta-analysis.2930 Of the 36 effect sizes included in one meta-analysis, only four were independent.28 The debunked 2009 paper14 was represented three times in one meta-analysis. Further, the pooled odds ratio (1.81) was incorrectly interpreted as a risk ratio, thereby inflating the risk estimate. The formula for the population attributable risk factor was incorrect and its application in this context was misleading.28 Despite numerous substantive and methodological critiques of this article,222324252627283132 the British Journal of Psychiatry rejected calls for its retraction in 2012 and in 2023.
In 2022, the journal’s owner, the Royal College of Psychiatrists, overturned the recommendations of an independent panel that the article should be retracted after the author and her lawyers threatened legal action.333435 It presented three arguments against retraction: the passage of time since initial publication, existence of a published debate, and the fact that the article had been subject to an earlier investigation.36 Five editorial board members resigned after the decision, suggesting interference with the journal’s editorial independence.333435 Serious scientific concerns about the 2011 meta-analysis have still not been adequately addressed.
Real world consequences
All four flawed studies were cited in multiple court cases as evidence that abortion increases the risk of mental health problems.373839404142434445464748495051525354555657 The meta-analysis has been cited in 25 court cases and 14 parliamentary hearings in six countries. The 2009 study was cited in at least 11 court cases and four legislative hearings.
When asked to defend this research against mounting published criticism, authors argued that, because these articles have not been corrected or retracted, they represent sound scientific work and scholarly journals stand behind them.4950 This leaves it to judicial and legislative bodies to determine the veracity of testimony based on these studies. Court decisions have been split: some courts accepted all or portions of this evidence,4657while others ruled it unreliable or inadmissible.4447
Despite its severe limitations, the meta-analysis was submitted as evidence of harmful effects of abortion in documents presented in the US Supreme Court case Dobbs v Jackson Women’s Health Organization,51545556 which culminated in the 2022 ruling that the US Constitution does not confer a right to abortion. In April 2023, it was again cited in a US district court ruling to invalidate FDA approval for the use of mifepristone for medication abortion.5257 Now before the US Supreme Court, the case could severely restrict abortion access in the US.
Publication ethics
Scientific knowledge accumulates in complex social contexts wherein people contest the trustworthiness of empirical evidence and put forward rival plausible explanations for results. We must maintain uncompromising standards of quality and integrity at every stage in the production and dissemination of scientific research to advance scientific knowledge, accurately inform public policy and clinical practice, and protect public health.
Although authors initiate some post-publication corrections and retractions,5859 ultimately editors are responsible for the content they publish. COPE guidelines state that editorial staff “must be committed to correcting the literature when needed.”5 Therefore, when there is clear evidence that published statements are unreliable or invalid, journals should request and publish corrections from authors or post a public expression of concern linked to the original article.56 When these steps are insufficient, authors and editors should move to retraction.5675860
COPE states that retraction of articles is needed when they contain “such seriously flawed or erroneous content or data that their findings and conclusions cannot be relied upon.”6 Editors must therefore make important—albeit difficult—distinctions between inconsequential imperfections in research articles, unreliable evidence or invalid arguments, and evidence of research misconduct. All four articles discussed above include unreliable or invalid content that authors and editors have not yet corrected or retracted.
Increased public scrutiny, better technical tools, and improved editorial oversight have led to rising retraction rates (from 4/10 000 articles in 2010 to 8-10/10 000 in 2020).586162 With the steep growth in number of papers published annually, retractions are also increasing. These trends pose challenges for journals, but many observers believe retractions reflect successful efforts to stamp out bad science.586162 Many journals now have formal policies on retraction,61 yet practices are inconsistent across journals and there is ample room for improvement.585961626364656667
Some editors have avoided retraction, as in the cases above, preferring resolution by academic debate, airing “both sides” without commenting on the methodological rigour or weight of the evidence supporting different conclusions.63646566 Some editors treated authors’ failures to meet basic expectations regarding research integrity as mere differences of opinion or scientific controversies worthy of debate.636467 Some incorrectly assumed that retractions can only be made by authors.65 Others asserted, contrary to COPE and ICMJE guidelines, that retraction requires evidence of fraud or fabrication. Others claimed that the passage of time renders retraction unnecessary36 (there is no statute of limitations for retractions).68 We think these positions miss the mark because the sole purpose of retraction is to correct the scientific record.
Failure to correct fundamental flaws in the scientific foundations of any empirical argument is shirking editorial responsibilities567 and allows inaccurate information to flourish under the publishers’ imprimatur. Responsibilities for scientific integrity and publication ethics must be given the highest priority and must be fully discharged by editors and journals, especially when there are real consequences for patients and public health.
Call to action
The post-publication debates on the four studies we described show that they contain unreliable evidence or invalid conclusions about the mental health outcomes of abortion. We believe that journal editors and their publishers have an ethical obligation to correct the scientific record in these cases. At a minimum, the papers published in The BMJ8 and Journal of Anxiety Disorders12 should be accompanied by published expressions of concern, given their failures to consider alternative plausible explanations.913 We believe that the Journal of Psychiatric Research14 and British Journal of Psychiatry21 papers should both be retracted because of the overwhelming and incontrovertible evidence of their methodological flaws, inaccurate results, and invalid conclusions.15161920222324252627283132
Decisions about retraction should never be compromised by politics, controversy, or legal threats. Scientific controversies are not properly adjudicated by the courts, and journal editors, owners, and publishers must be prepared to face and overcome legal threats, which are likely to be baseless, as the rejection of claims against retraction of a study in Frontiers in Psychology has shown.3569 As others have suggested, journals should consider making legal threats public to discourage further use of this tactic.70
Solutions and support for many of the dilemmas that editors and publishers face are at hand. Closer adherence to current guidelines on publication ethics, corrections, and retraction could reduce lapses in scientific integrity.567 More transparent and uniform approaches to retraction are needed to protect the published record and avoid unfair stigmatisation of authors.65 Further discussion is needed on the processes, priorities, and institutional supports for retraction. Journal owners and publishers must fully support editors in their efforts to maintain the integrity of the scientific record, especially on the rare occasions when editors must seriously consider retraction.6570 Allowing inaccurate information to remain in the scientific record can have lasting and deleterious effects on law, public policy, clinical practice, and public health.
Key messages
Failure to correct or retract unreliable research papers published in medical journals allows misinformation to spread under the journals’ imprimatur
Cumulative lapses in scientific integrity pose ongoing threats to public health and public trust in science
Unreliable evidence and invalid conclusions about the mental health outcomes of abortion was used to inform policies that restrict access to abortion in the US
Scientists, editors, journals, and publishers must ensure that published materials are accurate and must correct or retract articles when necessary to maintain the integrity of science
Acknowledgments
We thank BMJ editors and peer reviewers for insightful and constructive comments on earlier versions of this paper.
Footnotes
Contributors and sources: The authors are researchers, scholars, and physicians with expertise on topics related to public health, reproductive health, mental health, research methodology, systematic review methods, and meta-analysis. Ten contributors have served on editorial boards and six (JHL, BM, TMO, NFR, DES, NLS) have served as editorial assistant, associate editor, editor, or editor-in-chief of one or more scholarly journals. All co-authors contributed to this article and reviewed drafts. JHL is the guarantor.
Competing interests: We have read and understood BMJ policy on declaration of interests and have the following interests to declare: MAB, BM, and JRS have been paid as expert scientists in the area of abortion and mental health in legal cases challenging abortion policies. NLS received reimbursement for travel expenses related to pro bono expert testimony. MAB, RWB, DGF, CBP, CHR, JRS, JVD, and UDU were awarded grants to conduct research or reviews on the relationship between abortion and mental health. LBH is a named plaintiff in a case regarding abortion rights (SisterSong v State of Georgia). NLS served on the board of directors of Physicians for Reproductive Health. TMO received a speaker fee from Lundbeck Pharma A/S for a talk on prediction of postpartum depression. The views expressed here are those of the authors and do not necessarily reflect those of their employers or other organisations.
Provenance and peer review: Not commissioned; externally peer reviewed.