Analysis

An open letter to The BMJ editors on qualitative research

BMJ 2016; 352 doi: https://doi.org/10.1136/bmj.i563 (Published 10 February 2016) Cite this as: BMJ 2016;352:i563
  1. Trisha Greenhalgh, professor of primary care health sciences, Nuffield Department of Primary Care Health Sciences, University of Oxford, UK,
  2. Ellen Annandale, professor, Sociology, University of York, UK,
  3. Richard Ashcroft, professor of bioethics, Queen Mary University London, UK,
  4. James Barlow, professor of technology and innovation management–healthcare, Imperial College Business School, UK,
  5. Nick Black, professor of health services research, London School of Hygiene and Tropical Medicine, UK,
  6. Alan Bleakley, emeritus professor of medical education, University of Plymouth, UK,
  7. Ruth Boaden, professor of service operations management, Manchester Business School, UK ,
  8. Jeffrey Braithwaite, professor of health systems research, Australian Institute of Health Innovation, Sydney, Australia,
  9. Nicky Britten, professor of applied healthcare research, University of Exeter Medical School, UK,
  10. Franco Carnevale, professor, Ingram School of Nursing, McGill University, Canada,
  11. Kath Checkland, professor of health policy and primary care, Centre for Primary Care, University of Manchester, UK,
  12. Julianne Cheek, professor, Faculty of Business, Languages and Social Sciences, Ostfold University College, Norway,
  13. Alex Clark, professsor, Faculty of Nursing, University of Alberta, Canada,
  14. Simon Cohn, reader in anthropology, London School of Hygiene and Tropical Medicine, UK,
  15. Jack Coulehan, professor emeritus, Department of Preventative Medicine, Stony Brook University, NY, USA,
  16. Benjamin Crabtree, professor, Department of Family Medicine and Community Health, Rutgers University, NJ, USA,
  17. Steven Cummins, professor of population health, London School of Hygiene and Tropical Medicine, UK,
  18. Frank Davidoff, executive editor, Institute for Healthcare Improvement, Cambridge, MA, USA,
  19. Huw Davies, professor of healthcare policy and management, University of St Andrews, UK,
  20. Robert Dingwall, professor of sociology, Nottingham Trent University, UK,
  21. Mary Dixon-Woods, professor of medical sociology, Department of Health Sciences, University of Leicester, UK,
  22. Glyn Elwyn, professor, Dartmouth Institute for Health Policy and Clinical Practice, Dartmouth, NH, USA,
  23. Eivind Engebretsen, professor, Institute for Health and Society, University of Oslo, Norway,
  24. Ewan Ferlie, professor of public services management, Kings College London, UK,
  25. Naomi Fulop, professor of healthcare organisation and management, University College London, UK,
  26. John Gabbay, emeritus professor of public health, University of Southampton, UK,
  27. Marie-Pierre Gagnon, professor, Faculty of Nursing, Université Laval, Quebec, Canda,
  28. Dariusz Galasinski, professor of discourse and cultural studies, University of Wolverhampton, UK,
  29. Ruth Garside, senior lecturer in evidence synthesis, University of Exeter, UK,
  30. Lucy Gilson, professor of health policy and systems, University of Cape Town, South Africa,
  31. Peter Griffiths, professor of health services research, University of Southampton, UK,
  32. Penny Hawe, professor of public health, University of Sydney, Australia,
  33. Jan-Kees Helderman, associate professor of public administration, Radboud University Nijmegen, Netherlands,
  34. Brian Hodges, professor, Faculty of Medicine, University of Toronto, Canada,
  35. David Hunter, director, Centre for Public Policy and Health, Durham University, UK,
  36. Margaret Kearney, professor, University of Rochester, NY, USA,
  37. Celia Kitzinger, codirector, Coma and Disorders of Consciousness Research Centre, University of York, UK,
  38. Jenny Kitzinger, codirector, Coma and Disorders of Consciousness Research Centre, University of Cardiff, UK,
  39. Ayelet Kuper, assistant professor, Department of Medicine, University of Toronto, Canada,
  40. Saville Kushner, professor of public education, University of Auckland, New Zealand,
  41. Andree Le May, emerita professor of nursing, University of Southampton, UK,
  42. France Legare, Canada research chair in implementation of shared decision making in primary care, University of Laval, Canada,
  43. Lorelei Lingard, professor, Schulich School of Medicine and Dentistry, University of Western Ontario, Canada,
  44. Louise Locock, director of applied research, Health Experiences Research Group, University of Oxford, UK,
  45. Jill Maben, professor of nursing, King’s College London, UK,
  46. Mary Ellen Macdonald, associate professor, Faculty of Dentistry, McGill University, Canada,
  47. Frances Mair, professor of primary care research, University of Glasgow, UK,
  48. Russell Mannion, professor of health systems, University of Birmingham, UK,
  49. Martin Marshall, professor of healthcare improvement, University College London, UK ,
  50. Carl May, professor of healthcare innovation, Faculty of Health Sciences, University of Southampton, UK,
  51. Nicholas Mays, professor of health policy, London School of Hygiene and Tropical Medicine, UK ,
  52. Lorna McKee, professor of management, University of Aberdeen, UK,
  53. Marissa Miraldo, associate professor of health economics, Imperial College London, UK,
  54. David Morgan, professor, Department of Sociology, Portland State University, OR, USA,
  55. Janice Morse, professor, College of Nursing, University of Utah, UT, USA,
  56. Sarah Nettleton, professor of sociology, University of York, UK,
  57. Sandy Oliver, deputy director, EPPI-Centre, UCL Institute of Education, UK,
  58. Warrren Pearce, Institute for Science and Society, University of Nottingham, UK,
  59. Pierre Pluye, director, Methodological Developments, Department of Family Medicine, McGill University, Montreal, Canada,
  60. Catherine Pope, professor of medical sociology, University of Southampton, UK,
  61. Glenn Robert, professor of healthcare quality and innovation, King’s College London, UK,
  62. Celia Roberts, emerita professor of linguistics, King’s College London, UK,
  63. Stefania Rodella, Regional Agency for Health and Social Care, Bologna, Italy,
  64. Jo Rycroft-Malone, professor of implementation, University of Bangor, UK,
  65. Margarete Sandelowski, professor, School of Nursing, University of North Carolina at Chapel Hill, NC, USA,
  66. Paul Shekelle, director, Southern California Evidence-Based Practice Center, RAND Corporation, CA, USA,
  67. Fiona Stevenson, senior lecturer in medical sociology, University College London, UK,
  68. Sharon Straus, director, Division of Medicine, University of Toronto, Canada,
  69. Deborah Swinglehurst, senior clinical lecturer in primary healthcare, Queen Mary University of London, UK,
  70. Sally Thorne, professor, University of British Columbia School of Nursing, Vancouver, Canada,
  71. Göran Tomson, senior professor in international health systems research, Karolinska Institutet, Stockholm Sweden,
  72. Gerd Westert, professor of health services research and quality of care, Scientific Institute for Quality of Care, Nijmegen, Netherlands,
  73. Sue Wilkinson, honorary professor, Department of Sociology, University of York, UK,
  74. Brian Williams, dean of reseach enhancement, University of Stirling, UK,
  75. Terry Young, associate dean of health partnerships, Brunel University, UK,
  76. Sue Ziebland, director, Health Experiences Research Group, University of Oxford, UK
  1. Correspondence to: Trisha Greenhalgh trish.greenhalgh{at}phc.ox.ac.uk
  • Accepted 30 December 2015

Seventy six senior academics from 11 countries invite The BMJ’s editors to reconsider their policy of rejecting qualitative research on the grounds of low priority. They challenge the journal to develop a proactive, scholarly, and pluralist approach to research that aligns with its stated mission

We are concerned that The BMJ seems to have developed a policy of rejecting qualitative research on the grounds that such studies are “low priority,” “unlikely to be highly cited,” “lacking practical value,” or “not of interest to our readers” (box). Here, we argue that The BMJ should develop and publish a formal policy on qualitative and mixed method research and that this should include appropriate and explicit criteria for judging the relevance of submissions. We acknowledge that (as with all methods) some qualitative research is poor quality, badly written, inaccessible, or irrelevant to the journal’s readership. We also acknowledge that many of The BMJ’s readers (not to mention its reviewers and editors) may not have been formally trained to read, conduct, or evaluate qualitative studies. We see these caveats as opportunities not threats.

Excerpt from rejection letter tweeted by McGill Qualitative Health Research Group (@MQHRG), 30 September 2015

Thank you for sending us your paper. We read it with interest but I am sorry to say that qualitative studies are an extremely low priority for The BMJ. Our research shows that they are not as widely accessed, downloaded, or cited as other research.

We receive over 8000 submissions a year and accept less than 4%. We do therefore have to make hard decisions on just how interesting an article will be to our general clinical readers, how much it adds, and how much practical value it will be.

The BMJ’s mission is method agnostic

The BMJ says its mission is to lead the debate on health and to engage, inform, and stimulate all doctors, researchers, and other health professionals in ways that enable them to make better decisions and improve outcomes for patients.

Some clinical and policy questions are best answered by the results of randomised controlled trials or other quantitative approaches, but other decisions and outcomes are more usefully informed by qualitative studies. Qualitative studies help us understand why promising clinical interventions do not always work in the real world, how patients experience care, and how practitioners think. They also explore and explain the complex relations between the healthcare system and the outside world, such as the sociopolitical context in which healthcare is regulated, funded, and provided, and the ways in which clinicians and regulators interact with industry.

Some of The BMJ’s top papers have been qualitative

The BMJ recently celebrated 20 years of online presence by asking experts to name the most influential paper published in that period.1 The 20 nominated papers included 11 commentaries or editorials (highlighting the journal’s important role in publishing papers that contextualise and interpret research), three randomised controlled trials, three qualitative studies, two surveys, and one methodological paper.

The three qualitative papers explored how primary care clinicians develop and use collective “mindlines” instead of written guidelines2; what worries parents when their preschool children are acutely ill3; and the nature of collusion in the doctor-patient relationship when death is imminent.4 They have been cited by 572, 197, and 114 subsequent papers respectively (Google Scholar data). In contrast, the three nominated randomised trials have been cited by 321,5 78,6 and 387 subsequent papers.

We are not claiming that citation rates for these nominated papers are statistically representative. But they do show that good qualitative research with a clear and important clinical message can be highly cited, is popular with readers, and enriches The BMJ’s overall contribution to the knowledge base.

Different study designs provide complementary perspectives

Few research topics in clinical decision making and patient care can be sufficiently understood through quantitative research alone. Take patient safety, for example, in which quantitative studies have examined the effect size of interventions to improve safety and qualitative ones have examined equally important questions such as why the observed effect occurred and, in some cases, why the predicted effect did not occur.

The surgical safety checklist is a revealing case in point. A controlled before and after study published in the New England Journal of Medicine showed that in 3733 patients having non-cardiac surgery, the introduction of a surgical safety checklist was associated with a highly significant reduction in perioperative mortality (from 1.5% to 0.8%) and complication rate (from 11% to 7%).8

But attempts to replicate these impressive improvements have sometimes failed dramatically.9 10 Eighteen qualitative studies, summarised in a recent qualitative systematic review, help explain why.11 The operating theatre is a complex social space with established hierarchies and routines. Far from being a simple “technical” procedure, the checklist demands new forms of cooperation and communication between surgeons, anaesthetists, and nurses. Depending on a host of contextual factors, safety checks may substantially disrupt team routines and be resented rather than welcomed. When (and to the extent that) the checklist is treated as a tick-box exercise, it will fail to generate benefits and may even lead to harms.

From the policy maker’s perspective, qualitative studies of the professional, organisational, and political context of nationally driven checklist based patient safety initiatives can help explain both successes and failures.12 13

The BMJ has a long tradition of educating its readers about less familiar research methods

Statistics is a closed book to many jobbing clinicians. “Bite sized” methodological commentaries, often linked to exemplar papers published in the research section of The BMJ, have enabled its readers to grasp important concepts such as why continuous variables should not be dichotomised14 or why some apparent improvements are explained by regression to the mean.15 Through the journal’s Statistics Notes and Economics Notes series (of which over 100 have been published in the past 20 years), the quantitative research literacy of its clinician readership has significantly improved.

The BMJ has not yet introduced a comparable ongoing educational approach for qualitative research. It is 20 years since Pope and Mays edited the original BMJ Education and Debate series on qualitative methods, which covered interviews, focus groups, ethnography, case study, and criteria for assessing quality and establishing rigour.16 17 18 19 20 21 22 23 24 Their 2000 paper on how to analyse qualitative data remains The BMJ’s 12th most highly cited paper ever (Web of Science data).24 In 2008, The BMJ published a further series updating and extending the range of qualitative research methodologies and emphasising the importance of theory in interpreting evidence.25 26 27 28 29

An opportunity exists to supplement these popular series on qualitative theory and method with an occasional series of “qualitative notes” accompanying exemplars of empirical studies in qualitative research. Through such a series, the journal’s readership would gain in qualitative research literacy.

New challenges

The inclusion of qualitative research as a mainstream theme will undoubtedly raise new methodological, philosophical, and ethical questions. For example, the laudable principle of data archiving and sharing is supported by some but not all qualitative funding bodies (see, for example, the Qualidata archive, part of the UK Data Service https://discover.ukdataservice.ac.uk/?q=qualidata). A requirement to share data may generate tricky challenges in the trade-off between transparency and informant confidentiality, especially in the digital age when anonymisation of interview data may not be possible.30 31

We offer no simple solutions to such complex issues, but suggest that (as with comparable questions in quantitative research) The BMJ could provide a forum for methodological commentaries or online discussion.

A proposal

We believe it is time for a prospective study to assess whether The BMJ can come to value and be proud of qualitative research as part of its mission to lead the debate on health, inform clinical decision making, and improve outcomes for patients. We challenge The BMJ to allocate one slot a month for one year to a “landmark” qualitative paper along with an accompanying methodological commentary from an international expert. We offer to assist The BMJ to appoint an appropriate team of reviewers, guest editors, and commentators. We can also advise on training to build capacity and confidence of editorial staff to distinguish good from poor qualitative research and identify which of the many submissions it receives holds promise as “qualitative paper of the month.”

Conclusion

As pointed out by its editors in response to an earlier draft of this letter, The BMJ is by no means an outlier in its current policy on qualitative research. Many leading US journals (including JAMA and the New England Journal of Medicine) also consider such research low priority. We believe all such journals would benefit from revisiting their policies.

The BMJ, with its history of supporting qualitative research, is in a unique position to lead the field by ensuring that all types of research relevant to its mission are considered for publication; we believe its reputation as an international academic journal will be strengthened if it does so. Some qualitative papers will be highly cited and contribute directly to the journal’s impact factor. With others, the reputational benefit will be indirect and result from introducing the new ways of thinking that are essential to scientific progress.

Both the International Cochrane Collaboration and the UK Health Technology Assessment Programme, though initially predominantly focused on the quantitative, were persuaded to include qualitative and mixed methods research where appropriate.32 33 The Health Technology Assessment Programme’s monograph on qualitative methods33 subsequently became the most downloaded of its more than 700 online publications by a considerable margin. These organisations have decided that “quantitative versus qualitative” is yesterday’s war. We encourage The BMJ to join them.

Notes

Cite this as: BMJ 2016;352:i563

Footnotes

  • Contributors and sources: TG wrote the first draft of the letter, which was modified by ST, AK, and LLingard and then circulated to all other authors, many of whom suggested further amendments. All authors have seen and approved the final manuscript.

  • Competing interests: We have read and understood BMJ policy on declaration of interests and declare that two of us have received consultancy income from qualitative research and some of us have received royalties for books or book chapters on qualitative research. Our only other conflict of interest is that we value the contribution of qualitative research to medicine.

  • Provenance and peer review: Not commissioned; not externally peer reviewed.

References

View Abstract