How to improve healthcare improvement—an essay by Mary Dixon-WoodsBMJ 2019; 367 doi: https://doi.org/10.1136/bmj.l5514 (Published 01 October 2019) Cite this as: BMJ 2019;367:l5514
Read the full collection
In the NHS, as in health systems worldwide, patients are exposed to risks of avoidable harm 1 and unwarranted variations in quality.234 But too often, problems in the quality and safety of healthcare are merely described, even “admired,”5 rather than fixed; the effort invested in collecting information (which is essential) is not matched by effort in making improvement. The National Confidential Enquiry into Patient Outcome and Death, for example, has raised many of the same concerns in report after report.6 Catastrophic degradations of organisations and units have recurred throughout the history of the NHS, with depressingly similar features each time.789
More resources are clearly necessary to tackle many of these problems. There is no dispute about the preconditions for high quality, safe care: funding, staff, training, buildings, equipment, and other infrastructure. But quality health services depend not just on structures but on processes.10 Optimising the use of available resources requires continuous improvement of healthcare processes and systems.5
The NHS has seen many attempts to stimulate organisations to improve using incentive schemes, ranging from pay for performance (the Quality and Outcomes Framework in primary care, for example) to public reporting (such as annual quality accounts). They have had mixed results, and many have had unintended consequences.1112 Wanting to improve is not the same as knowing how to do it.
In response, attention has increasingly turned to a set of approaches known as quality improvement (QI). Though a definition of exactly what counts as a QI approach has escaped consensus, QI is often identified with a set of techniques adapted from industrial settings. They include the US Institute for Healthcare Improvement’s Model for Improvement, which, among other things, combines measurement with tests of small change (plan-do-study-act cycles).8 Other popular approaches include Lean and Six Sigma. QI can also involve specific interventions intended to improve processes and systems, ranging from checklists and “care bundles” of interventions (a set of evidence based practices intended to be done consistently) through to medicines reconciliation and clinical pathways.
QI has been advocated in healthcare for over 30 years13; policies emphasise the need for QI and QI practice is mandated for many healthcare professionals (including junior doctors). Yet the question, “Does quality improvement actually improve quality?” remains surprisingly difficult to answer.14 The evidence for the benefits of QI is mixed14 and generally of poor quality. It is important to resolve this unsatisfactory situation. That will require doing more to bring together the practice and the study of improvement, using research to improve improvement, and thinking beyond effectiveness when considering the study and practice of improvement.
Uniting practice and study
The practice and study of improvement need closer integration. Though QI programmes and interventions may be just as consequential for patient wellbeing as drugs, devices, and other biomedical interventions, research about improvement has often been seen as unnecessary or discretionary,1516 particularly by some of its more ardent advocates. This is partly because the challenges faced are urgent, and the solutions seem obvious, so just getting on with it seems the right thing to do.
But, as in many other areas of human activity, QI is pervaded by optimism bias. It is particularly affected by the “lovely baby” syndrome, which happens when formal evaluation is eschewed because something looks so good that it is assumed it must work. Five systematic reviews (published 2010-16) reporting on evaluations of Lean and Six Sigma did not identify a single randomised controlled trial.1718192021 A systematic review of redesigning care processes identified no randomised trials.22 A systematic review of the application of plan-do-study-act in healthcare identified no randomised trials.23 A systematic review of several QI methods in surgery identified just one randomised trial.56
The sobering reality is that some well intentioned, initially plausible improvement efforts fail when subjected to more rigorous evaluation.24 For instance, a controlled study of a large, well resourced programme that supported a group of NHS hospitals to implement the IHI’s Model for Improvement found no differences in the rate of improvement between participating and control organisations.2526 Specific interventions may, similarly, not survive the rigours of systematic testing. An example is a programme to reduce hospital admissions from nursing homes that showed promise in a small study in the US,27 but a later randomised implementation trial found no effect on admissions or emergency department attendances.28
Some interventions are probably just not worth the effort and opportunity cost: having nurses wear “do not disturb” tabards during drug rounds, is one example.29 And some QI efforts, perversely, may cause harm—as happened when a multicomponent intervention was found to be associated with an increase rather than a decrease in surgical site infections.30
Producing sound evidence for the effectiveness of improvement interventions and programmes is likely to require a multipronged approach. More large scale trials and other rigorous studies, with embedded qualitative inquiry, should be a priority for research funders.
Not every study of improvement needs to be a randomised trial. One valuable but underused strategy involves wrapping evaluation around initiatives that are happening anyway, especially when it is possible to take advantage of natural experiments or design roll-outs.31 Evaluation of the reorganisation of stroke care in London and Manchester32 and the study of the Matching Michigan programme to reduce central line infections are good examples.3334
It would be impossible to externally evaluate every QI project. Critically important therefore will be increasing the rigour with which QI efforts evaluate themselves, as shown by a recent study of an attempt to improve care of frail older people using a “hospital at home” approach in southwest England.35 This ingeniously designed study found no effect on outcomes and also showed that context matters.
Despite the potential value of high quality evaluation, QI reports are often weak,18 with, for example, interventions so poorly reported that reproducibility is frustrated.36 Recent reporting guidelines may help,37 but some problems are not straightforward to resolve. In particular, current structures for governance and publishing research are not always well suited to QI, including situations where researchers study programmes they have not themselves initiated. Systematic learning from QI needs to improve, which may require fresh thinking about how best to align the goals of practice and study, and to reconcile the needs of different stakeholders.38
Using research to improve improvement
Research can help to support the practice of improvement in many ways other than evaluation of its effectiveness. One important role lies in creating assets that can be used to improve practice, such as ways to visualise data, analytical methods, and validated measures that assess the aspects of care that most matter to patients and staff. This kind of work could, for example, help to reduce the current vast number of quality measures—there are more than 1200 indicators of structure and process in perioperative care alone.39
The study of improvement can also identify how improvement practice can get better. For instance, it has become clear that fidelity to the basic principles of improvement methods is a major problem: plan-do-study-act cycles are crucial to many improvement approaches, yet only 20% of the projects that report using the technique have done so properly.23 Research has also identified problems in measurement—teams trying to do improvement may struggle with definitions, data collection, and interpretation40—indicating that this too requires more investment.
Improvement research is particularly important to help cumulate, synthesise, and scale learning so that practice can move forward without reinventing solutions that already exist or reintroducing things that do not work. Such theorising can be highly practical,41 helping to clarify the mechanisms through which interventions are likely to work, supporting the optimisation of those interventions, and identifying their most appropriate targets.42
Research can systematise learning from “positive deviance,” approaches that examine individuals, teams, or organisations that show exceptionally good performance.43 Positive deviance can be used to identify successful designs for clinical processes that other organisations can apply.44
Crucially, positive deviance can also help to characterise the features of high performing contexts and ensure that the right lessons are learnt. For example, a distinguishing feature of many high performing organisations, including many currently rated as outstanding by the Care Quality Commission, is that they use structured methods of continuous quality improvement. But studies of high performing settings, such as the Southmead maternity unit in Bristol, indicate that although continuous improvement is key to their success, a specific branded improvement method is not necessary.45 This and other work shows that not all improvement needs to involve a well defined QI intervention, and not everything requires a discrete project with formal plan-do-study-act cycles.
More broadly, research has shown that QI is just one contributor to improving quality and safety. Organisations in many industries display similar variations to healthcare organisations, including large and persistent differences in performance and productivity between seemingly similar enterprises.46 Important work, some of it experimental, is beginning to show that it is the quality of their management practices that distinguishes them.47 These practices include continuous quality improvement as well as skills training, human resources, and operational management, for example. QI without the right contextual support is likely to have limited impact.
Important as they are, evaluations of the approaches and interventions in individual improvement programmes cannot answer every pertinent question about improvement.48 Other key questions concern the values and assumptions intrinsic to QI.
Consider the “product dominant” logic in many healthcare improvement efforts, which assumes that one party makes a product and conveys it to a consumer.49 Paul Batalden, one of the early pioneers of QI in healthcare, proposes that we need instead a “service dominant” logic, which assumes that health is co-produced with patients.49
More broadly, we must interrogate how problems of quality and safety are identified, defined, and selected for attention by whom, through which power structures, and with what consequences. Why, for instance, is so much attention given to individual professional behaviour when systems are likely to be a more productive focus?50 Why have quality and safety in mental illness and learning disability received less attention in practice, policy, and research51 despite high morbidity and mortality and evidence of both serious harm and failures of organisational learning? The concern extends to why the topic of social inequities in healthcare improvement has remained so muted52 and to the choice of subjects for study. Why is it, for example, that interventions like education and training, which have important roles in quality and safety and are undertaken at vast scale, are often treated as undeserving of evaluation or research?
How QI is organised institutionally also demands attention. It is often conducted as a highly local, almost artisan activity, with each organisation painstakingly working out its own solution for each problem. Much improvement work is conducted by professionals in training, often in the form of small, time limited projects conducted for accreditation. But working in this isolated way means a lack of critical mass to support the right kinds of expertise, such as the technical skill in human factors or ergonomics necessary to engineer a process or devise a safety solution. Having hundreds of organisations all trying to do their own thing also means much waste, and the absence of harmonisation across basic processes introduces inefficiencies and risks.14
A better approach to the interorganisational nature of health service provision requires solving the “problem of many hands.”53 We need ways to agree which kinds of sector-wide challenges need standardisation and interoperability; which solutions can be left to local customisation at implementation; and which should be developed entirely locally.14 Better development of solutions and interventions is likely to require more use of prototyping, modelling and simulation, and testing in different scenarios and under different conditions,14 ideally through coordinated, large scale efforts that incorporate high quality evaluation.
Finally, an approach that goes beyond effectiveness can also help in recognising the essential role of the professions in healthcare improvement. The past half century has seen a dramatic redefining of the role and status of the healthcare professions in health systems54: unprecedented external accountability, oversight, and surveillance are now the norm. But policy makers would do well to recognise how much more can be achieved through professional coalitions of the willing than through too many imposed, compliance focused diktats. Research is now showing how the professions can be hugely important institutional forces for good.5455 In particular, the professions have a unique and invaluable role in working as advocates for improvement, creating alliances with patients, providing training and education, contributing expertise and wisdom, coordinating improvement efforts, and giving political voice for problems that need to be solved at system level (such as, for example, equipment design).
Improvement efforts are critical to securing the future of the NHS. But they need an evidence base. Without sound evaluation, patients may be deprived of benefit, resources and energy may be wasted on ineffective QI interventions or on interventions that distribute risks unfairly, and organisations are left unable to make good decisions about trade-offs given their many competing priorities. The study of improvement has an important role in developing an evidence-base and in exploring questions beyond effectiveness alone, and in particular showing the need to establish improvement as a collective endeavour that can benefit from professional leadership.
Mary Dixon-Woods is the Health Foundation professor of healthcare improvement studies and director of The Healthcare Improvement Studies (THIS) Institute at the University of Cambridge, funded by the Health Foundation. Co-editor-in-chief of BMJ Quality and Safety, she is an honorary fellow of the Royal College of General Practitioners and the Royal College of Physicians. This article is based largely on the Harveian oration she gave at the RCP on 18 October 2018, in the year of the college’s 500th anniversary. The oration is available here: http://www.clinmed.rcpjournal.org/content/19/1/47 and the video version here: https://www.rcplondon.ac.uk/events/harveian-oration-and-dinner-2018
This article is one of a series commissioned by The BMJ based on ideas generated by a joint editorial group with members from the Health Foundation and The BMJ, including a patient/carer. The BMJ retained full editorial control over external peer review, editing, and publication. Open access fees and The BMJ’s quality improvement editor post are funded by the Health Foundation.
Competing interests: I have read and understood BMJ policy on declaration of interests and a statement is available here: https://www.bmj.com/about-bmj/advisory-panels/editorial-advisory-board/mary-dixonwoods
Provenance and peer review: Commissioned; not externally peer reviewed.
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.