Editor's Choice

Careful what you measure

BMJ 2007; 335 doi: https://doi.org/10.1136/bmj.39406.590822.47 (Published 22 November 2007) Cite this as: BMJ 2007;335:0
  1. Fiona Godlee, editor
  1. fgodlee{at}bmj.com

    Back in the 1920s, workers at the Hawthorne electrical factory in Cicero, Illinois, were given better lighting and their productivity improved. Further investigation found that the same improvement could be achieved by almost any change to their environment. One version of what has since become known as the Hawthorne effect says that the very act of measuring something changes what people do. It's one reason why we must choose wisely what we measure in health care.

    This week, in the fourth article in our series on performance measurement, Iona Heath and colleagues add a plea for more clinically meaningful measures (doi: 10.1136/bmj.39377.387373.AD). Focusing on process rather than clinical outcomes reduces clinical complexity to a series of boxes for ticking and encourages overtreatment and medicalisation, they say. Doctors are undermined and patients' individual needs are ignored. “Authentic dialogue between doctor and patient is disrupted and many doctors feel fundamentally compromised.” In a similar vein, a few weeks ago, Bruce Guthrie and colleagues argued that measuring performance in terms of whether patients' risk factors were recorded, rather than whether the patients were adequately treated, encourages therapeutic inertia (BMJ 2007;335:542-4; doi: 10.1136/bmj.39259.400069.AD).

    Given the complexity of health care, what are the chances of coming up with a single overall measure of performance? Slim, I fear, especially after reading the study by Ira Wilson and colleagues (doi: 10.1136/bmj.39364.520278.55). When HIV services in the United States were evaluated with a bundle of eight clinical measures, few of them scored highly across more than a handful of measures. Providing uniformly high quality of care is hard if not impossible, even within a dedicated service; so, people prioritise. This means that performance on one measure may tell you little about performance on others.

    The Hawthorne effect is temporary. After a while, productivity in the Illinois factory returned to normal until another change in the environment was made. Bruce Agins and Marc Holden argue that sustained improvements will come only if we have the proper infrastructure (doi: 10.1136/bmj.39359.605752.80). Next week (1 December), Helen Lester and Martin Roland will close our series on performance measurement, saying that performance can continue to improve only if we change the things we measure over time. They propose rotation of measures to encourage improvement across a range of conditions and areas.

    There can be little doubt that we must constantly evaluate how we are doing, against each other and over time. The problem is that the things that are easiest to measure are almost inevitably the least important, and vice versa. Responding to last week's “harrowing” account of the inhumane treatment of an elderly woman with mental health problems (BMJ 2007;335:994; doi: 10.1136/bmj.39378.483414.3A), David Oliver calls for performance pressures to be applied to basic care and communication, in the same way as they are applied to access times and balancing the books (doi: 10.1136/bmj.39402.362234.BE). Compassion and dignity are hard to measure. But if these are the things that really matter in medicine—and they should be—we must find ways to measure them.

    View Abstract