Making the best use of administrative data

BMJ 2013; 346 doi: https://doi.org/10.1136/bmj.f1284 (Published 27 February 2013) Cite this as: BMJ 2013;346:f1284
  1. Adam Steventon, senior research analyst
  1. 1Nuffield Trust, London W1G 7LP, UK
  1. adam.steventon{at}nuffieldtrust.org.uk

The difficulty of teasing out demand and supply

Making better use of routine administrative data is becoming an ever more integral part of delivering higher quality more efficient healthcare. Routine data are useful when evaluating complex interventions related to the management of patients with long term conditions, which need continuous monitoring and refinement.1 Other applications of routine data are predictive models, which have been developed to identify patients at high risk of future adverse events.2 Finally, payment mechanisms almost inevitably use (and generate) routine data, and routine data are the basis for many performance indicators.3

Problems with routine data are well known. They often do not tell us everything we need to know (for example, wider determinants of health), they rarely capture the direct views of patients and do not always correlate well with what patients tell us,4 they vary in quality and depth of data, and they can be “gamed.”5 Furthermore, as pointed out in a linked paper by Wennberg and colleagues (doi:10.1136/bmj.f549), it can be challenging to separate out healthcare need from supply in analyses of routine data.6 This problem of judging needs through the lens of diagnosis markers in routine data can be described simply. If measures of healthcare needs are built up from the diagnoses recorded in hospital datasets, a diagnosis can be present only if the patient has been to the hospital. The propensity to admit coupled with the intensity with which doctors observe and diagnose patients differs between areas, leading to a distorted picture of healthcare needs. This is not a new problem but has been at the heart of debates about funding for many years.7

The phenomenon has implications for resource allocation in capitated systems, where purchasers of healthcare allocate funding on the basis of the estimated needs of each enrolled person. Such a system is used by budget holding clinical commissioning groups in England, health insurance plans in the Netherlands, and Medicare Advantage plans in the United States, among others. In these systems, stripping out the effect of supply from estimates of need is important, if payments are to reflect need rather than patterns of service use (because of the potential to reward areas with modest level of need but inappropriately high levels of service use in the past). This problem, just like other problems with routine data, is well known, and there are ways to try to deal with it. For example, in the person based resource allocation (PBRA) formula that guides resource allocation for general practices in England, the effects of supply variables at the level of the individual were “frozen out” of the eventual allocations.8

Wennberg and colleagues assert that measures of healthcare need lack validity if they do not explain variations in age, sex, and ethnicity adjusted mortality rates between regions. When they tested three common methods of estimating need from diagnosis fields in Medicare claims data, they found that they explained only 10-12% of the variation at the region level. This approach assumes that needs are reflected in mortality rates, which brings us to the thorny question of what we mean by “need.” A formula for funding hospital care like PBRA should presumably reflect those elements of need that can be dealt with by hospital care, and we know that patterns of mortality are influenced by many factors outside the hospital.9

Nevertheless, researchers should consider the novel approach proposed by Wennberg and colleagues to separate the effect of supply from need. This was based on taking the number of times patients were seen by doctors in their last six months of life as a proxy for observation intensity in any one region. When they adjusted their estimates of need for this quantity, the proportion of mortality explained jumped to 21-24%. This is an interesting approach, but the proxy cannot be a perfect measure of observation intensity, because patients in the last six months of life will have varying levels of need and variable supply of alternatives to Medicare. Furthermore, the configuration of palliative care may not reflect how care services are delivered for other population groups.

The challenges of teasing out demand and supply mean that we need to test the value of this approach in other datasets. The approach also needs to be compared with existing methods, such as the “freezing” method used by PBRA, to see which is best at reducing bias in the estimates. Furthermore, validation requires more criteria, including those measured at different levels (such as the level of the organisations to which capitated payments are made).

Meanwhile, these challenges are not a reason to stop using routine data. All datasets have their drawbacks, and resources need to be allocated somehow. As Wennberg and colleagues suggest, we can get smarter in the way that we use the data. Ultimately, though, fancier statistical methods won’t solve the problem of datasets not telling us everything we need to know. The solution should involve putting more effort into collecting outcomes data from patients routinely, as well as safe data linkage.10 11 The United Kingdom has the huge advantage of well established computerised general practice records, which, given appropriate safeguards, can help shape the ways we consider population health in future, at least for those registered with general practitioners.


Cite this as: BMJ 2013;346:f1284


  • Research, doi:10.1136/bmj.f549
  • Competing interests: I have read and understood the BMJ Group policy on declaration of interests and have no interests to declare.

  • Provenance and peer review: Commissioned; not externally peer reviewed.


View Abstract

Sign in

Log in through your institution

Free trial

Register for a free trial to thebmj.com to receive unlimited access to all content on thebmj.com for 14 days.
Sign up for a free trial