Promoting efficiency in the NHS: problems with the labour productivity index
BMJ 1996; 313 doi: https://doi.org/10.1136/bmj.313.7068.1319 (Published 23 November 1996) Cite this as: BMJ 1996;313:1319
- John Applebya, senior lecturer in health economics
- Accepted 7 August 1996
The introduction of the internal market in the NHS has highlighted a number of issues concerning efficiency—its definition, measurement, and in particular mechanisms for its promotion. Until now the Department of Health's main tool for promoting efficiency has been the much criticised efficiency index. In effect health authorities have been given less money each year in the expectation that they would seek commensurate efficiencies from their providers, and each year the government has set a target for this “efficiency index”: this year's is 3%. Due partly to criticisms of the efficiency index and other performance measures1 2 3 and partly to the need for information to support local pay bargaining, the NHS Executive has recently introduced the labour productivity index, to be used for all NHS trusts in England.5 This paper discusses the possible perverse incentives that this might introduce for trusts and purchasers and the problems they face in interpreting (and hence acting on) this performance measure.
How the index is constructed
The construction of this new index is similar to that of the efficiency index, except that in order to produce a single number for a trust's total activity, each type of activity—inpatients, outpatients, health visitor contacts, etc—is weighted using the national average cost of producing one unit of the various types of activity—a finished consultant episode, a contact, etc. A trust's labour productivity index is then simply derived by multiplying each type of activity by its average cost, adding all these costs together and dividing the total by the number of employees in the trust (see box).
Thus, the labour productivity index (unlike the efficiency index) facilitates comparisons between trusts. The national average cost weights used for 1995–6 (based on 1993–4 data) are shown in table 1. Problems with separately identifying certain types of activity—the work done by health visitors and district nurses, for example—means that the national average costs are somewhat rough and ready. The range of values of the labour productivity index for the first quarter of 1995–6 are shown in fig 1 and show an apparent fourfold variation in productivity. These variations may well not reflect real differences in productivity. Also, given Goldstein's and Spiegelhalter's recent arguments for presenting institutional performance measures with their associated uncertainty intervals, for many trusts differences in labour productivity index values will not be statistically significant.4
National cost weights used to derive standardised activity
Distribution of trusts acording to their labour productivity index for first quarter of 1995–6
The purpose of the labour productivity index is to provide trusts with information to investigate their labour productivity in comparison with others.5 Moreover, the index is supposed also to be a useful tool in contract negotiations. As with other summary efficiency statistics, however, there are a number of questions concerning the index's interpretation, use, and ultimately its usefulness.
Interpreting the labour productivity index
Straightforwardly, the labour productivity index for trusts is a measure of the value of the output per trust employee. “Value” in this context is not the true cost or price of each trusts' total activity. Instead, national average costs have been used to weight different types of activity to enable them to be added together and thus to facilitate comparison of activity totals between trusts. Theoretically, therefore, a trust with an labour productivity index of 6000 can be interpreted not only as being more productive than a trust with an index of 3000, but of being twice as productive.
The problem with the index is that this interpretation is at best misleading and at worst plain wrong. Interpretive problems with the index arise from all three factors used in its calculation (activity, cost weights, and employees):
the nature and comparability of the “product” (or activity)
the accuracy of the national average cost weights
the use of the number of directly employed staff as the denominator.
THE PRODUCT
As can be seen from table 1, the list of “products” or activities used in calculating the labour productivity index appears fairly comprehensive, but major areas of activity—such as teaching, research, and diagnostic services—are excluded. Given variations in trusts' activity mix, these omissions will distort the index. Furthermore, the implicit assumption for those activities that are included is that a particular type of activity—a health visitor contact, say—is the same no matter where or by whom it is produced. This assumption is not correct, however, and takes no account of the possible variation in the quality or effectiveness of different contacts. Finally, no account is taken of variations in case mix across trusts. The effort (cost, labour, etc) to produce one unit of activity (an inpatient finished consultant episode, say) will vary across trusts and will be partly dependent on patient characteristics such as age, comorbidities, complexity, etc.
ACCURACY OF THE NATIONAL COST WEIGHTS
Again, table 1 shows that the national average costs of producing each type of unit of activity are in many cases exactly the same. This arises partly from the fact that it is difficult to discriminate between, say, the costs of a health visitor and district nurse contact with any degree of accuracy using nationally aggregated data. The extent to which such inaccuracy matters—for example, in terms of significantly altering trusts' ranking on the index—is an empirical question.
DIRECTLY EMPLOYED STAFF
The denominator used to calculate each trusts' labour productivity index is the number of whole time equivalent staff direcly employed by each trust. Excluded from this figure are thus any staff employed by organisations subcontracted by trusts. Trusts probably differ considerably in their use of subcontracted services and hence contracted out staff. Whether this matters or not depends on the impact such variation has on the ordering of and distance between trusts according to their index. Again, only empirical research will resolve this question.
How the index is calculated
The labour productivity index for trust j in any particular year is:
LPIj = (Sigma)i, j (Aij.wi)/Sj
where Aij is activity type i for trust j; wi is the national average cost weight for activity type i; and Sj is the total number of directly employed staff for trust j, measured in whole time equivalents.
Using the labour productivity index
The current labour productivity index data has been distributed solely to NHS trusts in England. The implication is that trusts will use this information to compare their labour productivity with other trusts. Presumably, trusts with comparatively low index values will be expected to find out why they are low and consequently do something about it. However, without any pressure—from the NHS Executive or purchasers—there is little reason to suppose that trusts will do anything at all. Even if trusts do respond, they will find it hard if not impossible to dig beneath the figures to explain their comparative position for the reasons noted above. The fact that a particular trust has a comparatively low index value could be due to a number of reasons unrelated to its actual or true productivity—fewer contracted out staff, for example. But unless every trust knows the level of every other trusts' contracted out staff it will be impossible for any one trust to judge the validity of this particular explanation.
The missing incentive for trusts to “do something” about their labour productivity index could perhaps be solved by encouraging purchasers to use the information. However, purchases face similar problems of intepretation if they are to discriminate between trusts on the basis of trusts' index values. Moreover, purchasers are potentially faced with difficult trade offs between one desirable provider quality—for example, a high labour productivity index—and others—for example, proximity to the population to be served. This is not a problem unique to the labour productivity index, but, equally, the index does not help to resolve it. Moreover, effective purchasing requires much more detailed information about trusts' performance than aggregated measures such as the labour productivity index provide.
If, through purchasers, the NHS Executive were to set annual targets for increases in the labour productivity index for trusts (just as it does now with the efficiency index) providers may rightly claim that the index's construction (using national cost weights) renders this a meaningless exercise. A trust's actual index value could be higher, lower, or the same as its index value relative to other trusts depending on whether that trust's actual costs of production were higher, lower, or the same as the national average. Moreover, the exclusion of subcontracted staff means that a trust can improve its labour productivity index merely by contracting out more staff—regardless of the costs involved.
Another possible use of the labour productivity index is in trusts' pay negotiations with their staff. For the past three years the Chancellor of the Exchquer has stated that any pay increases in the NHS must be linked to equivalent increases in productivity. The index provides a broad indication of the level of productivity for trusts and thus seems an obvious measure to use in order to set productivity targets linked to pay rises. Unfortunately, the use of national average costs as activity weights means that it is difficult to interpret an individual trust's year on year change in its index value. Depending on how the national cost weights and trusts' activity change from year to year, it is theoretically possible for an individual trust's index to go down (relative to other trusts) even though their actual productivity has gone up (compared with their own value of the previous year).
A useless statistic?
Many of these criticisms are acknowledged by the NHS Executive,5 which notes that many other factors need to be taken into account when interpreting and acting on the information provided by the labour productivity index. For example, leaving aside the particular problems with the index, is it measuring what would be useful to measure? Should a trust be considered “productive”—that is, having a high index value—if the activity its employees are producing is inappropriate or ineffective? The obvious question, however, is whether the interpretative difficulties of the labour productivity index are so great as to render it useless for the job it is designed to do (presumably increasing labour productivity in the NHS).
Conclusion
The labour productivity index is the latest in a long line of performance measures promulgated by the Department of Health. As with other summary measures, such as the efficiency index, it is theoretically flawed, wrongly constructed, and suffers from data limitations.
The index could be improved—for example, by:
including omitted activities such as teaching, research, and diagnostic services
weighting or standardising for case mix
using more refined cost weights
including contracted out staff.
Although these changes would improve the index, they would not tackle the problem that this index does not necessarily measure what would be useful to measure. Producing inappropriate or ineffective health care activity as cheaply as possible is neither productive nor efficient. Such problems may not, of course, exist in practice: a question that remains unanswered is how the index is actually used by trusts (and possibly purchasers). Is it seen as just another somewhat crude indicator of performance to be ignored? Or is it used in a minor way as one input among many in informing certain aspects of decision making by trusts and purchasers?
Irrespective of the current market type framework in which the NHS operates, there is a clear need to grapple with the construction of robust and meaningful comparative performance measures which help to promote efficiency. The costs of being inefficient are not only measured in wasted resources but also, and more importantly, in lost opportunities to relieve pain and suffering.
I thank Professor Ray Robinson, Dr Bronwyn Croxson, two anonymous referees, Dr Alastair Gray, and members of the health economists' study group for useful comments on previous drafts of this paper.