- Alastair McColl, lecturer in public health medicine ()a,
- Paul Roderick, senior lecturer in public health medicinea,
- John Gabbay, professor and director of public health medicinea,
- Helen Smith, senior lecturer in primary careb,
- Michael Moore, general practitionerc
- aWessex Institute for Health Research and Development, University of Southampton, Health Care Research Unit, Southampton SO16 6YD
- bPrimary Medical Care, University of Southampton, Southampton SO16 5ST
- cThree Swans' Surgery, Salisbury ST1 1DX
- Correspondence to: Dr McColl
- Accepted 3 September 1998
In England primary care groups will have a key role in promoting the health and improving the health care of their local population.1 By April 1999 these groups, involving all primary care professionals, will provide and commission health care for roughly 100 000 people in each locality. Primary care groups will be accountable to health authorities and “will agree targets for improving health, health services and value for money.”1There will be several primary care groups in each district health authority. This new approach offers primary care the opportunity to further integrate health promotion and health care at the individual and population levels.
The present UK government intends to manage the performance of the “new NHS”; the word performance appeared 88 times in its recent white paper.1 It has published a national framework for assessing performance as a consultation document,2and primary care groups within health authorities will be judged to have “performed” well on the basis of the indicators listed in table 1. Most are attributable in part to primary care, but only some are linked to interventions that will necessarily lead to improved health outcomes. The government has also proposed four targets for England in its green paper Our Healthier Nation.3 Approaches taken by health authorities, and presumably by primary care groups, will be “fully monitored by the Regional Offices of the NHS Executive.”3 These targets for reduced death rates from heart disease and stroke, cancer, suicide, and accidents are all outcome indicators but, again, are only partly attributable to primary care.
The NHS Executive and Department of Health have proposed a wide range of performance indicators many of which are applicable to future primary care groups
Some of these indicators reflect access and efficiency, but few of the effectiveness indicators are based on primary care interventions for which there is evidence that increased uptake results in improved health outcomes
We present a method to identify important primary care interventions of proved efficacy and suggest performance indicators that could monitor their use
Our evidence based approach may be a complementary way of identifying areas for performance indicators to those proposed by the NHS Executive and Department of Health
Our suggested indicators are more likely to help turn evidence into everyday practice and to have an impact on the population's health
Performance indicators for practices —Previous governments have attempted to use performance indicators for group practices of general practitioners, such as those linked to payments for uptake of immunisations and cervical smears. Health authorities have also tried to use practice based performance indicators,4 with varying degrees of success. 5 6 The recent availability of data on prescribing analysis and cost has allowed health authorities to look at practice prescribing in more detail and to develop indicators reflecting “good and bad” prescribing.7Campbell et al have identified a number of valid practice indicators from over 240 under consideration for use by health authorities in England and Wales.8
Performance indicators for primary care groups —To maximise their usefulness, performance indicators for primary care groups should meet certain minimal criteria before any consideration of their introduction into routine use. They should be attributable to health care,9 sensitive to change,10based on reliable and valid information, precisely defined, reflect important clinical areas, and include a variety of dimensions of care. The US National Library of Healthcare Indicators describes several “definable, measurable and improvable domains of performance” for its indicators.11These are attributes of organisational performance related to “doing the right things” (such as appropriateness, availability, and efficacy) and “doing things right” (such as effectiveness, efficiency, respect and caring, safety, and timeliness).11 For those indicators that reflect appropriateness, availability, efficacy, and effectiveness there should be robust evidence that the interventions on which they are based lead to improved health outcomes. Use of such indicators to monitor performance may be one way to promote the wider use of evidence based interventions —for example, in the secondary prevention of coronary heart disease. 1213
However, there is more to primary care than the use of evidence based interventions. Other important dimensions to primary care include consultation skills, the advocacy role of members of the primary care team for individual patients, communication within the practice team, access to primary care, managing a business within a regulatory framework, and coordination with community, secondary care, and local authority services. The use of evidence based interventions and related performance indicators as presented in this paper can therefore only represent some aspects of primary care. Further research is needed to address the feasibility of developing meaningful performance indicators reflecting these other dimensions
The aim of our study was to develop a method to identify important, evidence based interventions in primary care suitable for linking to performance indicators for primary care groups. Our objectives were to (a) identify interventions of proved efficacy for which primary care teams have a key responsibility; (b) estimate the number of preventable deaths or events in a primary care group locality of 100 000 people if all those eligible were receiving the intervention; and (c) compare the potential indicators we derived with the indicators currently proposed by the government.
There is no simple definition of primary care.14 Aspects of primary care include general practice, community nursing, midwifery, health visiting, pharmacy, dentistry, optometry, and other professions. For the purpose of this study, we identified primary care interventions of proved efficacy from systematic reviews and for which we judged primary care teams to have the major responsibility. We searched the Cochrane Database of Systematic Reviews and the Database of Abstracts of Reviews of Effectiveness15 and Effective Health Care bulletins and obtained the primary sources referred to in the abstracts.
Mant and Hicks proposed a method to compare the relative sensitivity of indicators to monitor differences in care for the hospital treatment of myocardial infarction.10 We developed their approach and for each primary care intervention estimated:
(a) Reduced risk of death or events for those receiving the intervention compared with those not receiving it over a certain period —the relative risk reduction (%)
(b) Mortality or event rate of those not receiving the intervention (the controls) over a certain period
(c) The difference in risk of death or events between those receiving the intervention and those not receiving it—the absolute risk reduction (a£b)
(d) The number of patients needed to receive the intervention in order to prevent one of them dying or developing an adverse event—the number needed to treat (1/c, the reciprocal of the absolute risk reduction16)
(e) The proportion (and number) of patients likely to be eligible to receive the intervention in a locality of 100 000
(f) The adjusted relative risk reduction to take into account those eligible for the intervention over a certain period (a£e)
(g) The adjusted absolute risk reduction (c£e%) and number of preventable deaths or events in the locality over a certain period if all those eligible received the intervention
(h) Current rate of uptake of the intervention in those eligible in the primary care group (estimated from published studies, local data, or local opinion)
(i) Additional number of preventable deaths or events if all those eligible in the locality received the intervention (g£(1−h)).
We also made brief comments on the interventions such as potential side effects and whether the intervention was likely to be cost effective.
Estimates of the potential impact of interventions
Table 2lists the primary care interventions we examined and whether we were able to obtain key information as described in the previous section. For the purposes of this brief discussion, we focus our illustrative method on the first eight interventions listed for which we were able to easily translate risk reduction into improvement in health outcome.
Table 3shows the relative risk reductions for these eight interventions together with the number of patients likely to be eligible in a locality and the number of preventable deaths or events in the locality if all those eligible received the intervention. (Full details of how we estimated stages (a) to (i) for each intervention and the assumptions we made are listed in the appendices on the BMJ website). Some interventions, despite having high numbers needed to treat, could have a considerable impact on the health of a population. For example, 108 people aged over 65 need to receive influenza vaccination each year to prevent one death, but in a population of 100 000 this intervention could prevent 146 deaths each year.
Table 4shows the estimated current uptake for each intervention, the additional number of preventable events if all eligible patients receive the intervention, and, briefly, the likely cost effectiveness of the intervention. The additional number of preventable deaths or events with full uptake is highly dependent on the estimated current uptake rate. The few studies that have examined these rates suggest that uptake is low. Considerable improvements in health outcomes would result if primary care groups with low uptake of these interventions —apart from use of statins for patients at low risk of coronary heart disease —improved their uptake rates. For example, a locality would prevent 24 deaths each year if all high risk patients took aspirin rather than the 50% who currently do so.
For these eight interventions that improve health outcomes, table 5 lists the possible performance indicators that could measure their use in primary care groups. There are indicators of the proportion of the population with diagnoses of hypertension, coronary heart disease, atrial fibrillation, and heart failure. Comparing observed with expected proportions could highlight inadequate detection of these diseases or incomplete Read coding within primary care. Data sources for all these indicators are available in practices with well computerised records.
Our sources of evidence were not comprehensive. We selected the eight interventions because of the ease of obtaining information, including the ability to translate evidence on efficacy into improvement in health outcomes at a population level. It would be possible to use this method for other primary care interventions and to overcome some of the difficulties listed in table 2. We used our sources of evidence in an illustrative way to demonstrate the potential for developing performance indicators based on interventions of proved efficacy.
We used end points from randomised controlled trials based in primary care, which are usually mortality and major non-fatal events. These end points for events such as further strokes or myocardial infarction are defined variously as, for example, “major coronary events,” “vascular deaths,” and “coronary heart disease events.” The terms presented in the tables are those used in the relevant trials. These events are rare within an individual general practice and underestimate the burden of morbidity.
Our method at present takes no account of years of life lost or the difference between prevalent and incident cases. We used prevalent cases, and we recognise that absolute gains would fall over time. The effects of the interventions included are over different time scales, and there are wide confidence intervals for the size of these effects and estimates of prevalence. The effect is also dependent on patient compliance; patient preferences and contraindications would further reduce the number eligible for these interventions. We interpreted odds ratios reported in systematic reviews as relative risks and may have therefore overstated any effect size.35Interventions interact in complex ways, but our method presents them in isolation. We only briefly mention potential side effects and likely cost effectiveness of these interventions. Ideally, we would want to compare the overall cost per life year gained for each intervention.
Despite these methodological difficulties, we believe that this approach is a useful complement to that used by the NHS Executive and Department of Health in developing performance indicators relevant to primary care. Our method identifies those interventions that are attributable to primary care and estimates the relative importance of these in terms of reduced mortality or non-fatal events. It helps to emphasise the importance of examining healthcare needs both at the individual and population levels by taking into account the prevalence of conditions and the current uptake of interventions. This method could be used to develop performance indicators for areas other than primary care.
Requirements for developing evidence based indicators
Before using the indicators proposed in table 5 it is essential to develop clear definitions of the numerators and denominators for each indicator. Sufficient numbers and standard definitions are required to enable comparisons between practices in a primary care group. Indicators require evaluation both before and after their introduction into routine use to fulfil practical and scientific criteria.36We are currently attempting to derive and evaluate the indicators in table 5 for all patients aged 45-69 in 19 local practices of a future primary care group. These indicators require collection of extra data, and some might argue that primary care teams cannot cope with yet more tasks. However, in the United Kingdom well over a million hours every month are already spent collecting data in primary care,37and yet there is little consensus on which data should be collected. Focusing data collection on meaningful indicators and abandoning it in less relevant areas could result in an overall reduction in workload. If the government is to use performance indicators as a method of improving health and health care it is important to encourage health professionals to focus on data collection linked to interventions over which they have substantial control and which improve health outcomes.
Comparison with the performance indicators currently proposed
There are considerable differences between the evidence based performance indicators that we generated (table 5) and those in the national framework for assessing performance relevant to primary care (table 1). Some of the latter are important, relating to efficiency and access, but many of the others could create perverse incentives to change practice. 3839 For example, the indicators relating to district nurses may encourage district nurses to reduce the number of appropriate visits to patients aged under 75 years. Similarly, in order to seem to “perform” well, general practitioners may reduce the number of appropriate hospital admissions for anyone aged over 75 or those with pyelonephritis, heart failure, or asthma. They may even stop notifying pertussis or measles. Our evidence based indicators may be less likely to encourage perverse incentives as they are based on robust evidence. However, health authorities and primary care groups would have to use such indicators appropriately and ensure that the risks and benefits of interventions were considered, especially in elderly patients.
Applying evidence from clinical trials and systematic reviews to individual patients in primary care is complex and challenging.40 Overcoming operational issues and changing clinical behaviour require a multifaceted approach. 41 42The use of performance indicators by themselves as a method to improve the effectiveness of health care in primary care groups is unlikely to succeed. However, the use of evidence based indicators linked to interventions that improve health outcomes, such as those suggested in table 5, could be an important adjunct if used in interactive practice or primary care group educational meetings.43 Primary care group indicators should be based on robust evidence. If not, their use is unlikely to lead to improved health outcomes. Our method may be a complementary way of identifying areas for performance indicators to those proposed by the NHS Executive and Department of Health. Our suggested indicators are more likely to help turn evidence into everyday practice and to have an impact on the population's health.
Contributors: AMcC and PR developed the original idea. AMcC, PR, JG, HS and MM jointly wrote the paper.
Funding The study was funded by the Department of Health. The views expressed here are those of the authors and not necessarily those of the Department of Health.
Conflict of interest None.