General Practice

Prescribing indicators for UK general practice: Delphi consultation study

Stephen M Campbell, research fellow ^a, Judy A Cantrill, clinical senior lecturer ^a, Dave Roberts, manager ^b. 

^a National Primary Care Research and Development Centre, University of Manchester, Manchester M13 9PL, ^b Prescribing Support Unit, Brunswick Court, Leeds LS2 7RJ

Correspondence to: S M Campbell stephen.campbell{at}man.ac.uk

	Abstract

Top Abstract Introduction Participants and methods Results Discussion References

Objectives: To identify prescribing indicators based on prescribing analysis and cost (PACT) data that have face validity for measuring quality or cost minimisation.
Design: Modified two round Delphi questionnaire requiring quantitative and qualitative answers.
Setting: Health authorities in England.
Participants: All health authority medical and pharmaceutical advisers in the first round and lead prescribing advisers for each health authority in the second round.
Main outcome measures: Face validity (median rating of 7-9 on a nine point scale without disagreement) and reliability (rating 8 or 9) of indicators for assessing quality and cost minimisation.
Results: Completed second round questionnaires were received from 79 respondents out of 99. The median rating was 7 for cost minimisation and 6 for quality, and in all except four cases individual respondents rated indicators significantly higher for cost than for quality. Of the 41 indicators tested, only seven were rated valid and reliable for cost minimisation and five for quality.
Conclusion: The 12 indicators rated as valid by leading prescribing advisers had a narrow focus and would allow only a limited examination of prescribing at a general practice, primary care group, or health authority level.

	Introduction

Top Abstract Introduction Participants and methods Results Discussion References

Quality of care within the NHS is a seminal focus of government policy. This focus on quality has driven the development of new organisational structures such as the National Institute for Clinical Excellence and the national performance framework to measure progress in six areas of health care. ^{1 2} In addition, clinical governance structures are being put in place to provide "a framework through which primary care groups will be accountable for continuously improving the quality of their services."¹

Prescribing indicators for general practice have been used in the NHS for over two decades³ and are likely to have a central role in the clinical governance activities of many primary care groups. This is because prescribing continues to grow at about 9% a year⁴ and two thirds of all general practice consultations generate a prescription.⁵ The national performance framework described cost effective prescribing as an important element of the "effective delivery of appropriate healthcare."²

Prescribing is a controversial area of quality assessment.⁶ Previous research has highlighted the importance of critical approaches to prescribing, ^{6 7} defining and measuring the appropriateness of prescribing,^8-10 variations in prescribing across general practices,¹¹ adherence to standards,¹² and the role of prescribing analysis and cost (PACT) data in general practice.¹³ Few validated quality indicators exist for prescribing in the public domain. ^{6 7} Avery et al concluded that "further research is needed into the development and use of indicators based on PACT."¹¹ The Prescribing Support Unit has developed a set of indicators based on PACT data. It advocates their use as a starting point when comparing the performance of health authorities or primary care groups with that of other authorities or groups or when comparing prescribing among general practices to identify outliers or those which are more likely to benefit from interventions to modify behaviour. PACT data are comprehensive, universal, and timely but they are not combined with diagnoses, data on specific patients, or any outcome measure. Projects are underway that aim to link prescribing and clinical data in order to produce quality indicators.

Anecdotal evidence suggests that prescribing indicators are more appropriately related to cost than quality, particularly at the practice level (the unit of analysis for most prescribing indicators). We report the findings of a two round Delphi consultation ^{14 15} that sought to identify which of the most commonly used prescribing indicators in the United Kingdom are face valid and reliable indicators of quality or cost minimisation. The questionnaire included only indicators of drug use that could be derived from PACT data and basic demographic features of practice populations.

	Participants and methods

Top Abstract Introduction Participants and methods Results Discussion References

A list of 31 prescribing indicators was generated from on two main sources: prescribing indicators with evidence of face validity in a previous Delphi consultation⁷ and, most importantly, prescribing indicators used at the time of the survey by the Prescribing Support Unit.³

In May 1999 we sent the first questionnaire of a modified two round Delphi consultation to every pharmaceutical and medical adviser in England (n=305). Respondents were asked to rate each indicator against two continuous 1 to 9 integer scales: "Is this indicator a useful measure of cost minimisation?" and "Is this indicator a useful measure of quality?" Definitions of these two constructs were provided in the covering letter and questionnaire sent to respondents. Cost minimisation was defined as "using the lowest cost preparation with no adverse effect on benefit" and quality as "optimisation of health/well-being for the whole practice population." Respondents were also asked to state whether they currently used each indicator. The questionnaire invited respondents to comment on each of the 31 indicators.

No indicators were discarded between rounds, but 10 indicators were added; two indicators related to statins and eight were minor variations on indicators used in the first round and were based on comments received in that round. The second round questionnaire therefore contained 82 ratings (41 each for cost minimisation and quality).

Participants who were sent the second round questionnaire were given three types of feedback from the first round for each indicator included in both rounds: a frequency distribution of scores (on scales of 1 to 9), a median (face validity) score for both scales, and qualitative comments (figure). Qualitative comments made during round one were transcribed and summarised. We included comments that illustrated the negative and positive attitudes expressed in the first round to provide contextual information on which respondents could base their ratings. We did not feed back to respondents their previous score.

View larger version (28K): [in this window] [in a new window]   Example of feedback on indicators included in second round questionnaire

After obtaining comments from a wide range of medical and pharmaceutical advisers (n=154) in the first round, we used the second round to achieve consensus among respondents at the health authority level. Second round questionnaires were sent in July 1999 to the lead prescribing adviser at each health authority in England (n=99) in order to obtain one single health authority return. Respondents were asked to rate each indicator using the same method as in the first round. Non-responders received one follow up mailing.

The validity data presented in this paper are based on second round median ratings only.¹⁶ We used a rating scale based on the RAND appropriateness method.¹⁶ Indicators with an overall median rating of 7, 8, or 9 without disagreement were rated face valid; indicators rated with an overall median of 1-3 and 4-6 were rated as invalid and equivocal respectively. Disagreement was defined as 30% or more scores in both the bottom (1-3) and top (6-9) tertile.¹⁷ Previous research has found that indicators rated 8 or 9 without disagreement are also reliable because higher rated indicators are more likely to be reproduced by a different panel of the same stakeholders rating the same set of indicators.¹⁸ Indicators rated with an overall median of 8 and 9 were therefore considered face valid and reliable.

Scores were analysed by using SPSS with non-parametric tests (Wilcoxon's z test) to examine whether indicators were significantly more likely to be rated valid for cost or quality.

	Results

Top Abstract Introduction Participants and methods Results Discussion References

Completed second round questionnaires were received from 79 respondents out of 99 (response rate of 79%). Overall median ratings for the complete set of 41 ratings in the second round were calculated for cost minimisation and quality. These were calculated from individual medians not from the raw scores. The median rating was 7 for cost minimisation and 6 for quality. Table 1 shows that 17 indicators were rated higher for cost, 16 higher for quality, and eight were rated identically. Overall, there was no significant difference in ratings for cost or quality (Wilcoxon's z =0.76, P=0.45). However, in all except four cases individual respondents rated indicators significantly higher for cost than for quality (by Wilcoxon's z test). The four exceptions were ratio of compound diuretics items to all diuretic items; ratio of antibiotic items for co-amoxiclav or 4-quinolones to the number of items for all antibiotics; percentage of total net ingredient cost on drugs of limited clinical value; and percentage of non-steroidal anti-inflammatory drug items from ibuprofen, diclofenac, and naproxen.

No indicators were rated with an overall median of nine. Twenty five indicators were rated face valid for cost minimisation and 18 for quality. Of these, nine were rated valid for both (table 1). Although the remaining indicators were all rated as equivocal quality indicators, nine were rated as invalid for cost minimisation. No indicators were rated invalid for quality. Twelve indicators were rated reliable, seven for cost minimisation and five for quality.

Only two of the indicators rated valid and reliable for cost or quality in this study were currently being used by over 50% of the sample (table 2). These were generic prescribing rate and ratio of bendrofluazide 2.5 mg items to all bendrofluazide items.

	Discussion

Top Abstract Introduction Participants and methods Results Discussion References

Clinical governance is designed to be a means by which health organisations at different service levels can maintain and improve quality of care. Prescribing will constitute a key clinical governance objective of many primary care groups. This study aimed to discover how many of the most frequently used prescribing indicators are rated valid by advisers responsible for managing prescribing in health authorities in England. Our findings suggest that advisers believe that prescribing indicators based on PACT at the population level are less valid for quality than for cost minimisation.

Thirty three of the 41 indicators rated in the second round were found to be face valid for either cost (n=25) or quality (n=18) or both (n=9). However, only 12 indicators were also rated reliableseven for cost and five for quality. These 12 indicators have a narrow focus and will allow only a restricted assessment of prescribingfor example, four of the seven indicators for cost minimisation relate to generic prescribing. Hence, the results obtained with these indicators need to be interpreted carefully and their limitations explicitly acknowledged.

PACT data make some, but by no means all, aspects of prescribing measurable. Three important decisions have to be made when collecting data on prescribing indicators. Firstly, what is the intended unit of analysis (for example, practice population, all individuals with a given condition, an individual)? Secondly, who is going to collect the data (health authorities, primary care groups, individual practices)? Thirdly, what are the resources required for data collection (patients' medical records or PACT)? Prescribing indicators can be used for various purposes, and it is vital for quality assessment that this purpose is made explicit.¹⁹ The validity of any type of indicator is related to its intended purpose. Additional resources are needed to produce and collect data for indicators relating to individual patients rather than populations and for indicators requiring examination of individual patients' records rather than PACT data. ^{9 20}

Our finding that five of the indicators were rated face valid for measuring quality does not fit within the model of indicators proposed by Queenborough and Roberts.²⁰ They advocated that only when prescribing and clinical data are linked can quality be measured, usually with reference to individual patient data. This view was shared by many respondents in the survey when answering an open ended question about how the quality of prescribing can or should be measured. Respondents felt that PACT indicators are process measures that need to be linked to clinical audit or patient outcome measures. This view also reflects the opinion that definitions and measurements of quality of care are most meaningful when applied to individual patients.²¹

Use of indicators
Our findings suggest three further caveats for people engaged in quality assessment or improvement, including primary care groups in the United Kingdom. Firstly, indicators are not measures of poor performance. Rather, they identify potential problems that may require investigation by other methods, usually audit. Secondly, it is important to be clear about what the indicators are intended to measure and what conclusions can be claimed from their use. Thirdly, for indicators to be useful for quality assessment or improvement, consistent and comparable data must be available across the relevant healthcare organisations.

	Acknowledgments

We thank Clive Jackson, director of the National Prescribing Centre, for help with sending out the questionnaire and the respondents who took part in the Delphi consultation.

Contributors: The study was devised by SMC, JAC, and DR. SMC and JAC did the Delphi consultation and analysed the data. All three authors wrote the paper, with SMC as the principal author. SMC and JAC are the guarantors.

	Footnotes

Funding: This project was funded out of National Primary Care Research and Development Centre core funding from the Department of Health. The Prescribing Support Unit is funded by the Department of Health.

Competing interests. None declared.

	References

Top Abstract Introduction Participants and methods Results Discussion References

1.	Department of Health. Quality in the new NHS: a first class service. London: DoH, 1998.
2.	Secretary of State for Health. The new NHS. London: Stationery Office, 1998. (Cm 3807.)
3.	Prescribing Support Unit. Prescribing measures and the application. An explanation. Leeds: PSU, 1998.
4.	National Prescribing Centre and NHS Executive. GP Prescribing Support Unita resource document for the new NHS. London: NPC, NHSE, 1998.
5.	Audit Commission. A prescription for improvement: towards more rational prescribing in general practice. London: HMSO, 1994.
6.	Roland M, Holden J, Campbell S. Quality assessment for general practice: supporting clinical governance in primary care groups. Manchester: National Primary Care Research and Development Centre, 1999.
7.	Campbell SM, Roland MO, Quayle JA, Buetow SA, Shekelle PG. Quality indicators for general practice: which ones can general practitioners and health authority managers agree are important and how useful are they? J Public Health Med 1998; 20: 414-421[Abstract/Free Full Text].
8.	Buetow SA, Sibbald B, Cantrill JA, Halliwell S. Appropriateness in health care: application to prescribing. Soc Sci Med 1997; 45: 261-271.
9.	Cantrill JA, Sibbald B, Buetow S. Indicators of the appropriateness of long term prescribing in general practice in the United Kingdom: consensus development, face and content validity, feasibility, and reliability. Qual Health Care 1998; 7: 130-135[Abstract].
10.	Blades S, Eccles M, McColl E, Campbell M. Understanding the appropriateness of prescribing in primary care. Eur J Gen Pract 1998; 4: 60-64.
11.	Avery AJ, Heron T, Lloyd D, Harris CM, Roberts D. Investigating relationships between a range of potential indicators of general practice prescribing: an observational study. J Clin Pharm Ther 1998; 23: 441-450[CrossRef][Medline].
12.	Bateman DN, Eccles M, Campbell M, Soutter J, Roberts SJ, Smith JM. Setting standards of prescribing performance in primary care: use of a consensus group of general practitioners and application of standards to practices in the north of England. Br J Gen Pract 1996; 46: 20-25[Medline].
13.	Majeed A, Evans N, Head P. What can PACT tell us about prescribing in general practice? BMJ 1997; 315: 1515-1519[Free Full Text].
14.	Cantrill JA, Sibbald B, Buetow S. The Delphi and nominal group techniques in health services research. Int J Pharm Pract 1996; 1: 67-71.
15.	Jones J, Hunter D. Consensus methods for medical and health services research. BMJ 1996; 311: 376-380[Free Full Text].
16.	Brook RH. The RAND/UCLA appropriateness method. Santa Monica: RAND, 1995.
17.	Brook RH, Chassin MR, Fink A, Solomon DH, Kosekoff J, Park RE. A method for the detailed assessment of the appropriateness of medical technologies. Int J Tech Ass Health Care 1986; 2: 53-63.
18.	Shekelle PG, Kahan JP, Park RE, Bernstein SJ, Leape LL, Kamberg CA, et al. Assessing appropriateness by expert panels: how reliable? J Gen Intern Med 1995; 10(suppl): 81.
19.	Cantrill J, Devlin M, Jackson C, Queenborough R. Improving quality in primary care: supporting pharmacists in primary care groups and trusts. Manchester: National Primary Care Research and Development Centre, 1999.
20.	Queenborough R, Roberts D. The relationship between the quality of prescribing indicators and their availability: a model for primary care groups. Prescriber 1999; 10: 47-51.
21.	Campbell SM, Roland MO, Buetow SA. Defining quality of care. Soc Sci Med (in press).

(Accepted 25 May 2000)