Randomised controlled trial of effect of feedback on general practitioners' prescribing in AustraliaBMJ 1999; 318 doi: http://dx.doi.org/10.1136/bmj.318.7182.507 (Published 20 February 1999) Cite this as: BMJ 1999;318:507
- Dianne L O'Connell, senior Brawn fellow ()a,
- David Henry, professor of clinical pharmacologya,
- Ron Tomlins, senior medical adviserb
- aDiscipline of Clinical Pharmacology, Faculty of Medicine and Health Sciences, University of Newcastle, New South Wales, Australia
- bProfessional Review Division, Health Insurance Commission, Tuggeranong, Australian Capital Territory, Australia
- Correspondence to: Dr D L O'Connell, Discipline of Clinical Pharmacology, Level 5, Clinical Sciences Building, Newcastle Mater Hospital, Waratah NSW 2298, Australia
- Accepted 3 December 1998
Objective:To evaluate the effect on general practitioners' prescribing of feedback on their levels of prescribing.
Design: Randomised controlled trial.
Setting: General practice in rural Australia.
Participants: 2440 full time recognised general practitioners practising in non-urban areas.
Intervention: Two sets of graphical displays (6 months apart) of their prescribing rates for 2 years, relative to those of their peers, were posted to participants. Data were provided for five main drug groups and were accompanied by educational newsletters. The control group received no information on their prescribing.
Main outcome measures:Prescribing rates in the intervention and control groups for the five main drug groups, total prescribing and potential substitute prescribing and ordering before and after the interventions.
Results: The intervention and control groups had similar baseline characteristics (age, sex, patient mix, practices). Median prescribing rates for the two groups were almost identical before and after the interventions. Any changes in prescribing observed in the intervention group were also seen in the control group. There was no evidence that feedback reduced the variability in prescribing nor did it differentially affect the very high or very low prescribers.
Conclusions: The form of feedback evaluated here—mailed, unsolicited, centralised, government sponsored, and based on aggregate data—had no impact on the prescribing levels of general practitioners.
Feedback of prescribing data to general practitioners is widely practised by government agencies
The belief is that this will lead to reduced variability and lower rates of prescribing of key drugs, but this has not been tested in randomised trials
In a large randomised trial Australian general practitioners received feedback comprising simple graphical displays of their prescribing data for five key groups of drugs
This had no impact on the level or variability of subsequent prescribing rates
Unsolicited, centralised, government sponsored feedback based on aggregate data had no impact on the prescribing levels of general practitioners
Around the world governments are struggling to contain healthcare costs. One topic that has received particular attention is prescribing.1 In the United Kingdom in 1995 general practitioners wrote about 550 million prescriptions at a total cost of £4700m.2 In Australia, the Commonwealth Government's spending on pharmaceutical products has been increasing by around 10% annually.3 The main factor leading to the growth in expenditure is the preference of prescribers for new and expensive drugs for conditions for which cheaper alternatives are available. Examples include drugs for hypertension, hyperlipidaemia, acid reflux, and depression.3 It is recognised that changing prescriber behaviour is difficult and often requires complex multifaceted interventions. 4 5 Such interventions, including education and academic detailing of opinion leaders, are labour intensive and expensive and may be difficult to apply at a national level. Consequently, there has been some enthusiasm for “simple” interventions that entail feedback of basic prescribing data to practitioners. This approach has been adopted in the United Kingdom through the feedback to general practitioners of analyses of prescribing and cost (PACT) data.2
Prescriber feedback on its own would be expected to have only a modest impact. 4 5 It is easy and cheap to implement on a large scale, however, and is potentially attractive to government agencies and other third party payers. Even small reductions in prescribing would be worth while. For instance, a 5% reduction in prescribing would result in a saving of £235m in the United Kingdom and $A140m in Australia annually. Feedback as a single intervention has not been evaluated previously in large scale randomised trials capable of detecting small changes in prescribing behaviour.
In Australia the agency responsible for payment for community delivered medical services, diagnostic tests, and pharmaceutical benefits is the Health Insurance Commission. Feedback of ordering of pathology tests has been provided by the agency for several years and in 1995 it was extended to prescribing activities. It was decided to trial the intervention before full implementation, and the results of the evaluation are reported here.
Participants were vocationally registered general practitioners working in “major rural” and other rural centres in Australia who were recorded as having a minimum income from the Commonwealth Government (through the Medicare programme) of $A60 000 annually. To avoid “contamination” of control participants, who might work closely with practitioners who were part of the intervention group, all general practitioners with a principal practice address in a given postcode were randomised to the intervention or control group.
Postcodes were grouped into geographically contiguous clusters. Stratification was based on thirds of the numbers of general practitioners and thirds of levels of prescribing activity within postcode clusters. Block randomisation of postcode clusters was carried out within each of the nine strata. The block size was four.
We studied five main drug groups: angiotensin converting enzyme inhibitors, lipid lowering drugs, histamine H2-receptor antagonists, non-steroidal anti-inflammatory drugs, and oral antibiotics. The first three groups were chosen because of the rapid growth in their use and costs. Non-steroidal anti-inflammatory drugs were chosen because their overuse is associated with substantial toxicity. Overuse of antibiotics has a range of adverse effects, including ecological consequences.
There were two interventions—in May 1995 and November 1995—each consisting of a graphical display of the individual's level of prescribing in the previous eight quarters. The number of original prescriptions written for each class of drugs was expressed as a rate per 100 services provided during the same period, and these were plotted on graphs which also displayed, for the purposes of comparison, the interquartile range for prescribing rates for all general practitioners enrolled into this trial. The interquartile range, representing the middle 50% of the distribution of the prescribing rate, was presented as a shaded area. In addition, prescribers were provided with a comparison of the age-sex profile of their patients compared with those attending the other general practices. In the first intervention the graphical displays and data were accompanied by an educational newsletter covering some general prescribing issues. The second intervention included 6 additional months of prescribing information, and the accompanying newsletter focused on specific issues relating to the prescribing of antibiotics.
Preliminary calculations of sample size based on pilot data from 344 general practitioners in nine postcode areas indicated that a trial with 2700 subjects would have 90% power to detect a 10% relative reduction in prescribing rates (less than 1.3 prescriptions per 100 Medicare services in absolute terms) for the intervention group compared with the control group at the 5% level (two sided) for each of the five groups of drugs. This took into account the clustering effect of postcode with estimated correlation coefficients within clusters ranging from 0.05 to 0.12. The achieved sample size of 2440 general practitioners yielded 90% power to detect relative reductions of 5-6% in prescribing rates (or absolute reductions of less than 1 prescription per 100 Medicare services) in the intervention group versus the control group. The actual correlation coefficients within clusters ranged from 0.14 (for lipid lowering drugs) to 0.24 (for antibiotics). The median cluster size was four general practitioners, with a range of 1 to 54.
The distributions of prescribing rates for each of the drug groups by quarter and by month were generated for each group for the 16 months before the first intervention and up to 4 months after the second intervention. Median prescribing rates were plotted and compared between groups. Interquartile ranges were also generated and compared.
The project was regarded as part of the routine audit and feedback operations of the Health Insurance Commission and was subjected to that organisation's normal approval processes. All participants in the study received feedback of their prescribing after the trial phase. General practitioners were informed of the purpose of the study in writing before the study started and had an opportunity to communicate any concerns to the commission. There was no coercion, and they were free to disregard the feedback data. Analysis was confined to anonymous data, and all of the investigators were bound by the confidentiality provisions of the Health Insurance Commission Act. DLO and DAH, who worked as consultants to the commission during the project, received advice (from their institutional ethics committee) that their participation did not require formal approval by an ethics committee.
The groups were well matched for the variables shown in table 1 and sex (87% men in both groups), despite the imbalance in the numbers of general practitioners per group (n=1294 in the intervention group and n=1146 in the control group). This imbalance occurred because six of the seven largest postcode clusters were, by chance, assigned to the intervention group.
Median prescribing rates (per 100 Medicare services) by month from January 1994 to March 1996 for each of the five groups of drugs and for all pharmaceuticals covered by the Pharmaceutical Benefits Scheme are shown in the figure. Prescribing of oral antibiotics shows seasonal variation, the rates being highest over the winter period in Australia (July, August, September). The higher prescribing rates for non-steroidal anti-inflammatory drugs in the last quarter of each year are due to patients obtaining and storing extra medications before their annual entitlement for subsidised prescriptions expires. Prescribing rates for oral antibiotics showed an overall decline, whereas rising rates were observed for the other classes.
In no case did the feedback of graphical displays of prescribing rates and newsletters have any measurable impact on prescribing rates. For each of the drug classes and for all prescriptions the curves for the intervention and control groups were nearly identical (see figure). There was also no evidence that feedback reduced the variability in prescribing (table 2). The interquartile ranges in each group remained almost identical, as did the 5th to 95th centile ranges. In addition, feedback did not seem to have a differential effect on general practitioners in the lowest and highest tenths of the prescribing rates for each drug group (data not shown). These results were not altered by further statistical analysis.
The findings of this study indicate that the form of prescriber feedback used here—mailed, unsolicited, centralised, government sponsored, and involving aggregate data—is unlikely to have any impact on the prescribing activities of general practitioners. This rigorous randomised controlled design showed no impact of the interventions, either on median prescribing rates or on variability in prescribing rates. The trial had sufficient statistical power to detect any important effect. It should be noted from the graphs that for some drug groups a decline in prescribing rate seemed to follow the intervention; in the absence of a valid control group (which in this study displayed the same trends), we might have concluded that the intervention was having some impact.
A recent Cochrane review of 37 randomised controlled trials concluded that audit and feedback of data on practice activity can sometimes be effective in changing the behaviour of healthcare professionals.6 There have, however, been few rigorous evaluations of the impact of feedback on prescribing activity.7–14 Of these eight randomised controlled trials only four concerned prescribing by general practitioners or family physicians in private practice, and only one evaluated feedback from a government department or authoritative agency.7 Other reviews have concluded that to be effective, feedback should offer clear alternatives to current practices; be part of an overall strategy for changing behaviour; and be presented close to the time of decision making. 8 15 16 The authority of the “messenger” is important, and the “message” needs to be repeated at intervals to sustain any effect.16 In addition, the process should be active rather than passive, and the participants should have agreed previously to review their practices. 16 17
Reasons for ineffectiveness of feedback
There are several possible reasons why the form of feedback described here was ineffective. The first is the ambiguity of the message. Though very high prescribing of non-steroidal anti-inflammatory drugs and antibiotics is undesirable, prescribing rates alone cannot be used to judge the quality of the use of other classes of drugs such as angiotensin converting enzyme inhibitors and lipid lowering drugs. Consequently, it is difficult for prescribers to respond to graphical displays of their own prescribing rates for these latter groups of drugs. The educational messages that accompanied the feedback in this trial were of a general nature and not individualised according to the profiles of individual general practitioners.
So the intervention studied here, although easy to implement on a national scale, lacked some of the features that are thought necessary to effect important behaviour change. The feedback was not close enough to the time of prescribing; it was seen to come from a national agency with no local input or ownership; there was no opportunity for discussion of the data in a problem based format; and the intervention did not offer alternatives to the drugs being highlighted in the feedback. Lastly, there were no real incentives to the participants to change their behaviour.
As a small (but important) impact might still have been expected we were surprised that the intervention had no discernible effect on prescribing behaviour. This is an important message, particularly for government agencies planning to conduct similar feedback in the belief that it will have a modest impact on prescribers or possibly sensitise them to other educational messages. In Australia payment for drugs and medical services takes place under a national health insurance programme (Medicare). The Health Insurance Commission, which is a statutory authority, has responsibility for the operational aspects and for investigating professional fraud and overservicing. The ambiguity in its role—as investigator rather than an educator—may have had a negative impact.
Limitations of study
There are some limitations to this study. The methods were rigorous, but there were weaknesses in some of the outcome measures. Australia has a complex system of reimbursement of pharmacists for cost of pharmaceutical products.18 The Health Insurance Commission will hold a record of a transaction only if the cost of the drug is higher than the contribution from the patient ($A18.00 at the time of the study). In the case of pensioners or holders of concession cards, however, the government covers virtually the full cost of all the study drugs. Data capture for non-steroidal anti-inflammatory drugs and antibiotics will have been incomplete, but this should have been equal in the intervention and control groups and therefore not a source of bias. Data capture for the other drug classes was complete, and there were no qualitative or quantitative differences in the response to the interventions between drug classes.
In conclusion, we believe that centralised, government sponsored prescriber feedback is not worth while and should not be seen as a high priority by government agencies. Prompt detailed feedback of individualised prescribing data in a clinical setting with an effective educational programme, however, may be effective. In Australia we believe these interventions should be carried out at the local level, possibly by divisions of general practice. The prescribing data that we used in this study could be useful but will need to be augmented by more clinically relevant datasets.
We thank staff at the Health Insurance Commission and the General Practice Branch, Department of Health and Family Services, particularly Jane Parker and Gordon Calcino, for help in setting up and executing the feedback and evaluation. We also thank Timothy Dobbins, who helped in the analysis of the data.
Contributors: RT had the original idea for the present study and handled the operational aspects of the study in the Health Insurance Commission. DLO and DAH are consultants to the commission and were responsible for the design, planning, and analysis of the study and had primary responsibility for writing the manuscript.
Competing interests: None.
Funding: Pharmaceutical Education Program, Pharmaceutical Benefits Branch, Commonwealth Department of Health and Family Services.