Primary care

Attitudes to the public release of comparative information on the quality of general practice care: qualitative study

Martin N Marshall, professor of general practice ^a, Julia Hiscock, senior researcher ^b, Bonnie Sibbald, professor of health services research ^a. 

^a National Primary Care Research and Development Centre, University of Manchester, Manchester M13 9PL, ^b National Centre for Social Research, London WC1V 0AX

Correspondence to: M N Marshall martin.marshall{at}man.ac.uk

	Abstract

Top Abstract Introduction Methods Results Discussion References

Objectives: To examine the attitudes of service users, general practitioners, and clinical governance leads based in primary care trusts to the public dissemination of comparative reports on quality of care in general practice, to guide the policy and practice of public disclosure of information in primary care.
Design: Qualitative focus group study using mock quality report cards as prompts for discussion.
Setting: 12 focus groups held in an urban area in north west England and a semirural area in the south of England.
Participants: 35 service users, 24 general practitioners, and 18 clinical governance leads.
Results: There was general support for the principle of publishing comparative information, but all three stakeholder groups expressed concerns about the practical implications. Attitudes were strongly influenced by experience of comparative reports from other sectorsfor example, school league tables. Service users distrusted what they saw as the political motivation driving the initiative, expressed a desire to "protect" their practices from political and managerial interference, and were uneasy about practices being encouraged to compete against each other. General practitioners focused on the unfairness of drawing comparisons from current data and the risks of "gaming" the results. Clinical governance leads thought that public disclosure would damage their developmental approach to implementing clinical governance. The initial negative response to the quality reports seemed to diminish on reflection.
Conclusions: Despite support for the principle of greater openness, the planned publication of information about quality of care in general practice is likely to face considerable opposition, not only from professional groups but also from the public. A greater understanding of the practical implications of public reporting is required before the potential benefits can be realised.

What is already known on this topic Disclosure of information about quality of care in the NHS has been strongly influenced by the report card movement in the United States This was based largely on hospital data, with no evidence to determine the attitudes of the British public to the publication of quality reports in general practice What this study adds The public and health professionals are in favour in principle of publishing information about quality in general practice but are concerned about the consequences for themselves, the practices, and the health system People regard public disclosure as a political initiative and are more inclined to trust their own experience or that of friends and family than to trust comparative data General practitioners perceive comparative reports as a burden, and clinical governance leads are concerned that the reports might damage their facilitative approach to improving quality

	Introduction

Top Abstract Introduction Methods Results Discussion References

The dissemination of reports comparing the quality of care provided by healthcare institutions and individual professionals represents an international trend and a central component of UK government plans for the reform of the NHS.¹ So called report cards in primary care are an explicit government policy, influenced noticeably by what has happened in the United States.² There is a dearth of evidence to guide the policy and practice of reporting on primary care in the United Kingdom.

We examined the attitudes of the key stakeholdersservice users, general practitioners, and quality improvement clinical managers based in primary care truststo the public dissemination of comparative information on general practice performance and compared this with evidence from the United States.

	Methods

Top Abstract Introduction Methods Results Discussion References

We conducted 12 focus groups; four each of service users, general practitioners, and clinical managers based in primary care trusts, the so called clinical governance leads. For each stakeholder group, half of the groups were held in the north west of England, centred on a high density urban area, and half were held in a rural or semirural locality on the south coast of England. The participants were selected to reflect a broad range of personal, geographical, and organisational characteristics (table).

Initial discussion was broad, exploring general views about the provision of comparative information in non-health sectors as well as health sectors. Following this general discussion, a mock report card was presented to the participants to stimulate and focus discussion (see bmj.com).

The results of the earlier groups were fed into the later groups, and three of the early groups were reconvened to encourage the participants to reflect on and to develop their own views about the issues discussed. The discussion was audiotaped, fully transcribed, and analysed using a computer assisted method ("framework") that facilitates both thematic analysis and case by case analysis and tracks both individual and group comments.³

	Results

Top Abstract Introduction Methods Results Discussion References

Four major themes emerged from the data: a difference between the initial reaction and the considered response to the report cards, the usefulness of the data to the key stakeholders, immediate concerns about the principle and practice of report cards, and the wider implications of disseminating comparative information.

Initial versus considered response
The initial reaction both to the idea of performance reports in general practice and to the mock report cards was strongly negative. The dominant feeling, expressed particularly strongly by the service users and general practitioners, was that such reports were unnecessary, unfair, and unwanted. The considered response from all three groups, however, was more positive. It seemed that the initial negative response was based on concerns about the practical problems and consequences of disclosure:

I've got nothing against it in principle. It's purely the practical outcome, the practical consequences of it. The way the press will use it. The way the government will use it. All to fulfil their personal agenda . . . They will use the information as suits them best and the welfare of the health service will not matter one iota. (General practitioner, male, large semirural practice, 20 years' experience)

The considered response, however, was based on matters of principlethat data on performance are important and useful to service providers, that if information is known then it is only right that it should be in the public domain, and that if it is made public then it would be inevitable and useful for it to be presented in such a way that allows meaningful comparisons between organisations.

Usefulness of data to service users
Most of the service users dismissed the idea of using report cards to select the "best" practices. For some of them this represented a preference for geographical convenience, for some a perception that they were not encouraged to exercise choice, and for others a view that they did not want to behave in a consumerist fashion as far as health care was concerned.

In general, however, the unwillingness to exercise choice related to the level of confidence that they had in the comparative information. Even if the data suggested that their own practice or doctor was substandard, they placed greater trust in their own experience or that of friends and family:

If I saw my own doctor being slagged off in the Good Doc Guide, I'd still go to him because personally he suits me and I've got faith in him, because I would know from my own personal experience. (Service user, female, over 60 years, rural area)

The data were given credence by service users in only two situations; when the results confirmed established views about performance and when informal sources of information were absent, such as when patients moved into a new area.

Immediate concerns about principles and practice of public reporting
The immediate concerns about report cards focused on the perception of a political motivation behind reporting, the issues of data quality, and the impact on professional morale and behaviour.

Cynical views were expressed by all of the stakeholder groups, particularly the general practitioners, about the politicians' desire to exert control over doctors, to get them to focus on the narrow areas of practice in the reports, and to use the data to serve political ends:

I'm very sceptical of figures and things like that, percentages, they can make them do what they want. They can manipulate them, they can doctor anything, can't they? (Service user, male, 41-60 years, rural area, 25 years with same general practitioner)

I suspect that it is a way of undermining the status of doctors in the eyes of their patients. (General practitioner, male, large rural practice, 20 years' experience)

The service users expressed a strong desire to protect their general practitioners from this political interference. Many of the general practitioners and service users thought that report cards were an attempt to shift the responsibility for performance from the government to the providers. Service users were particularly concerned that report cards would herald competition between practices:

You're trying to get them going against each other, aren't you? It's like competing, isn't it? (Service user, female, 18-40 years, urban area, 11 years with same general practitioner)

They did not think this desirable, and they were concerned that the "winners" would be those who were able to "play the game," rather than those with genuine good performance.

Concerns about data quality in general practice were expressed by the general practitioners and clinical governance leads. They also expressed doubts that the most important aspects of general practice were amenable to measurement and reporting:

Something that's measurable may not be worth measuring, and maybe you can't measure the things that are worth measuring. What damage do you do by releasing information just because you can measure it? (Clinical governance lead, male, general practitioner background, rural area)

General practitioners and clinical governance leads in particular were concerned about the impact of public disclosure on stress, morale, and job satisfaction of general practitioners. They saw report cards as another burden at a time of major stress for doctors.

All three stakeholder groups expressed concern that general practitioners would distort their behaviour to improve their reported performance. Service users focused on the risk of general practitioners preferentially registering patients who made their figures look good, whereas the clinicians admitted that report cards might change clinical behaviour.

The clinical governance leads supported the use of comparative information for internal purposes. They did, however, express concern that the public release of the information would encourage a "name and shame" culture in general practice and that this would run counter to their developmental and supportive approach to implementing clinical governance:

We'll get cover-ups, we'll get further entrenched in our blame culture and away from the culture where we can say "actually, I made a complete cock-up of that." We're trying to get to a stage where that can be discussed openly, but if we have to put all (this) stuff into the public domain, we won't. (Clinical governance lead, male, general practitioner background, urban area)

Wider implications of comparative reports
Even though most service users doubted that they would change practice themselves on the basis of the information, they expressed concern that others would do so and that this would result in the "good" practices being swamped. Both service users and general practitioners feared that performance reports would exacerbate inequalities because better educated and more articulate patients would use the information to select high performing practices, whereas the less educated and more vulnerable patients would be left with "ghetto" practices.

Several general practitioners and some service users expressed concern about the impact of the publication of comparative information on the relationship between patients and their doctors. They were worried that the data might undermine the patient's confidence and lead them to question past diagnoses and treatments.

	Discussion

Top Abstract Introduction Methods Results Discussion References

The production and dissemination of comparative quality reports represents a major policy commitment for the NHS. We found that although all of the key stakeholder groups shared this commitment in principle, there were considerable concerns about the practical processes and consequences of implementing this initiative in general practice.

It is unclear whether this opposition will be sustained or whether it is just a question of time before all stakeholder groups engage in the process. It is perhaps inappropriate to expect members of the public in the United Kingdom to suddenly behave like rational consumers, making judgments about relative performance and refusing to access apparently poor practices. Although it is possible that the better informed and more empowered citizens of the future will make greater demands for information, some people might always view objective data as less relevant and less meaningful than informal sources of information. Nevertheless, there is evidence that both public and professional views of comparative reports become more positive over time.⁴

Public attitudes to comparative information have received little attention in the United Kingdom. The only example that we could find suggests that service users show little interest in hospital data.⁵ In the United States the public are more positive about the provision of information, although they seem to make little practical use of it.⁶ One of the differences between public reporting in the United States and current initiatives in the United Kingdom is the source of the reports. Early reporting systems in the United States, led by the federal government, engendered a similar abreaction from the key stakeholders, whereas more recent initiatives representing coalitions of interest groups have been better received. ^{7 8} It is therefore possible that non-governmental initiatives in the United Kingdom, such as the reports produced by the Dr Foster group⁹ and the planned release of comparative data by the Commission for Health Improvement, might be seen in a more positive light than initiatives led by the Department of Health.

The willingness of both the professional and the lay participants to consider the wider implications of comparative reports suggests an unwillingness to destabilise the system by refusing to access apparently poorly performing organisations. If this is the case then report cards in the United Kingdom are being introduced in a different context from that in the United States. This interpretation implies that current expectations of report cards in the United Kingdom should focus more on their potential to improve the accountability and quality of the service and less on consumer empowerment.

Our findings are limited by the methodology and should be interpreted within the context of the current environment in the NHS. The extent to which the wider population holds the views expressed by the focus group participants is unknown. Much discussion on health policy in the United Kingdom is influenced by what is happening in the United States and predicated on the assumption that a consumerist approach to health care will drive quality improvement. We, however, found little support for this view. In addition, examining attitudes to a future initiative inevitably requires a degree of speculation, and it is possible that the attitudes would have been different if the views were based on real experiences of using report cards.

These findings should not derail an initiative that has the potential to improve accountability and stimulate improvements in quality. However, the technical barriers, the antipathy of the general public, the impact on professional morale, and the opportunity costs of focusing on public reporting at the expense of other health service reforms, should not be discounted. Policy makers, managers, and health professionals should understand these barriers, recognise the limitations of directly transferring experience from the United States, and ensure that the implementation of public reporting in the United Kingdom is guided by relevant evidence.

	Acknowledgments

Contributors: See bmj.com

	Footnotes

Funding: This study was funded by the UK Department of Health through core support for the National Primary Care Research and Development Centre, University of Manchester. The views expressed in the paper represent those of the authors and not necessarily those of the funding body.

Conflicts of interest: None declared.

This is an abridged version; the full version is on bmj.com

The mock pilot card appears on bmj.com

	References

Top Abstract Introduction Methods Results Discussion References

1.	Department of Health. The NHS plan: a plan for investment, a plan for reform. London: Department of Health, 2000.
2.	Marshall MN, Shekelle PG, Leatherman S, Brook RH. Dying to know: public release of comparative data in health care. Nuffield Trust: London, 2000.
3.	Ritchie J, Lewis J. Qualitative research practice. London: Sage (in press).
4.	Schneider E, Lieberman T. Publicly disclosed information about the quality of health care: response of the US public. Qual Health Care 2001; 10: 96-103[Abstract/Free Full Text].
5.	Mannion R, Goddard M. The impact of performance measurement in the NHS. Report 1: Empirical analysis of the impact of public dissemination of the Scottish Clinical Resource and Audit Group data. York: Centre for Health Economics, University of York, 2000.
6.	Marshall MN, Shekelle PG, Leatherman S, Brook RH. What do we expect to gain from the public release of performance data? A review of the evidence. JAMA 2000; 283: 1866-1874[Abstract/Free Full Text].
7.	Vladeck BC, Goodwin EJ, Myers LP, Sinisi M. The HCFA "death list." Health Aff 1988; 7: 122-125[CrossRef][Medline].
8.	Epstein AM. Public release of performance data: a progress report from the front. JAMA 2000; 283: 1884-1886[Free Full Text].
9.	Dr Foster. Dr Foster good hospital guide. London: Vermilion, 2002.

(Accepted 27 August 2002)