Profiling performance in primary care in the United StatesBMJ 2003; 326 doi: https://doi.org/10.1136/bmj.326.7392.744 (Published 05 April 2003) Cite this as: BMJ 2003;326:744
- Norbert Goldfield, medical directora,
- Shamini Gnani, specialist registrarb,
- Azeem Majeed, professor ()b
- a 3M Health Information Systems, Wallingford, CT 06492, USA
- b Primary Care Research Unit, School of Public Policy, University College London, London WC1H 9QU
- Correspondence to: A Majeed
Purchasers of health care in both the United States (governments, employers, health plans) and the United Kingdom (government) need to be able to measure the quality of services they are paying for.1 Moreover, public concerns about the variable quality of health services have increased in both countries. Measuring the performance of primary care physicians and healthcare providers is one method of meeting these challenges. 2 3 We review the development of this approach (commonly termed “profiling”) in the United States.
Physicians' performance is increasingly being profiled in the United States to release performance data to the public and make routinely collected data available to healthcare purchasers and regulators
The United Kingdom is likely to follow suit
To justify the burden and costs of profiling, close collaboration between physicians, healthcare organisations, and other stakeholders is needed
The performance measures used in profiling need to be standardised; duplication of effort needs to be minimised; and the objectives, measures, and methods used need to be transparent
Linking of hitherto disparate data elements such as diagnoses, pharmacy data, and laboratory results will increase the sophistication and coverage of physician profiling
Physician or provider profiling is an attempt to measure the performance of doctors and providers of health care by supplying interested parties with information on the structure, process, and outcomes of health care.4 Its rationale is that analysing patterns of care will help to reduce the variation in performance among doctors and lead to improvements in the quality of health care.5 Two main types of profiling are used in the United States. Clinical profiling examines doctors' styles of practice by looking at the types of treatment and services that they use and the outcomes of care. Economic profiling examines the financial aspects of a doctor's practice. Researchers in the United States have now developed many sophisticated tools for profiling physicians (see bmj.com).
Ideally, profiling should provide doctors with meaningful information on their clinical performance to help improve the quality of the services they provide. However, purchasers have largely developed physician profiling as a tool to control costs and ensure they are getting value for money, rather than as a method of measuring and improving the quality of care. Profiling has also been used for other purposes, including providing information to consumers to help in their selection of physician and healthcare plan (box 1).
One of the criticisms of profiling made by US physicians is its emphasis on measuring and reporting patterns of use of resources and costs of care. Doctors' professional bodies would like profiling to focus on measuring doctors' clinical performance. In contrast, health plans and regulatory bodies wish to extend profiling beyond cost data to include information such as patient satisfaction surveys and medical liability claims. Unfortunately, for a number of reasons (including the American Medical Association's focus on a system for profiling physicians that has now been abandoned), many doctors have been reluctant to become involved in developing profiling, and as a result its development has largely been determined by purchasers in both public and private sectors. Furthermore, most of the physician profiles that are currently used in the United States do not meet the ideal criteria for profiles (box 2).
Box 1 : How has physician profiling been used in the United States?
To provide physicians, medical practices, and hospitals with information on the cost and quality of their care
To provide patients with information to guide their choice of doctor and to assess the quality of services they are receiving
To provide health plans and health maintenance organisations with information to help them plan and monitor the utilisation, quality, network management, and accreditation of health services
To provide employers and regulatory agencies with information to guide their decisions on issues ranging from publication of malpractice information to the choice of medical groups to serve in their networks
Box 2 : Ideal physician profiles
Are of interest to as many consumers of physician profiles as possible
Have a firm scientific basis while recognising that much of clinical medicine is still an art and contains many controversies
Meet certain statistical thresholds of validity and reliability
Involve the providers in developing and implementing the profiles
Cost as little as possible to produce, with benefits outweighing costs of production
Respect patients' confidentiality and obtain their consent to use of information from their medical records or similar sources
Are adjusted for risk to allow for differing severity of illnesses
Reflecting the structure of the US healthcare system, most of the organisations involved in profiling are private sector organisations. One of the most widely used profiling systems in the United States is the health plan employer data and information set (HEDIS), developed by the National Committee for Quality Assurance. In turn, business groups, such as the Pacific Business Group on Health, often release profiling information on specific medical groups to the public. Other sources of information for profiling include medical records, clinical information systems, and patient surveys. For example, a standardised survey of health plan members, the consumer assessment of health plans, provides comparative data on health plans and is financed by the federal government.6 Although the federal government may finance the development of such instruments, their implementation has been left largely to private sector organisations.
Collaborative efforts between clinicians represent another model, and one that may have particular relevance for the United Kingdom. Led by the Maine Medical Assessment Foundation, several organisations now provide mechanisms for clinicians not only to receive profiles but also to participate in their development and improvement.7 Although these initiatives may lead to the development of more robust profiling, led by physicians in various specialties, securing funding may be difficult. For example, the project in Maine has now ended because it was unable to obtain sufficient support from doctors' professional bodies or from healthcare purchasers.
Such quality improvement programmes are likely to succeed, therefore, only when consumers are interested in the association between payments to healthcare providers and quality of care. The American Medical Association's physician led profiling programme foundered in 2001 because of cost over-runs and a questionable choice of profiling variables. The association is currently working with specialist societies to refocus its profiling efforts by specialty.
Some states and organisations in the United States have now started to make physician profiles available to the public. 8 9 The information published ranges from doctors' basic details—education, years in practice, any history of malpractice, etc—to patients' evaluations of physicians. This has generated considerable controversy about, for example, the scientific validity of listing malpractice claims (not verdicts) against physicians and the impact of releasing information on patient satisfaction with medical groups.10 Despite this, there have been calls to increase the amount of publicly available information by releasing all relevant clinical information on physicians' performance, together with data on mortality, patient satisfaction, and other important characteristics of clinical practice.
Public disclosure of comparative information on managed care organisations and hospitals is now well established in the United States, but until relatively recently data on individuals or small groups of physicians have not been released. Large organisations, such as hospitals and managed care organisations, have the resources to help deal with the adverse publicity that sometimes follows the release of profiling information, but physicians working alone or in a group practice may to have to deal with the media or with concerned patients.11
Impact of public release
Patients sometimes use the information contained in profiles to change their health plans or providers, including their medical groups.12 However, a number of obstacles, ranging from the need to make information easier to read to the targeting of variables of interest to particular subgroups of consumers, continue to limit involvement of consumers in using profiling data.13 Also, patients often are also more interested in the process of care (what will be done to them) than in the outcomes of care (what will happen to their health).14
The evidence for the effect of public disclosure programmes on the processes and outcomes of care is limited to observational studies. Most report improvements in care after state-wide or local programmes of public reporting have been introduced, but whether these changes were the result of the disclosures or due to other factors is not known. Public disclosure does seem to alter providers' behaviour, because providers may feel that their reputations are at stake or open to public scrutiny. Furthermore, while profiling may represent an opportunity to market their excellence, it may also encourage physicians to avoid more seriously ill patients.15
Despite the drawbacks, pressure for the public release of information about physicians will continue. Hence, the public disclosure of data on particular physicians will increase further, in spite of the unwillingness of the medical profession to participate in developing these profiles. Although researchers have often criticised particular profiling methods, this has not prevented the results being released to the public. However, because of the relative lack of interest in physician profiling among professional bodies, compared with their interest in other issues such as malpractice reform and the level of government payments to physicians, the pressure for public dissemination of profiling information is coming from the private sector.
Implications for the United Kingdom
Patterns of care in general practice in the United Kingdom vary widely, as does the quality of practices' clinical information systems.16 The performance of general practitioners is already to some extent being measured to identify such variations (figure). It is unclear from US experience whether physician profiling will help to reduce wide variations in practice or lead to improvements in the quality of care. One reason for this is that little effort has been made to use profiles to change medical practice. The most important lesson we can glean from the US experience is that simply releasing information to the doctor will accomplish little. Release of profiling information must be followed up with intensive efforts to work with the members of the primary healthcare team to change the processes of care. Furthermore, as in the United States, the clinical and sociodemographic characteristics of patients have an important influence on performance. 17 18
Box 3 : Research questions in development of profiles
In what areas of performance are physicians, the public, purchasers, and other users of profiling data interested?
What are the best methods of involving physicians and other users in the development of physician profiles?
What methods of producing reliable and valid physician profiles at reasonable cost can be developed?
What educational techniques can be used to enable all users of profiling data, including physicians and the public, to use them to improve the quality and appropriateness of clinical care?
How can adjustment be made for clinical demographic and case mix variables when measuring performance and profiling physicians?
Another important lesson is that different patient groups do not benefit equally from the public release of profiling data. Vulnerable groups such as poor, less educated, or chronically sick people and members of minority ethnic groups are least likely to make use of these data.19 Hence, the groups with the greatest need for health care make least use of performance data to guide their decisions about their use of health services. Presenting profiling data in ways that vulnerable groups can make meaningful use of will be a major challenge for the NHS.
Finally, the range of data and the tools for measuring performance have increased substantially in recent years, but these improvements have not been used systematically to reduce the important variations in the US healthcare system. Physicians are often provided with profiles, but neither they nor purchasers generally use them to change clinical practice. It has always been much easier for purchasers of health care to pass on the inexorably increasing costs of care to consumers, employers, and government, the people and organisations who pay for health care. In contrast, trying to change physicians' practice requires a considerable, sustained effort, and the benefits are more long term.
In the United States, the development of physician profiling is mainly being influenced by the rapidly increasing costs of health care and purchasers' need to know they are getting value for money. Two other factors also have an important role: the pressure for profiles to be released externally; and developments in health informatics, leading to the integration of information systems, a considerable reduction in the costs of producing profiles, and a simultaneous increase in their sophistication.
Several policy considerations will guide the development and implementation of profiling in the United States. Firstly, although the American Medical Association has backed off from its ambitious plans to profile physicians' practice, specialists' societies have become more interested in this area. Secondly, profiles will need to have more scientific validity and will have to make use of new sources of data. In particular, the links with pharmacy and administrative data will soon lead to links with other important data elements, including information derived from patients and from laboratory results.
In Britain the pressures are in general similar to those in the United States, but the government also plans to introduce new regulatory and reaccreditation procedures. The United Kingdom may be able to take advantage of the tools that have been useful in profiling in the United States, and the NHS has set out an ambitious strategy for developing integrated health records, which could also be used in physician profiling. In both countries, the areas of clinical and professional activity covered by physician profiling are likely to be extended further. Finally, the US experience suggests that many patients, particularly those from disadvantaged groups, will find it difficult to make use of profiling data. Hence, consumer and patient groups, rather than individual patients, may well be the main targets of profiling for public use in the United Kingdom.
Although many questions about physician profiling remain unanswered (box 3), doctors in both countries need to work with governments and with purchasers of health services to ensure that well conceived physician profiles will result in useful improvements in care.20 Otherwise, profiling may be developed mainly as a punitive and regulatory mechanism (for example, to “name and shame” individual doctors), resulting in a further deterioration in doctors' morale in both countries.
This is the third of four articles in a series edited by Andrew Bindman and Azeem Majeed
Funding No specific source of funding.
Competing interests NG is an employee of the 3M Corporation and has developed some of the public domain and proprietary tools used in profiling physicians in the United States and in other countries. AM holds a national primary care scientist award funded by Department of Health, and has received funding for comparative research on primary care in the USA and UK from the Commonwealth Fund of New York.
A list of relevant websites appears on bmj.com