External assessment of health care

Common approaches

Many countries have voluntary and statutory mechanisms for periodic external assessment of healthcare organisations against defined standards, and some have been systematically compared.1^–3 They are all meant to assure or improve some elements of quality, but they are usually run by different organisations without national coordination to make them consistent, mutually supportive, economical, and effective. Broadly, these mechanisms include variants on five approaches (box 2).

The International Organization for Standardization provides standards against which organisations or functions may be certificated by accredited auditors. These have been applied in health care, specifically to radiology and laboratory systems, and more generally to quality systems in clinical departments.4

The Baldrige criteria have evolved into national and international assessment programmes such as the Australian Business Excellence Model (www.aqc.org.au/) and the European Foundation for Quality Management (www.efqm.org/).5

Peer review is based on collegiate, usually single discipline, programmes to assess and give formal accreditation to training programmes but is now also extended to clinical services.6

Accreditation relies on independent voluntary programmes developed from a focus on training into multidisciplinary assessments of healthcare functions, organisations, and networks. These have spread from Western countries into Latin America,7 Africa,8 and South East Asia 9 10 during the 1990s. Mandatory programmes have recently been adopted in France,11 Italy,12 and Scotland.13

Registration and licensing are statutory programmes to ensure that staff or provider organisations achieve minimum standards of competence. There are also inspectorates for specific functions to ensure public health and safety.

National requirements

Several countries have recently received recommendations on their ability to ensure high standards in health care nationally. The general conclusions on the role of external agencies have been remarkably similar.

The US president's advisory commission on consumer protection and quality in health care recommended in 1998 that public and private programmes of external review should make their standards, survey protocols, decision criteria, and results available to the public at “little or no cost.”14 The organisations themselves should work towards a common set of standards, coordinate their activities to avoid conflict and duplication, and commit themselves to a national quality forum. This forum aims to devise a national strategy for measuring and reporting healthcare quality and in 1999 began to standardise performance measures for the nation's 5000 acute general hospitals.15

In 1999 the US inspector general of the Department of Health reviewed the external quality oversight of hospitals that participate in Medicare.16 She concluded that voluntary “collegiate” accreditation by the Joint Commission on Accreditation of Healthcare Organisations and “regulatory” Medicare certification by state agencies had considerable strengths (box 3) but also major deficiencies. She recommended that both systems should harmonise their methods, disclose more details of hospital performance on the internet, and be held more fully accountable at federal level for their performance in reviewing hospitals.

An Australian taskforce recommended in 1996 that the government should formally acknowledge independent assessment programmes that met defined criteria and should enable them to disseminate information about their processes and findings to the public.17 Two years later an expert advisory group recommended “that accreditation or certification of healthcare organisations be strongly encouraged with incentives, or indeed made mandatory, but choice of accreditation/certification/award approaches be allowed.”18

In Scotland the Carter report on acute services recommended a single mandatory system of accreditation for hospitals and primary care.19 This should be patient centred, clinically focused, and complementary to internal quality improvement, and its explicit, measurable standards and reports should be in the public domain. This recommendation led to the Clinical Standards Board for Scotland.

International solutions

Countries have good reasons to be able to show that healthcare standards are not only consistent within their own territory but also that they are comparable with those of their neighbours, suppliers, and competitors. Several recent European and international initiatives are making traditional assessment methods more accessible, convergent, and relevant to health care.

International Organization for Standardization—The ISO 9000 series of standards were designed for manufacturing industries and have been criticised for using language that is difficult to interpret in terms of health services. The 2000 version will be more readily applied, and US and European initiatives are under way to develop ISO guidelines specific to health care.

European Foundation for Quality Management—The original “business excellence” model has given way to “excellence” in the 1999 version and has shifted emphasis from “enabling processes” to results of concern to patients, staff, and society

Accreditation—The international arm of the US Joint Commission on Accreditation of Healthcare Organisations has developed a set of multinational accreditation standards.20 In addition the International Society for Quality in Health Care has developed (”ALPHA”) standards and criteria (available from the society's website http://www.isqua.org.au/) against which an accreditation programme may apply to have its standards and process assessed and internationally accredited.21 These also offer a template for standardisation and self assessment to any external assessment programme.

Programmes in Britain

The royal commission on the NHS recommended in 1979 that a special health authority be set up as a development agency and guardian of standards.22 In the early 1980s several monitoring agencies were suggested or piloted,23 but, despite favourable response from national professional bodies to leaked proposals, no such national agency featured in the government's white paper of 1989 Working for Patients.24

In the absence of any governmental lead, several small peer review and (some large) accreditation programmes emerged as external voluntary mechanisms for organisational development. There are now over 35 such programmes with a wealth of standards and trained assessors but little integration, consistency, or reciprocity between them. Their number could be doubled if each royal college, faculty, and professional association were to establish independent accreditation programmes as a collegiate approach to clinical governance. NHS institutions also have their share of visits from clinical training programmes, inspectors (such as for fire regulations, environmental health, etc), and other watchdogs that have begun to publish standards (such as the NHS Information Authority Information Management Centre for data quality and NHS Controls Assurance for risk management and controls assurance).

The Clinical Standards Board for Scotland and the National Institute for Clinical Excellence (NICE), and Commission for Health Improvement (CHI) for England and Wales have been established to improve standards in the NHS. After years of policy vacuum, an early common task must be to tidy up: they must synthesise the experience of Britain and other countries25; provide public access to their own valid standards, reliable assessments, and fair judgments; and, above all, avoid duplication and inconsistency in defining and measuring standards. In short, they should be open to assessment against international criteria and lead the way to consistency and reciprocity within and between systems for improving patient services, clinical training, and public accountability.

Britain could borrow from the US and Australian recommendations for partnership between state and independent programmes for external assessment and define the terms of collaboration. Independent and statutory programmes could be jointly assessed and harnessed according to general criteria drawn from UK policy and experience overseas and from the more specific ALPHA standards.

We need to catalogue, harmonise, and orchestrate organisational standards and their assessment, not only in the NHS but also in the independent and social care sectors. The National Institute for Clinical Excellence has a clear responsibility for defining clinical standards in England and Wales. The Commission for Health Improvement is concerned with the organisation and delivery of clinical governance and national service frameworks, but it has no mandate to define or orchestrate organisational standards (even for its own reviews), and it is specifically excluded from the independent sector. In Scotland the Clinical Standards Board integrates some key features of these two bodies, particularly the task of defining and measuring standards, both clinical and organisational. With yet broader vision, the Scottish Executive has adopted a charter that sets out principles for public and professional inspectorates whose role includes evaluation of cases in the public interest, including health, education, and social work services (www.scotland. gov.uk). This offers a starting point for coherence and learning within and between sectors, and an example for the rest of Britain.

The UK Accreditation Forum (http://www.caspe.co.uk/) was set up in 1998 to support accreditation and peer review programmes, and the Academy of Medical Royal Colleges (www.aomrc.org.uk/) is working towards more coherent procedures for hospital visiting for recognition of training. Neither body has the resources or the authority to standardise standards or to regulate the regulators across the country.

What we need is a formal means to pool current experience, to drive convergence, and to help new programmes to be efficient, complementary, and effective—a resource centre to do for organisational and management standards what NICE, the Cochrane Centre, and the Scottish Intercollegiate Guidelines Network are doing for clinical practice. Its task should be to ensure that organisational standards, assessments, and general results are in the public domain; that the legitimate interests of the public, professions, providers, and funding bodies are balanced and supported; that lessons from successes and failures are systematically embedded in common core standards for assessment; that assessment methods and reporting are consistent in time, place, and service; and that expenditure on the development and operation of external assessment programmes is demonstrably justified by improvements in patient care.

Conclusions

Schemes for inspection, registration, revalidation, and review are proliferating with little national coordination or regard for the evidence of what has worked or not worked for health care in Britain or overseas. This leads to uncertainty among service providers about which standards to adopt, inefficiency in developing new inspection and development programmes, duplication and inconsistency of external assessments, and an excessive burden on the services under scrutiny. The collegial and statutory mechanisms need a public-private partnership, perhaps similar to the National Quality Forum in the United States, to bring clarity, consistency, and transparency to external assessment in Britain.