The rise of regulation in the NHSBMJ 2002; 324 doi: https://doi.org/10.1136/bmj.324.7343.967 (Published 20 April 2002) Cite this as: BMJ 2002;324:967
- Kieran Walshe, Harkness fellow in health policy ()
- Correspondence to: K Walshe, Manchester Centre for Healthcare Management, University of Manchester, Manchester M13 9PL
- Accepted 20 September 2001
The current British government has created five national agencies to regulate the NHS. These new arrivals are simply additions to an already crowded regulatory landscape. But, if politicians can be persuaded to let go, the new regulators of the NHS could provide a genuinely new approach to improving performance and management
During the past four years the British government has created five new national agencies to regulate the NHS in England (box).1 2 3 4 The government has moved away from using markets, competition, and contracting to manage performance in the NHS. But it has been unwilling to rely on traditional bureaucratic structures to exert control, and has turned increasingly to regulation.
In this article, I describe how and why organisational regulation in the NHS in England has grown in recent years. I examine how regulation was used in the NHS in the past and describe the characteristics of the new regulatory agencies. Finally, I use information from the wider literature on regulation to examine the regulatory model adopted by these new agencies and to explore what they might learn from regulation in other settings.
Over the past 20 years, regulation has increased in Britain's private and public sectors
In the past four years the British government has created five new national agencies to regulate the NHS
The new agencies could be the start of a new approach to improving performance and management in the NHS
The new agencies could learn from regulators in other sectors and adopt a responsive approach to regulation
New national agencies to regulate the NHS
National Institute for Clinical Excellence
Commission for Health Improvement
National Patient Safety Agency
National Clinical Assessment Authority
Regulation is “sustained and focused control exercised by a public agency over activities which are valued by a community.”5 The key features of regulation are that it involves a third party—the regulator—in market transactions and interorganisational relationships and that it places responsibility for overseeing performance with a single entity—the regulator. Economists see regulation largely as a remedy for market failure. 6 7 However, as the definition suggests, regulation is also often used to achieve wider social goals—equity, diversity, or social solidarity—and to hold powerful corporate, professional, or social interests to account.8 Few areas of modern society have not been touched by regulation, but in many countries healthcare organisations are subject to quite intensive regulation.
Why is regulation increasing?
The recent rise of regulation in the NHS is part of the growth of the “regulatory state”9 or “audit society”10 in the British private and public sectors over the past 20 years. Between 1979 and 1997, despite being overtly committed to deregulation, the Conservative administration created a host of new regulatory agencies—many to oversee newly privatised industries.11 The Conservatives increasingly used regulation to manage the performance of public sector organisations—creating or strengthening regulatory agencies for schools, higher education, social care, and many other public services.12 The costs of regulating the public sector in the United Kingdom in 1995 were estimated to be between £770m and £1bn (about 0.3% of public expenditure). At least 135 different bodies regulated public sector organisations, and the cost of regulation doubled, or even quadrupled, between 1976 and 1995.13
Reasons for growth in regulation
Regulation of the public sector in the United Kingdom has grown in part because of changes to the management and structure of public sector organisations. The “new public management” gave organisations greater autonomy, placed them at arm's length from the government, separated purchasing and providing functions, and increased competition.14 These changes also brought new accountabilities and controls, including more regulation. Increasing regulation also results from a shift in how society holds public services to account—from a reliance on accountability through elected central and local governments to a desire for more direct and extensive oversight.15
In creating these new regulatory agencies the government may simply be seeking new ways to get things done in public sector organisations. Regulation is intended to be used alongside (not instead of) other mechanisms, such as traditional bureaucratic control and limited competition.16 From the government's perspective, the quasi-independent status of regulatory agencies distances politicians from difficult issues or unpleasant decisions. The responsibility for problems is shifted to the regulator, but the reach and scope of governmental control is retained, or even increased.
Existing regulation in the NHS
The NHS has a long and diverse tradition of regulation. Most NHS organisations already report to a host of bodies who regulate or review what they do and how they do it (table 1). Current regulators vary widely in their statutory authority, powers, scope of action, and approach. The resulting mosaic of regulatory arrangements is highly fragmented and some roles are duplicated. Most of the regulators shown in table 1 deal with a single facet of the NHS organisations that they regulate—health and safety, medical education, the administration of mental health legislation, etc—rather than the organisation as a whole. The regulators' demands on the NHS can conflict or overlap because arrangements for regulators to share information and systems to coordinate fieldwork are few and far between. Organisations in the NHS often complain of “inspectorial overload.”17 Even if several agencies had serious concerns about the performance of a particular organisation, it is unlikely that any one of them would be able to see the bigger picture of organisational failure.
The creation of five new agencies for regulating the NHS does nothing to resolve this confusion—the new arrivals are simply additions to the already crowded regulatory landscape. The remit and scope of the new regulators overlaps with those of existing regulators and each other, although the creation of an overarching body to coordinate the NHS's regulatory agencies has been proposed.18
The new regulators of the NHS
table 2summarises the characteristics of the five new regulatory agencies for the NHS in England. Not all see themselves as regulators—perhaps because the term has negative connotations—but they do all fit the definition of regulation.
In some respects, the new regulators are different from the existing regulatory agencies (table 1):
They are well resourced organisations, for whom regulation is the primary mission, rather than one function among many that they undertake
Their broad remit to oversee NHS organisations is not limited to particular service areas or functions, like that of many of the existing regulators
They are all essentially agents of the government—all are accountable to the Department of Health and have their boards appointed by the secretary of state. They have little independence and, taken together, represent a significant strengthening of central government's control of the NHS
Perhaps most importantly, these new regulators are all concerned primarily with the clinical quality of healthcare. Past regulation has often focused on more peripheral administrative and managerial matters, not on clinical practice.
Regulatory paradigm: deterrence or compliance
Regulatory agencies tend to subscribe to one of two paradigms of regulation—deterrence or compliance (table 3).19 The deterrence model assumes that the organisations being regulated are “amoral calculators”20 that put profit or other motivations before the public good, will ignore regulations if it pays them to do so, and so have to be forced to behave well by strict regulation, demanding standards, and tough enforcement. In comparison, the compliance model assumes that the organisations being regulated are fundamentally good hearted and well meaning, and they would generally do the right thing if they could. Regulators provide support and advice, and they are understanding and forgiving of lapses when they occur.
In practice, regulated organisations vary widely in their motivations. Although regulators sometimes adopt a mixed or differentiated approach that takes account of such variations, they are often forced by legislative, political, or other pressures to use a particular regulatory style, regardless of its appropriateness. In the past, regulators in the NHS (and in the United Kingdom generally) have adhered to a model of regulation largely based on compliance.21 However, the approach used in the public sector in the United Kingdom has been increasingly oriented towards deterrence.22 This change seems to be favoured by politicians, the media, and the public. The new healthcare regulators may face pressures to use a deterrence style of regulation, whether or not it is effective.23
Regulation and performance improvement: lessons for new NHS regulators
Most published work deals with regulation in settings outside healthcare24 25 26 or with general regulatory theory,27 28 29 but the amount of literature on healthcare regulation is increasing.30 31 32 What advice can this literature give to the new NHS regulators?
Firstly, it seems that effective regulation is “responsive”28—a regulatory agency should recognise and respond to the diversity of organisations it regulates, making its regulatory regime adapt to how individual organisations behave. The regulator eschews “one size fits all” policies—inspecting every organisation in the same way, at the same frequency, using the same standards—in favour of a more flexible and graduated approach that is constantly adapted to the content and outcome of each regulatory encounter. The deterrence and compliance models set up a false dichotomy—an effective regulator uses both approaches at different times, with different organisations, to meet different objectives.
Secondly, the responsive approach means that the regulator needs to have a wide range of regulatory interventions—both incentives and sanctions—and to use them appropriately (figure). Most regulatory activity should take place at the lower levels of the pyramid—the methods at the top should need to be used only rarely, but they need to be available to sustain the integrity and credibility of the regulatory system. Regulators should be able to move freely up and down the hierarchy of methods. Legislation or policies that force regulators to use particular methods or levels are likely to reduce their effectiveness.
Thirdly, effective regulators work with and through other stakeholders in the organisations they regulate, rather than treating the relationship between them and the organisation as bilateral. This approach—tripartism—involves designing regulatory methods to involve groups such as patients, consumers, staff, and partner organisations in regulation. Regulators may seem to be powerful, but they actually have very limited resources, and they can never oversee more than a tiny proportion of regulated activities. Tripartism allows them to extend their oversight by using other stakeholders as informants and agents for change.
Finally, regulatory agencies need to balance independence and accountability. Regulators need to be independent to maintain their credibility, to allow them to act impartially as an “honest broker” when they need to, and to enable them to take actions that may be unpopular with some stakeholders. On the other hand, regulators should be held accountable for what they do and for the effects of regulation, and all stakeholders need to be involved, to some extent, in accountability arrangements to avoid regulation being captured by one group or another. For regulatory agencies that are part of government, a mechanism for distancing them from their political masters, while still maintaining appropriate accountability, is needed.
The rise of regulation in the NHS seems, at first sight, to represent a long term strengthening of central government's control of managerial and clinical practice. The new regulators are government agencies, headed by ministerial appointments. The Department of Health provides their budgets and enforcement powers. Their creation could allow a kind of centralised micromanagement, in which there is less and less scope for local variation, to develop.
However, if the politicians can be persuaded to let go, the new regulators of the NHS could provide a genuinely new approach to improving performance and management. A longstanding tradition makes ministers directly accountable for everything that happens in the NHS; this could be replaced by a more indirect and distanced relationship, in which managed regulation by intermediate and quasi-independent organisations would play a large part. In that environment, the new regulatory agencies could adopt a more responsive approach to regulation—learning from the experience of regulation in other settings and focusing regulation on delivering real improvement for patients.
Funding KW was supported by the Commonwealth Fund, a private independent foundation based in New York City. The views in this article are those of the author and not necessarily those of the Commonwealth Fund, its director, officers, or staff.
Competing interests None declared