- J Chard, senior audit coordinator1,
- M Kuczawski, audit coordinator1,
- N Black, professor of health services research2,
- J van der Meulen, professor of clinical epidemiology12
- on behalf of the POiS Audit Steering Committee
- 1Clinical Effectiveness Unit, Royal College of Surgeons of England, London WC2A 3PE, UK
- 2Department of Health Services Research and Policy, London School of Hygiene and Tropical Medicine, London
- Correspondence to: J van der Meulen, Department of Health Services Research and Policy, London School of Hygiene and Tropical Medicine, London WC1H 9SH
- Accepted 3 October 2011
Objective To compare characteristics of patients and outcomes after elective surgery in independent sector treatment centres (ISTCs) and NHS providers.
Design Follow-up study with outcomes reported by patients three to six months after surgery.
Setting 25 ISTCs and 72 NHS providers in England.
Population Consecutive patients undergoing hip or knee replacement (5671 in ISTCs and 14 292 in NHS), inguinal hernia repair (640 and 2023, respectively), or surgery for varicose veins (248 and 1336, respectively).
Main outcomes Symptoms and disability reported by patients (Oxford hip and knee scores on a 48 point scale; Aberdeen varicose vein questionnaire) and quality of life (EuroQol EQ-5D score).
Results Patients in ISTCs were healthier than those in NHS providers, had less severe preoperative symptoms, and were more affluent, though the differences were small. With adjustment, patients undergoing joint replacements in NHS providers had poorer outcomes: difference of −1.7 (95% confidence interval −2.5 to −0.9) on the Oxford hip score and −0.9 (−1.6 to −0.2) on the Oxford knee score. They more often reported complications: odds ratio 1.3 (95% confidence interval 1.1 to 1.5) for hip and 1.4 (1.2 to 1.6) for knee. There were no significant differences in outcomes after surgery for hernia or varicose veins, except that NHS patients more often reported poor results after hernia repair (1.4, 1.0 to 1.9) and additional surgery after varicose vein surgery (2.8, 1.2 to 6.8).
Conclusion Patients undergoing surgery in ISTCs were slightly healthier and had less severe conditions than those undergoing surgery in NHS providers. Some outcomes were better in ISTCs, but differences were small compared with the impact ISTCs could have on the provision of elective services.
Independent sector treatment centres (ISTCs), operated by private companies, provide elective diagnostic and surgical treatments for the English National Health Service (NHS).1 The ISTC programme was created in 2002 as an extension of an existing NHS treatment centre programme that began in 1999. The idea behind the introduction of treatment centres was that the efficiency of elective care would improve if routine treatments were separated from complex and emergency care.2 The aim was not only to reduce waiting times but also to give patients a greater choice of provider for their treatment and to improve outcomes. The main justification for using private companies rather than continuing with an NHS run model was the need to rapidly expand services, which could not solely be met by the NHS.3
Two phases of ISTC procurement have taken place. The first phase started in 2002 and focused on creating new capacity. By December 2009, 27 ISTCs were operational and two were under construction. The second phase focused mainly on using existing capacity within the independent sector and started procurement in 2005.
While the concept of establishing treatment centres to increase capacity was welcomed, the use of private companies and “overseas” surgeons to achieve this was criticised.4 It was feared that ISTCs might have significantly worse outcomes for patients and increased rates of complications.5 Another concern was the negative impact of ISTCs as a result of siphoning off the easier cases, which are essential for surgical training.4
In 2006, an inquiry by the Commons Select Committee on Health and Social Services concluded that the quality of care provided by ISTCs could not be properly evaluated because of the poor quality of the data collected by the centres and the historical lack of routine data on outcomes collected by the NHS.4 The committee recommended that comparable and standardised data be collected. A subsequent review carried out by the Healthcare Commission in 2007 confirmed that better data on individual patients were essential to answer the concerns over the quality of care in ISTCs, which was welcomed by clinicians.6 7
In response, the Patient Outcomes in Surgery (POiS) audit was established to compare operative characteristics of patients and outcomes reported by patients adjusted for case mix and complication rates for hip and knee replacement, inguinal hernia repair, and varicose vein surgery undertaken by ISTCs compared with those undertaken by NHS providers (referred to in this paper as NHS patients). The POiS audit built on a pilot study examining the feasibility of routine collection of patient reported outcome measures (PROMs) after elective surgery in 769 patients treated in six ISTCs and 1895 treated by 20 NHS providers.8 The pilot study found that patient reported outcomes were slightly better in ISTCs than in the NHS, but these results were preliminary given concerns about their generalisability, poor statistical power, and the limited adjustment for differences in case mix.
Enrolment of providers
All ISTCs undertaking any of the four procedures were invited to join the POiS audit and at least two NHS providers were invited for each ISTC. Of the 16 ISTCs undertaking orthopaedic surgery, 14 participated, and of the 109 invited NHS providers, 51 participated (with at least two NHS providers in each of the 10 NHS strategic health authority regions). Participation was lower for inguinal hernia repair and varicose vein surgery: nine of the 21 invited ISTCs and 21 of the 49 invited NHS providers participated (with at least one NHS provider in each strategic health authority region, except for the London region).
Recruitment of patients
Consecutive recruitment of patients aged at least 15 ran from 1 June 2008 to 30 September 2009 for unilateral primary hip and knee replacement and from 1 December 2008 to 30 September 2009 for inguinal hernia repair and varicose vein surgery. A member of staff at a preoperative assessment clinic or on the day of admission for surgery invited patients to join the audit. Patients undergoing inguinal hernia repair or varicose vein surgery between 1 April and 30 September 2009 were recruited via the NHS PROMs programme.9 The design of this programme closely matched that of the POiS audit.
Participating patients completed a preoperative questionnaire that included items to measure severity of symptoms and disability specific to their condition as well as a generic health related quality of life measure. The Oxford hip score and the Oxford knee score both included 12 questions covering pain and ability related to the hip or knee problem.10 11 Responses to each question (expressed on a 4 point scale) are summed up to provide an overall score ranging from 0 (worst health status) to 48 (best health status). The Aberdeen varicose vein questionnaire has 13 items, each of which is weighted depending on the severity of the symptoms and added together to produce an overall score from 0 (no problem) to 100 (severe symptoms and disability).12 No suitable disease specific instrument was available for patients undergoing inguinal hernia repair.
The EQ-5D was used to describe generic health related quality of life based on five dimensions of health (mobility, self care, usual activities, pain/discomfort, and anxiety/depression) with responses at three levels (“no problems,” “some or moderate problems,” and “extreme problems”).11 An EQ-5D index score is calculated by matching the pattern of the five responses against a set of utilities derived from the general UK population, producing a single score ranging from −0.59 (worse than death), 0 (death), to 1 (perfect health).13
In addition, in response to the question “Have you been told by a doctor that you have any of the following?” patients reported their comorbidities using a 12 item index developed by the audit team. These include heart disease, high blood pressure, stroke, leg pain when walking, lung disease, diabetes, kidney disease, disease of the nervous system, liver disease, cancer within the past five years, depression, and arthritis. Patients were also asked whether they had had previous surgery for the same hip, knee, hernia, or varicose vein problem.
Socioeconomic status was measured with the English index of multiple deprivation 2007 (IMD) for the lower super output area according to the patient’s postcode.14 These 32 482 areas cover an average population of around 1500 people or 400 households. We derived five socioeconomic groups based on fifths of the national ranking of the lower super output areas.
A postoperative questionnaire was posted to the patient’s home address three months after hernia repair or varicose vein surgery and six months after hip or knee replacement. Non-responders were sent a reminder letter and a replacement questionnaire five weeks after the original mailing. The questionnaire contained the same measures as the preoperative questionnaire and an overall assessment of the outcome of the surgery (“How would you describe the results of your operation?”) with responses on a 5 point scale (“poor,” “fair,” “good,” “very good,” and “excellent”). We also asked about additional surgery on the site of the first operation and postoperative complications (allergy or reaction to drugs, urinary complications, bleeding, or wound problems).15
Recruitment rates were calculated for each provider as the number of recruited patients (consenting patients who returned a preoperative questionnaire and met the inclusion criteria) divided by the total number of eligible patients during the recruitment period according to hospital episode statistics database. Postoperative response rates were calculated as the number of returned questionnaires divided by the number of recruited patients (defined above).
Descriptive results are given as means with standard deviations and as percentages. We used multiple linear regression to estimate the difference for continuously distributed outcomes and multiple logistic regression to estimate odds ratios for dichotomous outcomes. Robust standard errors were used to allow for clustering of outcomes within providers. Results are presented with 95% confidence intervals. They are adjusted for age, sex, preoperative PROMs scores, general health, socioeconomic status, number of comorbidities, whether help was received to complete the questionnaire, duration of the problem, living circumstances, and length of follow-up. All reported P values are two sided, and P<0.05 was considered significant.
Recruitment rates were higher in ISTCs than in the NHS providers (table 1⇓): 57% v 45% for hip replacement, 58% v 48% for knee replacement, 37% v 27% for inguinal hernia repair, and 36% v 33% for varicose vein surgery. Recruited patients were similar in age, sex, and socioeconomic status compared with all eligible NHS patients based on hospital episode statistics data for all four procedures.
Response rates to the postoperative questionnaire were higher in ISTC patients for hip (90% v 84%) and knee replacement (88% v 81%) but higher in NHS patients for inguinal hernia repair (77% v 71%) and varicose vein surgery (69% v 64%) (table 1). For all four procedures, patients who returned a postoperative questionnaire were older and less likely to live in socioeconomically deprived areas. In patients undergoing a hip or knee replacement, response rates were higher in those who had better outcomes in terms of severity of symptoms and generic quality of life.
Hip replacement—Before surgery, NHS patients had more severe symptoms (lower hip score) and poorer quality of life (lower EQ-4D score) than ISTC patients (table 2⇓). NHS patients were also more likely to live in socioeconomically deprived areas and have more comorbidities. Outcomes six months after the hip replacement were worse in patients treated by NHS providers even after adjustment for preoperative differences. After adjustment, the hip score of NHS patients was 1.7 lower and the EQ-5D was 0.03 lower. Also, NHS patients were more likely to report a poor operation result (odds ratio 1.3) and to have postoperative complications (1.3).
Knee replacement—Preoperative differences between patients having a knee replacement in ISTCs and in the NHS were similar to those observed for hip replacement (table 3⇓). Outcomes six months after the knee replacement were worse in patients treated by NHS providers. After adjustment, the knee score of NHS patients was 0.9 lower and the EQ-5D was 0.02 lower. NHS patients were more likely to report a fair or poor operation result (odds ratio 1.2) and complications (1.4).
Inguinal hernia repair—Patients who underwent inguinal hernia repair in the NHS reported a poorer preoperative quality of life (lower EQ-5D). NHS patients were also on average older, more likely to live in socioeconomically deprived areas, and have more comorbidities (table 4⇓). Most outcomes three months after the hernia repair were similar in the two treatment settings. The only significant difference was that patients treated by NHS providers were more likely to report only a fair or poor operation result (odds ratio 1.4), even after differences in case mix were taken into account.
Varicose vein surgery—There were small preoperative differences between patients who had their surgery in an ISTC or in an NHS unit (similar Aberdeen varicose vein and EQ-5D scores). NHS patients were older and had more comorbidities, though these differences were small (table 5⇓). The only difference in outcome between the two settings was that patients treated by NHS providers were more likely to have undergone another operation. The risk was higher in the NHS (odds ratio 2.8), even after differences in case mix were taken into account.
Patients treated in independent sector treatment centres (ISTCs) tend to be healthier and have less severe primary conditions. With adjustment for such differences, the POiS audit shows that patients who underwent a hip or knee replacement in ISTCs had better outcomes in terms of severity of symptoms, health related quality of life, and postoperative complications. These differences, however, were small, their clinical relevance is slight, and they could be attributable to differences in case mix that were not fully taken into account.
There were no significant differences in outcomes reported by patients after hernia repair and varicose vein surgery. The higher rate of reoperation after varicose vein surgery in patients treated by NHS providers could be explained by a difference in operative management (multistage approach instead of a single intervention) rather than a difference in quality of care.
Participation in the POiS audit was voluntary for both ISTCs and NHS providers, and not all of those invited took part. For all four procedures, however, there were only small differences in terms of age, sex, and socioeconomic status between patients who participated in the audit and the wider population of NHS patients undergoing these procedures.
Rates of patient recruitment for the four procedures ranged from 30% to 60%, which was lower than anticipated. This was partly because towards the end of the recruitment period, ISTCs and NHS providers were preparing to switch over to the NHS PROMs programme.9 Recruitment was lower in NHS providers than in the ISTCs. We aimed to reduce the potential impact of selective inclusion by adjusting for differences in case mix.
The Healthcare Commission review that was carried out in 2007 found that patients treated in ISTCs were more satisfied with their experience of care than those treated by NHS providers.6 This could have had an impact on how patients reported the outcome of their surgery because patients who report that they had a better experience might also be more inclined to report better outcomes.16
The regression models that we used to adjust the outcomes for differences in preoperative characteristics could not include two recognised risk factors for poor surgical outcome: poor fitness for surgery (ASA grade)17 and high body mass index (BMI). Although the POiS audit had aimed to collect data on these risk factors, either through data linkage with the National Joint Registry for England and Wales18 or through data reported by the surgeons, the level of missing data was too high. For example, ASA grade was missing for 30% and BMI for 65% of patients undergoing hip and knee replacement. The limited data that were available showed that patients who had an operation in an ISTC were fitter for surgery than those treated by NHS providers, providing further evidence that observed differences in outcome were probably subject to residual confounding.
We calculated recruitment rates with the number of patients undergoing surgery according to HES as the denominator. Reporting patients to HES was mandatory for both ISTCs and NHS providers. Although there was under-reporting by ISTCs before 2006, during the period covered by our study the reporting rates had improved. It is unlikely that use of HES data had a major impact on the estimated recruitment rates.
In line with their contracts, ISTCs treat patients with a more favourable case mix profile. The differences in severity of symptoms before surgery between ISTC and NHS patients, however, were small. For example, the preoperative hip or knee score in NHS patients was less than one point lower than in ISTC patients. Consequently, our results lessen concerns that ISTCs are “cherry picking” the more healthy patients and that they have a negative impact on service configuration and surgical training.19
We have not confirmed earlier reports of higher rates of complications and reoperation in ISTCs,4 5 indeed patients treated in ISTCs reported lower rates. It is important to note, however, that the interpretation of the higher reoperation rates after varicose vein repair in the NHS is not straightforward. It might reflect that multistage procedures are more often used by NHS providers given that the data reported by patients on whether they had another operation did not allow a distinction between planned and unplanned further surgery.
Our findings support the idea that separating elective surgical care from emergency services could improve the quality of care. This might result from a more predictable work flow, which will increase senior supervision of complex cases.2 In addition, the creation of treatment centres could also allow a redesign of the physical environment and encourage innovations in the process of care, including nurse led standardised preoperative assessment and case managers guiding patients through well honed care pathways. The creation of ISTCs could have facilitated the implementation of new models of care in the NHS providers, contributing to the lack of any great difference in outcomes between the two sectors. We need to understand which elements of these models of care lead to better outcomes.
A final challenge relates to determining the clinical significance of the modest differences we observed in severity of postoperative symptoms and health related quality of life. There are no generally accepted “minimally important differences” (that is, the smallest differences that a patient perceives as beneficial and would mandate a change in the patient’s management) for the outcome measures that we used.20 The earlier mentioned study of the feasibility of using patient reported outcome measures suggested minimally important differences of 8.4 for the Oxford hip score and 3.8 for the Oxford knee score, which are four to five times higher than the differences that we actually observed.8 This does not imply that the differences in outcomes between ISTCs and the NHS can be ignored but suggests that the differences are minor compared with the impact the ISTC programme might have on the provision of elective services.
What is already known on this topic
In 2002, the Department of Health started the procurement of independent sector treatment centres (ISTCs) in England to reduce waiting times, give patients more choice, and encourage innovative models for the provision of non-emergency services
Clinicians and some politicians have raised concerns about the quality of care provided by ISTCs and their destabilising impact on wider NHS service provision and surgical training
What this study adds
ISTCs treated patients who were healthier, had less severe symptoms, and were from more affluent areas than patients treated by NHS providers
Patients who underwent a hip or knee replacement in an ISTC had slightly better outcomes than patients treated by NHS providers in terms of symptoms and disability and health related quality of life. Such differences were minor and unlikely to be clinically significant
Cite this as: BMJ 2011;343:d6404
We thank the staff and patients at participating providers and other members of staff at the clinical effectiveness unit of the Royal College of Surgeons of England (John Browne, Lynn Copley, Jackie Horrocks, Andrew Hutchings, Jenny Neuberger) for their support.
A steering committee was established to provide independent oversight and governance to the POiS audit. Representatives from all the main stakeholder groups were invited to join the steering committee.
Mike Gill (chair), Andrew Woodhead (National Joint Registry), Carolyn Naisby (Chartered Society of Physiotherapy), Jean Jacques de Gorter (Spire hospitals and independent sector), Nick Black (London School of Hygiene and Tropical Medicine), Timothy Wilton (British Orthopaedic Association), Jo C Hawkes (patient/lay representative, BOA Patient Liaison Group), Beda Oliver (patient/lay representative, RCS Patient Liaison Group), John McIvor (NHS Confederation, on behalf of the Commissioners), Neil Betteridge (Arthritis Care), Paul Evans (consultant orthopaedic surgeon and medical director, Peninsula NHS Treatment Centre), and Brian Rees (consultant general surgeon and Royal College of Surgeons of England).
In addition, representatives from the department of Health acted as observers for the sponsoring organisation: Richard Dale (medical director, Commercial Directorate), Jane Moore (policy advisor), David Nuttall, and Rob Moorhead.
The following members have stepped down from the steering committee: J J de Gorter (replaced by Sheila Peskett, Ramsay Health Care UK), J C Hawkes, N Betteridge (replaced by Jo Cumming, Arthritis Care), R Dale (replaced by Anna Casburn-Jones, head of Clinical Care), and J Moore (replaced by Gerard Hetherington, director Clinical Care).
Contributors: JvdM and NB conceived and designed the audit. JC and MK were responsible for the implementation of the audit, coordination of the data collection, and analysis, supported by JvdM. JC and MK wrote the first draft. JvdM wrote the final paper including comments from all other authors. JvdM is guarantor.
Funding: The POiS audit was commissioned and sponsored by the Department of Health Commercial Directorate. Representatives of the department attended the meetings of the audit’s steering committee as observers. The authors are responsible for the design of the audit, data analysis, the preparation of the manuscript, and the decision to publish.
Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare: no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.
Ethical approval: Not required.
Data sharing: The final report of the Patient Outcomes in Surgery Audit is available from www.rcseng.ac.uk/surgical_research_units/ceu/docs/POiSAuditReport2011.pdf.
This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.