- Ruth E Crowther, research fellowa,
- Max Marshall (), reader in community psychiatrya,
- Gary R Bond, Chancellor's professorb,
- Peter Huxley, professor of social workc
- a School of Psychiatry and Behavioural Sciences, University of Manchester, Manchester PR2 9HT
- b Department of Psychology, Indiana University-Purdue University Indianapolis, IN 46202-3275, USA
- c Health Services Research, King's College Institute of Psychiatry, London SE5 8AF
- Correspondence to: M Marshall, Academic Unit, Royal Preston Hospital, Preston PR2 9HT
- Accepted 20 October 2000
Objective: To determine the most effective way of helping people with severe mental illness to obtain competitive employment—that is, a job paid at the market rate, and for which anyone can apply.
Design: Systematic review.
Participants: Eligible studies were randomised controlled trials comparing prevocational training or supported employment (for people with severe mental illness) with each other or with standard community care.
Outcome measures: The primary outcome was number of subjects in competitive employment. Secondary outcomes were other employment outcomes, clinical outcomes, and costs.
Results: Eleven trials met the inclusion criteria. Five (1204 subjects) compared prevocational training with standard community care, one (256 subjects) compared supported employment with standard community care, and five (484 subjects) compared supported employment with prevocational training. Subjects in supported employment were more likely to be in competitive employment than those who received prevocational training at 4, 6, 9, 12, 15, and 18 months (for example, 34% v 12% at 12 months; number needed to treat 4.45, 95% confidence interval 3.37 to 6.59). This effect was still present, although at a reduced level, after a sensitivity analysis that retained only the highest quality trials (31% v 12%; 5.3, 3.6 to 10.4). People in supported employment earned more and worked more hours per month than those who had had prevocational training.
Conclusion: Supported employment is more effective than prevocational training at helping people with severe mental illness obtain competitive employment.
In the United States it is estimated that 75-85% of people with severe mental illness are unemployed, 1 2 whereas estimates in the United Kingdom range from 61% to 73%. 3 4 Yet despite these high unemployment rates, surveys consistently show that most people with severe mental illness want to work. 5 6
There are compelling ethical, social, and clinical reasons for helping people with mental illness to work. From an ethical standpoint, the right to work is enshrined in the Universal Declaration of Human Rights 1948 and has been incorporated into national legislation, such as the UK Disability Discrimination Act 1995. From a social standpoint, high unemployment rates are an index of the social exclusion of people with mental illness, which the US and UK governments, among others, are committed to reducing. 7 8 From a clinical standpoint, employment may lead to improvements in outcome through increasing self esteem, alleviating psychiatric symptoms, and reducing dependency.7
Prevocational training and supported employment are different ways of helping people with severe mental illness return to work. Prevocational training assumes that people with severe mental illness require a period of preparation before entering into competitive employment—that is, a job paid at the market rate, and for which anyone can apply. This includes sheltered workshops, transitional employment (working in a job that is “owned” by a rehabilitation agency), work crews, skills training, and other preparatory activities.9
Supported employment places clients in competitive jobs without extended preparation and provides on the job support from trained “job coaches” or employment specialists.10 The core principles of supported employment are that (a) the goal is competitive employment in work settings integrated into a community's economy, (b) clients are expected to obtain jobs directly, rather than after lengthy pre-employment training, (c) rehabilitation is an integral component of treatment of mental health rather than a separate service, (d) services are based on client's preferences and choices, (e) assessment is continuous and based on real work experiences, and (f) follow on support is continued indefinitely. 10 11
In the United States there are about 3000 “psychiatric rehabilitation providers” offering some form of prevocational training, whereas there are more than 36 000 people with mental illness in supported employment schemes. 12 13 In the United Kingdom prevocational training is still the norm, but there are at least 80 agencies offering supported employment.14
It is unclear how far prevocational training and supported employment are effective at helping people with severe mental illness to obtain competitive employment. We aimed to evaluate the effectiveness of the two approaches.
Search strategy and inclusion criteria
We electronically searched CINAHL (1982-98), Embase (1980-98), Medline (1966-98), and PsychLIT (1987-98). The search proceeded by exploding the appropriate index term for mental disorder in each database and combining this with a free text search using (supp* employ*) or (employment) or (psychosocial rehab*) or (psychiatric rehab*) or (occupational rehab*) or (soc* rehab*) or (work rehab*) or (job rehab*) or (sheltered work*) or (transitional employ*) or (rehabilitation counselling) or (vocation*) or (fountain house*) or (fountain-house*) or (clubhouse*) or (club-house*). The results of this search were then combined with a search using the Cochrane Collaboration search string for potential trials and reviews.15 We also carried out a free text search on the collaboration's register of randomised controlled trials. The sensitivity of the search strategy was evaluated by determining how many trials cited in the reference lists of the identified trials and reviews had not been detected. Of three undetected trials, two were not listed on any of the databases, and the third trial was indexed under “delivery of health care/integrated.” A further search using this index term detected no further trials.
Two reviewers (MM and RC) independently assessed the published reports of the identified trials and decided which met the inclusion criteria. These were that the trial was a randomised controlled one, analysed on an intention to treat basis, in which prevocational training or supported employment was compared with standard community care or each other; outcome data were provided on 50% or more of randomised subjects; and most subjects were aged 18 to 65 and had severe mental illness (schizophrenia, bipolar disorder, or depression with psychotic features). Studies meeting inclusion criteria were allocated to one of three comparisons (prevocational training versus community control, supported employment versus community control, or supported employment versus prevocational training). Prevocational training was defined as any approach in which participants were expected to undergo a period of preparation before being encouraged to seek competitive employment. Supported employment was defined as any approach that attempted to place clients immediately in competitive employment (although a short period of orientation was acceptable if of less than one month's duration and not involving sheltered work, employment training, or transitional employment). Interrater agreement was assessed for overall eligibility of trials (κ=0.89) and for allocation of trials to comparisons (κ=1).
Trials were classified according to allocation concealment15 as A, adequate (the method for assigning participants to interventions was robust to patient and clinician bias and clearly described); B, unclear; and C, inadequate (the method of assignment was not robust to patient and clinician bias). When the method was unclear, trialists were contacted for further details.
Blinding of patients and treating clinicians is not possible in trials of vocational rehabilitation. It is also difficult for those evaluating outcome to remain blind to group allocation, as they are obliged to collect data that indicates group allocation (for example, days in different types of employment). The trials were, however, rated on independence of evaluators from those providing the intervention.
Data extraction and analysis
The primary outcome was number of clients in competitive employment at various times. Other secondary employment outcomes were number of subjects in any form of employment (including transitional, sheltered, or voluntary work), mean hours per month in competitive employment, and mean monthly earnings. In addition data were extracted on clinical and social outcome (including number of people participating and number admitted to hospital) and costs (mean monthly costs of the programme and of all health care). Categorical data and continuous data were extracted independently by MM and RC and cross checked by double entry. Continuous data were excluded if collected using an unpublished scale or based on a subset of items from a scale (such data are known to be biased in psychiatric trials).16 For categorical data we calculated the relative risk with confidence intervals. The number needed to treat for one person to obtain competitive employment was calculated as the inverse of the absolute risk reduction for being unemployed. Confidence intervals for the number needed to treat were calculated using the Arcus Quickstat program (Research Solutions, Cambridge).
Heterogeneity was defined as a significance level of 0.1 by the χ2 test for heterogeneity. Where heterogeneity was present the data were reanalysed using a random effects model. We conducted a sensitivity analysis, excluding trials with allocation concealment in categories B or C, non-independent evaluators, or follow up rates of less than 75%.
We identified 40 trials and 13 reviews. We excluded 29 trials because the trial was not randomised (11 trials), the participants did not have severe mental illness (3), the intervention did not involve vocational rehabilitation (6), the number of participants was unclear (2), the trial compared a modification of prevocational training with unmodified prevocational training (4), and the trial compared prevocational training with continuing care in hospital (3). Eleven trials met the inclusion criteria (see tables on website) and were allocated to comparisons of prevocational training with standard care (five trials; 1204 subjects), supported employment with standard care (one trial; 256), and supported employment with prevocational training (five trials; 491).
Prevocational training versus standard care
Of the five trials comparing prevocational training with standard care, one had adequate allocation concealment, two used independent evaluators, and four had follow up rates of greater than 75%. No trial was eligible for the sensitivity analysis. Two trials provided data on the primary outcome of number of subjects in competitive employment, but these showed no evidence that prevocational training was superior to control (18 months (28 subjects): relative risk 1.18, 95% confidence interval 0.87 to 1.61; 24 months (215): 0.95, 0.77 to 1.17). 17 19 Three trials reported data on number of subjects in any form of employment, showing no evidence that prevocational training was superior to control at 3, 6, 9, 12, and 18 months. 17 18 20 Two trials found no difference in the number of clients participating in the programme between prevocational training and control groups (284 subjects; relative risk 0.97, 0.73 to 1.30). 18 20 Three trials showed that significantly fewer patients were admitted to hospital among those receiving prevocational training (887 subjects; 0.79, 0.65 to 0.95). 17 18 21 Heterogeneity was present in this outcome, and on reanalysis using a random effects model the difference failed to reach significance (0.71, 0.48 to 1.04). One trial reported no significant difference in self esteem between prevocational training and control groups (28 subjects; 25.5 (SD 6.6) and 23.3 (7.3), respectively).19 One trial reported mean monthly total healthcare costs of $417.90 (£292.83) for the prevocational training group and $651.50 (£456.52) for controls, but no statistical analysis was reported.18
Supported employment versus standard care
Only one trial provided data for supported employment compared with standard care.22 Although the trial used independent raters, the method of allocation concealment was unclear and the follow up rate was only 71%. A further problem was that the intervention combined supported employment with assertive community treatment, whereas the control was standard community care. For 256 subjects there was no difference in those in competitive employment between supported employment and control at 12 months (relative risk 1.01, 0.93 to 1.09), but there was a significant difference favouring supported employment at 24 months (0.92, 0.85 to 0.99) and 36 months (0.88, 0.82 to 0.96). Clients receiving supported employment were more likely to be in any form of employment at 12 months (0.79, 0.70 to 0.90; number needed to treat 5.5) and to earn more per month (supported employment group $60.50 (£42.39), control group $26.90 (£18.85); P<0.05). Participation rates and number of hospital admissions were not significantly different between clients receiving supported employment and controls (0.74, 0.55 to 1.01 and 0.83, 0.63 to 1.10, respectively). Mean monthly healthcare costs were significantly higher for clients in the supported employment group ($1599.00 (£1120.45) versus $527.30 (£369.49) for controls), but this finding is difficult to interpret as clients receiving supported employment also received assertive community treatment.
Supported employment versus prevocational training
Of the five trials comparing supported employment with prevocational training, four had adequate allocation concealment, four used independent evaluators, and all had follow up rates greater than 75%. In one trial, however, the intervention combined supported employment with assertive community treatment, whereas the control was standard community care.27 Data from the five trials showed a significant difference in favour of supported employment at 4, 6, 9, 12, 15, and 18 months for those likely to be in competitive employment (for example, at 12 months 34% in supported employment and 12% who received prevocational training, relative risk 0.76, 0.69 to 0.84; number needed to treat 4.45, 3.37 to 6.59 (figure)). Heterogeneity was present at 12 months, but the difference in favour of supported employment remained significant after reanalysis using a random effects model (0.76, 0.64 to 0.89). One trial reported the number of subjects in any form of employment, finding no significant difference between supported employment and prevocational training at 6, 12, and 18 months.27 Three trials found that clients in supported employment had significantly more hours per month in competitive employment than those who received prevocational training (table 1). Three of four trials found that clients in supported employment had higher mean monthly earnings that those who received prevocational training (table 1). There were insufficient data to determine whether there was a difference in participation rates between supported employment and prevocational training at 6, 12, and 18 months. Two trials reported data on self esteem, quality of life, and severity of symptoms but found no significant differences. 24 25 One trial reported that programme costs of supported employment were greater than those for prevocational training, but that overall healthcare costs were less for people in supported employment.23 Another trial found no significant difference in programme costs and overall healthcare costs.24
We conducted a sensitivity analysis on the primary outcome of number of subjects in competitive employment, excluding trials with allocation concealment in categories B or C, those that used non-independent evaluators, or those with follow up rates of less than 75%. This left data from three trials, 24 25 27 which continued to show a significant difference in favour of supported employment at 4, 6, 9, 12, 15, and 18 months. This difference remained after exclusion of the trial that combined supported employment with assertive community treatment (for example, at 12 months 31% in supported employment and 12% who received prevocational training; number needed to treat 5.3, 3.6 to 10.4).27
Supported employment was more effective than prevocational training at helping people with severe mental illness to obtain competitive employment. This finding was robust to a sensitivity analysis that excluded all but the two highest quality trials and was supported by data for other employment outcomes. Data on clinical and social functioning and costs were inconclusive but suggested no major differences between supported employment and prevocational training. The five trials of supported employment versus prevocational training showed good recruitment of women, people from ethnic minorities, and people with schizophrenia (table 2), which suggests that the main finding of the review can be applied to the general population of patients with severe mental illness who desire to work. Generalisability is, however, limited by the fact that all the trials were conducted in the United States. It remains uncertain whether supported employment will be more effective than prevocational training in countries with less dynamic economies and dissimilar welfare structures.
Only one trial compared supported employment with standard community care. Although this trial suggested that supported employment was superior to standard community care, its findings are difficult to interpret as the group receiving supported employment also received assertive community treatment.
The included trials of prevocational training compared with standard community care were of limited quality, and none met the criteria for the sensitivity analysis. The data available from these trials were insufficient to make judgments on the effectiveness of prevocational training over standard community care. Only two of five trials in this comparison reported data on the primary outcome of competitive employment. This omission may reflect selective reporting of results. Interestingly, clients receiving prevocational training were significantly less likely to be admitted to hospital than those receiving standard community care, but there was heterogeneity with this outcome, and the finding was not significant when analysed by a random effects model. Trials of supported employment did not usually report data on hospital admissions.
With the passing of the Disability Discrimination Act 1995, the UK government signalled its commitment to helping disabled people return to the workplace. People disabled by severe mental illness have particularly high unemployment rates. Our review indicates that supported employment is a more effective way of helping such people find competitive employment than is prevocational training. The UK government should therefore encourage agencies concerned with vocational rehabilitation to develop and evaluate supported employment schemes similar to those in the United States.
What is already known on this topic
Prevocational training (a period of preparation before entering competitive employment) and supported employment (placement in competitive employment while offering on the job support) are ways of helping people with severe mental illness obtain work
Both methods are widely practised, but it is unclear which is most effective, and so far the available data have not been subject to a systematic review and meta-analysis
What this study adds
Supported employment is more effective than prevocational training at helping people with severe mental illness to obtain and keep competitive employment
Contributors: MM and PH conceived the review and obtained funding. REC, MM, and PH designed the protocol with assistance from GRB. REC and MM performed the literature search, appraised the papers, and extracted the data. GRB assisted in obtaining the US studies and contacted trialists for additional information. All authors analysed the data and jointly wrote the paper. MM will act as guarantor for the paper.
Funding The research was supported by the NHS Health Technology Assessment Program (grant number 96/41/3). The views expressed in this paper are not necessarily those of this programme.
Competing interests GRB has a close collaborative relationship with Bob Drake and Debbie Becker, developers of the individual placement and support model.
Details of the trials are available on the BMJ's website