Assessing the outcome of compulsory psychiatric treatment in the community: epidemiological study in Western AustraliaBMJ 2002; 324 doi: https://doi.org/10.1136/bmj.324.7348.1244 (Published 25 May 2002) Cite this as: BMJ 2002;324:1244
- Neil J Preston, research psychologist ()a,
- Steve Kisely, associate professorb,
- Jianguo Xiao, biostatisticianc
- a Mental Health Directorate, Fremantle Hospital and Health Service, PO Box 480 Fremantle, WA 6160, Australia
- b University Department of Psychiatry at Fremantle Hospital, University of Western Australia, 16 The Terrace, Fremantle
- c Health Information Centre, Health Department of Western Australia, 189 Royal Street, East Perth, WA 6004
- Correspondence to: N J Preston
- Accepted 3 December 2001
Objective: To examine whether community treatment orders for psychiatric patients reduce subsequent use of health services in comparison with control patients not placed on an order.
Design: Epidemiological study with a before and after, two stage design of matching and multivariate analysis, controlling for sociodemographic variables, clinical features, and psychiatric history.
Setting: All community based and inpatient psychiatric services in Western Australia, covering a population of 1.7 million people.
Participants: 228 subjects placed on a community treatment order, matched with an equal number of controls to give a total of 456 patients.
Main outcome measures: Inpatient admissions, bed days, and outpatient contacts one year after subjects were placed on a community treatment order or the index date of matched controls.
Results: Both subjects and their matched controls had reduced inpatient admissions and bed days in hospital. Subjects had significantly more outpatient contacts. Multivariate analysis indicated that being placed on a community treatment order was associated with increased outpatient contacts in the subsequent year compared with the control group. Otherwise, orders did not affect subsequent use of health services. Other factors associated with increased use of health services were age and inpatient admissions, bed days, and outpatient contacts before the order or index date. No covariates were shown to be associated with changes in within pair differences in inpatient admissions or bed days.
Conclusions: The introduction of compulsory treatment in the community does not lead to reduced use of health services.
What is already known on this topic
What is already known on this topic Various forms of compulsory treatment in the community have been suggested as being effective in reducing use of services by patients with mental health disorders
Studies have often lacked epidemiological sampling frames and control for possible confounding factors
What this study adds
What this study adds Patients placed on community treatment orders and those not on such orders had reduced hospital admissions and bed days one year later
Placement of an order did not predict subsequent use of services
Community treatment orders may not be an effective alternative to assertive community treatment programmes
Enforcing psychiatric treatment in the community has become a feature in Australia, New Zealand, the United Kingdom, and the United States.1 2 3 In the United States more than half the states have some form of compulsory community treatment,2 and in Australasia similar provisions exist in Victoria, Western Australia, New South Wales, and New Zealand.3 4 5 Initiatives in the United Kingdom have included extended leave for patients leaving hospital and the supervision register. 6 7 The recent white paper Reforming the Mental Health Act contains provisions for compulsory treatment in the community, although there will still be no powers to give medication forcibly outside a clinical setting.8
Studies indicating limited but improved outcomes in terms of readmission to hospital, length of stay, and adherence to treatment have often not controlled for selection bias, variations in treatment, and differing criteria for compulsory treatment in the community.3 In South Carolina chronicity of psychosis was an important determining factor for compulsory treatment in the community,9 and in the United Kingdom Sensky et al identified recent dangerousness and non-adherence as determining factors.6 Community treatment orders in New South Wales are mostly used for unmarried male patients with schizophrenia.10 Involuntary outpatient treatment in many American states does not include the power to force medication in the community, but community treatment orders in Australasia do. In addition, studies often do not include a control group to take into account the possibility that participants were recruited when particularly disturbed and that subsequent reductions in hospital use may be due to other factors. In one study with a control group of patients not subject to a compulsory treatment order, the control group showed a similar reduction in time in hospital.11
In the United Kingdom the extended leave provision of the Mental Health Act has been evaluated as a proxy for the community treatment order, although it does not cover compulsory treatment in the community. Sensky et al found that extended leave improved adherence, reduced time spent in hospital, and reduced levels of dangerousness in comparison with a control group of patients of similar age, sex, and diagnosis.6 The introduction of supervised discharge in 1985 meant that a patient could be conveyed to a designated location for medical treatment, occupation, or training but was still not obliged to accept treatment; this legislative measure has never been formally evaluated.
Even when studies have used a cohort-control design, it is difficult to know whether to attribute the health gain to the order or to non-specific effects of increased contact with healthcare professionals. 3 12 13 14 Swartz et al found that although patients randomised to prolonged involuntary community treatment had reduced hospital readmissions and bed days, it was difficult to separate out how much of the improvement was due to compulsory treatment and how much to intensive community management.14 A further problem is the difficulty in ensuring that the control group is as severely ill as the group placed on a community treatment order.10
The new Mental Health Act of Western Australia was implemented in November 1997. As in other parts of Australia and New Zealand, it includes the provision of involuntary treatment in the community through the introduction of a community treatment order. The order is made out by a qualified mental health specialist, who must specify which medical practitioner is to supervise the patient's treatment or care, where the patient is to receive care, the frequency at which the medical practitioner is to report to the specialist, and the duration of the order, which must not exceed three months in the first instance. The Mental Health Review Board must conduct a review between four and six months after instigation of the order. A lawyer, a psychiatrist, and a layperson make up the review board and hear evidence from both treating doctors and patients.
The aim of our study was to compare the rate of inpatient admissions, bed days, and outpatient contacts of patients one year before and one year after placement on a community treatment order by using a two stage design. Data for subjects were compared with those for a matched control group and then entered into a multiple regression equation to examine variables predicting the use of health services after placement on an order. The purpose was to identify similar patients not placed on an order as matched controls and examine whether keeping patient characteristics constant could contribute to similar outcomes for inpatient admissions, bed days, and outpatient contacts in the year before the index date. This was done in order to ensure that subjects and controls had equal exposure to the treatment condition before the order. If outcomes before the order could be held constant, any differences in outcomes after the order could be more readily attributed to the placement of the order than to differences in patient characteristics.
We used the Mental Health Information System, which records inpatient, outpatient, and community attendances at psychiatric inpatient units, general hospitals, and community mental health services in the state. This information system was linked with another database administered by the Mental Health Review Board in terms of involuntary admissions under the Mental Health Act and subsequently de-identified. Use of these two registers enabled us to include all patients who had been made subject to a community treatment order for the entire state of Western Australia, giving access to enough patients for an epidemiological analysis.
Selection of subjects
We selected subjects on the basis of when they were placed on a community treatment order. We refer to the date of first placement on an order from either an authorised hospital or the community as the “index date.” We selected patients placed on orders between 13 November 1997 (the date of implementation of the Mental Health Act 1996) and 31 November 1998. Of the 313 patients who were preliminarily selected, seven died during the one year observation period after their index dates and 32 had not had a full year's contact with the mental health registry before their index dates, leaving 274 subjects.
Selection of matched controls
We selected matched controls as follows. For each subject we selected a group of candidates of the same sex and Aboriginal ethnicity and with an age difference of less than 2 years. We weighted clinical and demographic characteristics of subjects and controls in terms of their relative importance in the matching process. We gave diagnosis at index date the greatest relative importance in the selection algorithm, followed by length of stay, number of hospital admissions and occasions of service use (>80% and <120% of subject's length of stay, hospital admissions, and occasions of service use), and involuntary status in the year before the index date. We created a subject-control matching score that summed all the relative importance scores for the above clinical and demographic variables for each candidate. We then chose a matched control with the greatest matching score for the subject as the final matched control for that subject. Of 7881 matched controls who were preliminarily selected in the first stage of the matching process, 266 were determined as the final matched controls. Suitable matched controls could not be identified for 38 subjects, leaving 228 patients in each group. Subjects who died during the study were dropped from the study along with their matched controls to ensure that all subjects and controls had equal observation periods before and after the index date.
We used the paired samples t test to test for differences between subjects and controls in admissions, bed days, and outpatient contacts before the index date and subsequent changes in bed days and outpatient contacts within each group. We performed each analysis on the logarithmically transformed dependent variables.
To further adjust for possible differences between the two groups, we generated standard multiple regression models to examine the contribution of patient characteristics and use of services before the index date to subsequent inpatient admissions, inpatient bed days, and outpatient contacts. Because of the non-normal distribution (skewness for admissions, inpatient bed days, and outpatient contacts after the index date were 2.55, 3.85, and 2.66), we performed logarithmic transformation on each dependent variable and inpatient admissions, bed days, and outpatient contacts before the index date. We examined predictor variables for each regression model by entering service use, including inpatient admissions, bed days, and outpatient contacts, before the index date and data on patient characteristics such as age, sex, Aboriginal ethnicity, and whether subject or control. We examined the contribution of diagnosis by creating dichotomous dummy variables for schizophrenia, mood disorders, and disorders of substance use. We kept personality disorders as the reference group for diagnosis. We examined social relationship variables by creating dichotomous dummy variables for marital status, with classifications of single versus once coupled and divorced or separated versus single or married. We reclassified occupational data into dichotomous dummy variables of professional occupations, trade and clerical workers, and labourers. We treated the home duty, student, or pensioned classification as the reference group for occupational status. We examined regional contributions to the dependent variables by classifying metropolitan versus rural versus remote regional areas. We tested for multivariate outliers on the predicted variables by using Mahalanobis distance and multicolinearity among independent variables on the predicted values before performing the regression analysis.
We also used the multiple linear regression model proposed by Hopper and Seeman to further assess within pair differences in use of health services before and after the index date, with adjustment for possible confounders.15
Use of health services before the index date —Paired samples t test between subjects and controls (table 1) revealed no significant differences in admissions before the index date (t=1.333, P=0.184). However, both inpatient bed days (t=3.104, P=0.002) and outpatient contacts (t=6.357, P<0.00005) were significantly higher for subjects than for controls.
Use of health services before and after the index date —Paired samples t tests showed significant reductions in inpatient admissions and bed days for both subjects and controls and significant increases in outpatient contacts for subjects only (table 2).
Predicting inpatient bed days after the index date —Significant predictors of bed days within the model included age, admissions, and inpatient bed days before the index date (table 3). Group membership (subject versus control) did not significantly predict subsequent inpatient bed days.
Predicting inpatient admissions after the index date —Admissions and outpatient contacts before the index date were significant predictors of subsequent admissions (table 3). Aboriginal ethnicity showed a tendency for Aboriginal people to be admitted more frequently than non-Aboriginal people. Community treatment order status did not significantly predict subsequent inpatient admissions.
Predicting outpatient contacts after the index date —Outpatient contacts before the index date and being placed on a community treatment order were associated with significantly higher subsequent outpatient contacts (table 3). Residential location showed a tendency for more outpatient contacts to occur in metropolitan areas than in rural or remote areas of Western Australia.
Within pair differences —We assessed the effects of placement on a community treatment order on inpatient admissions and bed days over the year before and after the index date by using multiple regression models with within pair differences as dependent variables. No covariates were shown to be associated with the within pair differences in inpatient admissions and bed days, and no variation in results was seen after adjustment for covariates (age, Aboriginal ethnicity, diagnosis type, marital status, occupation, region).
Limitations of the study
Although the matching process allowed selection of controls with similar characteristics to the patients placed on community treatment orders, significant differences in prior outpatient contacts and inpatient bed days existed. This meant that these variables had to be included in the regression models to control for their effects. This has further limitations, as controlling for variables may not be sufficient and additional confounders may be present. For example, it was not possible to control for variables such as social disability or dangerousness with the available database. Patients placed on an order may have greater degrees of dangerousness or social disability. These factors, in turn, may effect subsequent use of health services.
As this was an epidemiological study, we could not study clinical outcomes. However, the size of the study and the use of an epidemiological sampling frame that spanned the introduction of community treatment orders and covered all patients placed on such an order meant that the sample was representative of patients placed on an order and not subject to referral or selection bias. The design also permitted before and after comparisons of patients placed on an order and those not placed on an order for a greater number of subjects than has been previously possible. 6 10 12 The regression modelling allowed for multivariate examination of a wide range of predictors of subsequent use of health services for both subject and control groups.
Both groups showed a reduction in inpatient bed days, suggesting a regression to the mean in that patients with high rates of use will have lower rates of subsequent use. The finding that this effect was seen equally in both groups may indicate that the groups were similar. The regression results for within pair differences in inpatient admissions or bed days, and their covariates, also showed the similarity of the two groups.
Predictors of service use
Each of the three regression models showed that previous use of services, including number of admissions and outpatient contacts, contributed to subsequent use of health services. Other predictors, including education, occupational status, and diagnosis, did not contribute to subsequent use of services. Neither did being placed on a community treatment order contribute to subsequent bed days or admissions, indicating similar outcomes between subject and control groups. Being placed on an order did, however, predict higher subsequent outpatient contacts. Such results indicate that history of inpatient treatment was a greater predictor of subsequent use of services than being placed on a community treatment order. These results show that although placement on an order leads to a greater likelihood of increased outpatient contacts, this increase does not contribute to a greater reduction in inpatient bed days than is seen in the patients who were not placed on an order.
Although patients who were placed on a community treatment order showed reductions in inpatient admissions and length of inpatient stay one year after placement of the order, these improvements were no greater than those seen in controls who had not been placed on an order, after adjustment for diagnosis, sex, and previous use of health services. It is unlikely that this can be attributed to selection bias between the two groups, as service use and patient characteristics were controlled for in the regression models.
Are community treatment orders effective?
So the question still remains—are community treatment orders effective? This study provides mixed results. Although orders reduce admission rates and bed days, the effect is no greater than that seen in a group of patients who are not on such an order, after adjustment for possible confounders. The study therefore raises questions about the effectiveness of such an invasive procedure as enforcing treatment in the community. If efficacy is in part defined by reduced hospital admissions and length of stay, our results suggest that the policy is no more effective than not enforcing community treatment. Our results are consistent with previous studies of community treatment orders, which showed no significant improvements in outcome compared with matched controls, 11 12 but not with findings from England and Massachusetts, where patients on “extended leave” or involuntary outpatient treatment spent less time in hospital than did matched controls. 6 12 Possible explanations might be differing selection criteria and the fact that only age, sex, and diagnosis were controlled for in the design of both these studies. On the other hand, the British study included an assessment of dangerousness that was not recorded in the Western Australia Mental Health Information System.
This study examines only whether community treatment orders reduce immediate use of health services, as this has been the main reason for their introduction and hence the focus of previous research into their effectiveness.6 Such orders may reduce use of health services in the longer term or produce benefits in other areas, such as psychosocial functioning.
Some researchers have called for randomised assignment of patients to community treatment orders as a way of measuring the efficacy within a quasi-experimental paradigm,14 but this can be logistically difficult if the order is tied to statutory legislation. An alternative would be to compare patients from jurisdictions with and without community treatment orders and matched on sociodemographic, clinical, and health service characteristics. Such comparisons would be desirable before the introduction of similar measures in other jurisdictions and would ensure that policy in this area is evidence based. This would be consistent with the increased interest in evidence based public policy in several countries.16
This study shows that legislative solutions such as community treatment orders may not always offer a solution to the need to provide appropriate services for psychiatric patients within limited resources. It is important to examine what role such orders have in providing effective mental health treatment and whether therapeutic gains could be better delivered by enhancing the quality and assertiveness of community treatment for high risk patients.
We thank Tom Pinder and Alan Joyce at the Mental Health Information Centre, Health Department of Western Australia, and Neville Barber and Sue Lewis at the Mental Health Review Board for their assistance in the merging and extraction of Mental Health Information System data and community treatment order data.
Contributors: NJP had the original idea for the study and is the guarantor. NJP and SK participated in conceptualising the research issues and determining the hypotheses to be tested, undertook the literature review, and wrote much of the manuscript. JX extracted the data and created the database. JX and NJP conducted the statistical analyses.
Funding Medical Research Foundation of Fremantle Hospital, Western Australia.
Competing interests None declared.