Jump to: Page Content, Site Navigation, Site Search,
You are seeing this message because your web browser does not support basic web standards. Find out more about why this message is appearing and what you can do to make your experience on this site better.
BMJ 2007;335:336 (18 August), doi:10.1136/bmj.39251.599259.55 (published 13 July 2007)
Tom Burns, professor of social psychiatry1, Jocelyn Catty, research fellow2, Michael Dash, research assistant2, Chris Roberts, statistician3, Austin Lockwood, research nurse4, Max Marshall, professor of community psychiatry4
1 University of Oxford, Warneford Hospital, Oxford OX3 7JX, 2 Division of Mental Health, St George's, University of London, London SW17 0RE, 3 School of Epidemiology and Health Sciences, University of Manchester, Manchester, 4 Division of Psychiatry and Behavioural Sciences, University of Manchester, Preston
Correspondence to: T Burns Tom.burns{at}psych.ox.ac.uk
Design Systematic review with meta-regression techniques applied to data from randomised controlled trials.
Data Sources Cochrane central register of controlled trials, CINAHL, Embase, Medline, and PsychINFO databases from inception to January 2007. Additional anonymised data on patients were obtained for multicentre trials.
Review methods Included trials examined intensive case management compared with standard care or low intensity case management for people with severe mental illness living in the community. We used a fidelity scale to rate adherence to the model of assertive community treatment. Multicentre trials were disaggregated into individual centres with fidelity data specific for each centre. A multivariate meta-regression used mean days per month in hospital as the dependent variable.
Results We identified 1335 abstracts with a total of 5961 participants. Of these, 49 were eligible and 29 provided appropriate data. Trials with high hospital use at baseline (before the trial) or in the control group were more likely to find that intensive case management reduced the use of hospital care (coefficient –0.23, 95% confidence interval –0.36 to –0.09, for hospital use at baseline; –0.44, –0.57 to –0.31, for hospital use in control groups). Case management teams organised according to the model of assertive community treatment were more likely to reduce the use of hospital care (coefficient –0.31, –0.59 to –0.03), but this finding was less robust in sensitivity analyses and was not found for staffing levels recommended for assertive community treatment.
Conclusions Intensive case management works best when participants tend to use a lot of hospital care and less well when they do not. When hospital use is high, intensive case management can reduce it, but it is less successful when hospital use is already low. The benefits of intensive case management might be marginal in settings that have already achieved low rates of bed use, and team organisation is more important than the details of staffing. It might not be necessary to apply the full model of assertive community treatment to achieve reductions in inpatient care.
Assertive outreach was previously referred to in the UK as intensive case management, a somewhat broader term emphasising small caseloads. It is the term used for the UK application of the North American service assertive community treatment, a carefully specified approach to case management,2 which includes daily team meetings, case sharing, 24 hour availability, and multidisciplinary working with doctors as full team members.
Under intensive case management, each person with severe mental illness at high risk of readmission is allocated a nurse, social worker, or other clinician (a "case manager") who carries a small caseload of between 10 and 20 patients. This case manager takes primary responsibility for keeping contact with the patient, assessing their needs, and ensuring that these needs are met.5 Intensive case management is one of the most thoroughly evaluated non-pharmaceutical interventions in psychiatry, but numerous trials over the past 35 years have failed to show that it consistently reduces the use of hospital care. While some trials have shown a large reduction, others have found no effect, and some have shown a significant increase.
This inconsistent effect on hospital care might be explained by differences in the contexts in which the trials were conducted. Some have suggested that intensive case management is effective only in the US, with the presumption that care in US control groups is inferior to that in state funded welfare systems such as in the UK.3 Alternatively, case managers in these studies might not be working as a team, as advocated by the model of assertive community treatment.6 Others have proposed that intensive case management is less effective in settings where control groups are contaminated with case management practices (a particular feature of more recent trials).7 8 9 Another intriguing suggestion is that intensive case management is effective only against a background of high reliance on hospital care.10 11
An alternative hypothesis is that the inconsistent results are caused by different models of intensive case management being practised in different trials. In particular it has been suggested that hospital care was reduced only in trials where assertive community treatment was practised.
We aimed to test all these hypotheses by applying meta-regression techniques to data from randomised controlled trials of intensive case management.
Study selection
We excluded trials in which the experimental intervention was an acute crisis team or if the control condition was hospital admission, remaining in hospital, or an alternative form of intensive case management. We also excluded those in which most participants were aged under 18 or over 65 or had a primary diagnosis of organic brain disorder or learning disability. Two authors (MM and AL or JC and MD) screened each abstract and sought potentially relevant articles. They independently reviewed articles and constructed a table of included studies. Any disagreements were resolved by discussion with a third reviewer (JC or TB, respectively). We categorised each included trial for allocation concealment according to the Cochrane Collaboration Handbook14 and included only trials rated A or B.
Data extraction
The dependent variable in the meta-regression was time in hospital, defined as mean number of days per month in hospital. Data on the dependent variable were eligible only if provided on an intention to treat basis and available for more than half the trial participants. Two independent reviewers (MM and AL) extracted data from trial reports and cross checked data by double entry. If possible, we calculated the dependent variable on the basis of a 24 month follow-up period as this offered the best trade off between length of follow-up and attrition of participants. When a trial did not report 24 month data, we calculated the dependent variable from the nearest available follow-up point. When a trial reported a mean but no SD, we imputed the missing parameter using a regression analysis of SD against mean, based on data from trials that provided SDs.
We also noted the degree of low intensity case management in the control group (all, some, or none), the country where trial took place (US or non-US); mean days per month in hospital for participants in the two years before the study began (baseline hospital use); the year the study began; trial size; and a rating of fidelity of the intervention to assertive community treatment on the "team membership" and "team structure and organisation" subscales of the index of fidelity to assertive community treatment (IFACT).15 This index was derived from a survey of 20 clinical experts in assertive community treatment and validated in a survey of 18 programmes. The team membership subscale comprises four items: ratio of patients to staff, total size of the team, and the extent of psychiatric and nursing input to the team. The structure and organisation subscale comprises seven items: whether the team is the primary source of care for its patients, is situated away from the hospital, meets daily, shares responsibility for caseloads, is available 24 hours a day, has a team leader who is also a case manager, and offers unlimited time for its services. We chose the index because the subscales are brief and can be completed from published data. For each item on the index, a score of 1 indicates high fidelity to the model. Fidelity data were obtained from published and unpublished trial reports; direct contact with trialists (who either completed the scale from memory or supplied fidelity data collected contemporaneously); and data previously obtained directly from trialists by a previous review.16 17 Two raters (MM and AL) independently combined these data into a single fidelity score. Discrepancies were resolved by discussion and, if necessary, by contacting trialists. Items for which no data were available were assigned a null value because we assumed that trialists would report their efforts to achieve fidelity to the assertive community treatment model.
Multicentre trials of intensive case management often struggle to implement a uniform approach,18 with centres operating at different degrees of fidelity to assertive community treatment. We exploited this to increase the power of the meta-regression by disaggregating multicentre studies into component centres with outcome and fidelity data for each. Trialists provided data either in summary form or as anonymous data on individual patients with the permission of their institutional review boards. We verified independently calculated centre data against original trial reports.
Data synthesis
The box shows the relation between the covariates and the hypotheses tested. We also included trial size as a covariate to check for publication bias. We used Stata to carry out a random effects meta-analysis with random effects for variation between centres and between studies using a three level random effects model with a numerical constraint on the level 1 variation. This was fitted with generalised linear latent and mixed models (GLLAMM), a Stata algorithm with Gaussian quadrature based maximum likelihood estimation.19
|
Initially, we excluded baseline hospital use as a covariate as this was not available for all centres. Subsequently, we repeated the meta-analysis including only centres for which we had baseline data on hospital use. To reduce the possibility of type 2 errors, this second analysis included only covariates that were found to have a significant association with mean number of days in hospital in the first analysis.
We then carried out two sensitivity analyses on the full dataset to verify our findings. Firstly, we included the mean of the control group as a covariate instead of the baseline measure of hospital use. This had the advantage of being available for all centres but was potentially flawed because of mathematical coupling and regression to the mean. Therefore we carried out a second sensitivity analysis by modelling the means of the treatment group rather than the intervention effect. In this model, the relation between the intervention effect and the underlying effect can be assessed by including a random intercept, which assesses the underlying mean for that study or centre, and a random coefficient for intervention whose variance is equivalent to the variance between studies in a conventional meta-analysis of intervention effects. For the covariance between these random effects we can estimate a regression coefficient for the relation between intervention effect and the control group mean.
Finally, we performed two further sensitivity analyses: the first excluded trials with imputed SDs; and the second excluded trials where the Cochrane randomisation category was B.
|
We had complete data for low intensity case management in the control group (all, some, or none), country (US or non-US), the year the trial started, and the size of the trial. In 26 centres the control group was standard care without any low intensity case management; in 13 at least 10% of the control group members received low intensity case management; and in 13 all control group members received low intensity case management. Thirty five centres were from the US (3271 patients); 15 were from the European Union (2494 patients), including 13 from the UK (n=1541 patients); and two (n=196 patients) were from elsewhere (Canada and Australia). One trial began in the 1970s, 28 in the 1980s, 21 in the 1990s, and two thereafter. We directly contacted trialists of 20 out of 29 eligible trials to obtained fidelity data. For the 52 centres, we obtained 95% complete fidelity data on the team membership and organisation subscales of the fidelity index (mean total fidelity score 6.6, 95% confidence interval 7.16 to 6.02; maximum score 11, see www.psychiatry.ox.ac.uk/socpsych/bmjmrtable); 42 centres provided data on use of inpatient care before the trial. The 10 centres that did not report it included two services exclusively treating patients with their first psychotic breakdown.w18w23
A preliminary meta-analysis showed a small but significant effect in favour of intensive case management but with significant heterogeneity between centres (pooled intervention effect –0.46, 95% confidence interval –0.84 to –0.08, P=0.019; variation between centres 0.32, variation between trials 0.32).
Meta-regression on days in hospital
Table 1 shows the meta-regression of mean difference in days per month in hospital (unstandardised) against all covariates except baseline days in hospital (as this covariate was not available for all centres).
It shows that the more a case management team is organised like an assertive outreach team, the better it is at reducing time spent in hospital. The regression coefficient for team organisation indicates a decrease of 0.44 (95% confidence interval 0.72 to 0.17) days in hospital per month for each one point increase on the index organisation scale. The scatter plot of organisation subscore versus mean days in hospital (fig 2) shows that the effect of intensive case management on hospital use is negligible if the team organisation fidelity score is low.
No other covariates were significant, including the index team membership subscale (fig 3), and there was no evidence of variation within trials between centres.
There was, however, continuing evidence of variation between trials, suggesting that differences on the organisation subscore did not entirely explain the heterogeneity between trials.
|
|
|
|
When we dropped centres with an imputed SD from the analysis, index organisation score and baseline level of hospital use were no longer significant. This was because a reduction in the number of data points increased the influence of an outlying centre visible in figures 4 and 5.![]()
When we omitted this centre from the analysis the effects remained (see www.psychiatry.ox.ac.uk/socpsych/bmjmrtable).
|
|
Strengths and weaknesses
We made maximum use of the available data, not only by identifying all eligible randomised trials but also by obtaining previously unpublished fidelity data directly from trialists and substantial amounts of data on individual patients to facilitate the disaggregation of multicentre trials into their component centres. However, our fidelity data were only 95% complete. Nine of 14 trials with missing fidelity items had lower hospital use in the intervention arm, so if all these items had received a positive rating, this could have strengthened the relation between fidelity to the model and reduction of days in hospital. We also found it necessary to use imputed SDs for the data provided by 16 of the 52 centres, and a sensitivity analysis without these centres reduced the strength of some of the observed associations.
Notwithstanding these reservations, our study applies an empirical test to the key hypotheses that have been advanced to explain why intensive case management is so inconsistent in reducing hospital care. We have shown that hypotheses based on where and when the trial was conducted can be discounted. We have also shown that fidelity to the assertive community treatment model, insofar as it is important, relates only to aspects of the model concerned with team organisation (in essence, the extent to which case managers work as a team rather than as independent practitioners). Specific staffing features of the assertive community treatment model (such as team size, low caseloads, and professional make up of the team) are not important in reducing hospital use among the team's patients.
Implications of findings
Why is it that the level of hospital use among a trial's participants is so important in determining the effectiveness of intensive case management? We think that low levels of hospital use are a proxy for good community services. Where community services are good, hospital care is used sparingly and only when absolutely necessary. Under such circumstances even intensive case management teams find it difficult to have an impact on hospital use. When community services are poor, it is usually fairly easy for patients to spend long periods of time in hospital, and intensive case management teams find it relatively easy to reduce such wastefulness. There are, however, other explanations. Firstly, low levels of hospital use might indicate that a trial's participants are less severely ill and hence less likely to benefit from intensive case management. Secondly, low levels of hospital use may indicate that it is difficult to obtain admission within the setting in which the trial is being conducted, leaving little scope for the intensive case management team to achieve further reductions.
The main clinical implication of our study is that the introduction of intensive case management teams will not lead to substantial reductions in hospital use in settings where average hospital use is already low. Teams can optimise their ability to reduce hospital use by organising themselves in the manner recommended for assertive community treatment, and by focusing on patients with a history of high hospital use. Replicating staffing requirements of assertive community treatment does not confer measurable benefits. Our study confirms a growing recognition that we should research the practices of teams rather than their labels.22w29
|
Contributors: TB is guarantor. TB and MM conceived the review. JC, MD, MM, and AL retrieved the papers and contacted authors. MM and AL prepared the tables and figures. MM, AL, and CR extracted the data, calculated estimates of effects, and dealt with the statistics. All authors read and commented on the paper.
Funding: Policy research programme, Department of Health.
Competing interests: None declared.
Ethical approval: Not required.
![]()
CiteULike
Complore
Connotea
Del.icio.us
Digg
Reddit
StumbleUpon
Technorati What's this?
Read all Rapid Responses