Papers

Randomised comparison of the effectiveness and costs of community and hospital based mental health services for children with behavioural disorders

Richard Harrington, professor ^a, Sarah Peters, research associate ^a, Jonathan Green, senior lecturer ^a, Sarah Byford, research fellow ^b, Jane Woods, research assistant ^a, Ruth McGowan, specialist registrar ^a. 

^a Department of Child and Adolescent Psychiatry, University of Manchester, Manchester M27 4HA, ^b Centre for Health Economics, University of York, York YO1 5DD

Correspondence to: R Harrington, University Department of Child and Adolescent Psychiatry, Royal Manchester Children's Hospital, Pendlebury, Manchester M27 4HA R.C.Harrington{at}man.ac.uk

	Abstract

Top Abstract Introduction Participants and methods Results Discussion References

Objective: To test the hypothesis that a community based intervention by secondary child and adolescent mental health services would be significantly more effective and less costly than a hospital based intervention.
Design: Open study with two randomised parallel groups.
Setting: Two health districts in the north of England.
Participants: Parents of 3 to 10 year old children with behavioural disorder who had been referred to child and adolescent mental health services.
Intervention: Parental education groups.
Main outcome measures: Parents' and teachers' reports of the child's behaviour, parental depression, parental criticism of the child, impact of the child's behaviour on the family.
Results: 141 subjects were randomised to community (n=72) or hospital (n=69) treatment. Primary outcome data were obtained on 115 (82%) cases a year later. Intention to treat analyses showed no significant differences between the community and hospital based groups on any of the outcome measures, or on costs. Parental depression was common and predicted the child's outcome.
Conclusions: Location of child mental health services may be less important than the range of services that they provide, which should include effective treatment for parents' mental health problems.

	Introduction

Top Abstract Introduction Participants and methods Results Discussion References

Views about where to provide secondary mental health services for children and adolescents have changed repeatedly over the past 50 years. Child psychiatry started as a community discipline in child guidance clinics.¹ There were, however, many practical difficulties in the administration of these clinics,¹ and during the 1970s and '80s many clinics closed and were replaced by hospital based services. ^{1 2} Over the past 15 years, however, hospital based services too have been criticisedon the grounds that they are inaccessible, stigmatising, expensive, poorly integrated with community services, and less likely to produce gains that generalise to other environments, such as school. ^{3 4} Political pressure has also been applied to child mental health services to return to the community.²

The assumption that community based child mental health services lead to better outcomes than hospital based services has not been tested in a randomised trial in the United Kingdom. We conducted such a trial, whose main hypothesis was that for children with behavioural disorders a community based intervention would be significantly more effective and less costly than a hospital based intervention.

	Participants and methods

Top Abstract Introduction Participants and methods Results Discussion References

Participants
The study was based on the parents of children with behavioural disorders. Parents were eligible if they were judged able to participate in groupsfor example, if they did not have a major mental disorderand had children who were (a) aged 3-10 years, (b) had a clinical diagnosis of oppositional disorder,⁵ (c) had normal intelligence (clinical judgment), and (d) had been referred to the child and adolescent mental health service in either of two health districts in the north of England.

Interventions
During the study each district child and adolescent mental health service provided the same intervention either in a children's hospital or in a community settingfor example, health centres and community resource centres. As this was a pragmatic trial of the interventions used in the NHS, each service used their routine interventions for behavioural disorder for the age group being studied. In one of the districts, this was the videotape modelling parental group education programme of Webster-Stratton and colleagues.⁶ The other district used a programme of parental education groups with parallel child groups. In both districts the interventions were provided by various professionals, including community psychiatric nurses, psychologists, social workers, and psychiatrists. Therapists who took part in the study were trained in at least two parental groups before leading a group.

Procedures for assignment and blinding
The unit of randomisation was the parent/index child pair. After written consent had been obtained, an independent statistician at a distant site randomly allocated participants to the community or hospital based interventions, stratified by health district. Group allocation was concealed from the outcome assessor, who was asked at the end of the study to guess which intervention had been given to which parent. These guesses were no better than chance (59/141 or 42% correct). As studies of psychosocial treatments can only ever be single blind, participants' expectancies of treatment could bias the results. The parents' expectancy of treatment after randomisation was therefore assessed with a 0 to 8 continuous scale, where 8 was a very high expectancy that treatment would help.⁷ The mean level of expectancy among parents was similar in both groups (community group: 5.7 (95% confidence interval 5.3 to 6.1); hospital group: 5.8 (5.4 to 6.3)).

Assessment of effectiveness
Measures were completed before treatment and at two follow up stagesthree months after treatment started and at about one year. Because parental perception of a child's problems is one of the most important determinants of the use of services,⁸ the primary outcome was parental report of the child's behaviour.⁹ Secondary outcomes comprised the teacher's report of the child's behaviour,⁹ parental reports of the impact of the child's behaviour on the family,¹⁰ parental criticism of the child (assessed by counting the number of critical comments during a five minute speech sample¹¹), and parental perception of parenting problems.¹² Parental depression was assessed with the Beck questionnaire.¹³ In line with previous research in the United Kingdom with the parents of children with behavioural disorder,¹⁴ a score of 15 or more on the Beck scale was categorised as "high," indicating a high level of maternal depression.

Assessment of costs
Information on the use of all services by both the children and their primary carer (usually the mother) during the trial was collected from the primary carer at the final follow up interview. A questionnaire was designed for the purpose of the study and based on data collection methods developed in a previous trial.¹⁵ The perspective of the trial was that of all service providers, including the NHS, social services departments, education departments, and voluntary and private sectors. This enabled the differential impact of the parenting skills groups on each sector to be clearly quantified. In addition, the cost of travel to attend the sessions and the cost of the crèche facilities were recorded. Unit costs were for the financial year 1998-9 and were collected from local service providers or national published unit costs^16-18 or calculated directly from relevant salary scales. All future costs were discounted at an annual rate of 5%. Further details on the costing of services are available on request.

Statistical analysis
We projected a sample size of about 40 cases in each group for two reasons. Firstly, before starting the study we had asked the purchaser (the health authority) and the provider of the existing child and adolescent mental health service in one of the districts (Salford) how large the difference in treatment effect between a community and hospital service would have to be to influence their plans for child and adolescent mental health services. These services were being reviewed at the time of the study, and both the purchaser and the provider made a commitment to implement the findings. They agreed that only a large difference (effect size of 0.8 for the mean difference in parental report of child behaviour between community and hospital treatment at the 3 month assessment) would lead to changes in the ways that services were delivered. Thirty four cases per group are needed for a 90% chance of detecting this difference with a two sided test at the conventional 5% significance level.¹⁹ Secondly, a Canadian study comparing community and clinic based child mental health services found a significant difference in parental reports of child behaviour with about 40 cases in each group.⁴

	Results

Top Abstract Introduction Participants and methods Results Discussion References

Participant flow and follow up
The figure shows the trial profile. In all, 187 eligible parents were referred to the trial. Of these, 25 failed to respond to an invitation to take partfor example, did not attendand 21 refused randomisation. The remaining 141 parents were then randomly allocated to community (n=72) or hospital (n=69) treatment; 115/141 (82%) cases had complete data at baseline and at follow up one year later.

View larger version (20K): [in this window] [in a new window]   Trial profile of eligible participants

Participants' and children's characteristics
The median age of the children was 6.9 (range 3-10) years, and 112 (79%) were boys. The primary carer, who completed the parental questionnaires, was the mother in 136 out of 141 cases. In 94 out of 141 (67%) cases the parents were receiving state benefits, and in 61 out of 141 (43%) cases the parents were single. Seventy three out of 134 (54%) parents who completed the Beck depression questionnaire at baseline had a "high" score. The groups did not differ significantly in respect of these characteristics.

Effectiveness
The two groups did not differ significantly on any of the measures at baseline or at either of the two follow up assessments (table 1). There were no significant effects of location of treatment (community v hospital ) on changes in any of the outcomes (table 2). A high proportion of children (89/116 (77%)) had a high score (127) on parental report of the intensity of behavioural problems at the three month assessment. The risk of children having a high score after treatment was greater (odds ratio 2.8; 1.3 to 6.1) for those whose primary carer had a high score on the Beck depression questionnaire at baseline (48/55) compared with those with a low score (14) (36/56). Confounding by social class or behavioural problems at baseline did not account for this association.

Costs
Altogether, 118 out of 141 (84%) parents (61 in the community group, 57 in the hospital group) completed the resource questionnaire at the final follow up interview; these responses were included in the economic evaluation (table 3). The mean length of follow up was similar in both groups (65.7 weeks in the community group, 65.0 in the hospital group). To remove the influence of this small difference, however, costs per week (as well as overall cost) were calculated. No significant differences between the hospital and community groups were found in the mean overall cost per child (difference £904 (£1254 to £3062); P=0.41), per primary carer (£611 (£143 to £1365); P=0.11), or in total (£1515 (£742 to £3772); P=0.19), or in terms of cost per week. To assess the robustness and generalisability of the results, several univariate sensitivity analyses were carried out. These analyses did not affect the results.

	Discussion

Top Abstract Introduction Participants and methods Results Discussion References

The present study did not find that community based treatment was more effective than hospital based treatment. This finding contrasts with that of Cunningham and colleagues⁴ in Canada, who reported that children with behavioural problems treated in the community had better outcomes than those referred to a specialised clinic. It is, however, difficult to interpret the results of that study because outcome data were obtained on less than a third of randomised subjects.

Methodological issues
Three issues should be borne in mind when interpreting the results of the present study. Firstly, the trial was powered on the basis of a significant difference in clinical outcomes. The sample size may have been too small to detect a significant difference in costs. Indeed, the actual cost differences found between the two groups were large, with the hospital group costing 30% less overall than the community group. Secondly, there are several different models of community child and adolescent mental health services. This study modelled the common situation in which secondary services were located in just one or two community settings. Different results might have been obtained if we had studied primary care interventions, such as training health visitors in parental education methods,²⁶ or interventions in which mental health professionals support community schemes such as befriending.²⁷ These interventions have, however, seldom been evaluated in randomised trials in the United Kingdom. Thirdly, the results from this study may not apply to the treatment of other child psychiatric disorders or, indeed, to the treatment of behavioural problems using other methods.

Planning services for children with behavioural disorders
We conclude that in planning services for children with behavioural disorders, greater attention must be paid to factors other than the location. Our findings suggest, for example, that a service for children with behavioural problems must also be able to call not just on parental education groups but also on interventions such as effective treatments for parental depression. The finding that parental depression was more strongly associated with poor child outcomes in the presence of poor compliance with treatment suggests, however, that poor compliance might be partly responsible for the failure of treatment in the children of depressed parents. This group may therefore need extra help to attend child mental health services. As a substantial proportion of the costs associated with behavioural disorders in children are borne by the educational services, there may also be scope for further developing the role of mental health workers in schools.

	Acknowledgments

We thank Fiona Campbell and Daphne Kounali of the department of statistics at Hope Hospital, Salford, who conducted the randomisation, and the therapists who conducted the parental education groups; Chrissie Pickin of Salford and Trafford Health Authority; and Sibyl Zaden of the University of California, who rated the five minute speech samples.

Contributors: RH and JG had the idea for the study, obtained the grant, and managed the project. RH conducted the analyses and wrote up the study. SP and JW conducted the outcome assessments, contributed to the design, and revised the final manuscript. SB designed the economic evaluation and conducted the economic analysis. RMcG conducted many of the parental groups and revised the manuscript. RH will act as guarantor for the paper.

	Footnotes

Funding: Research grant from the NHS Executive's Motherhood and Child Health initiative. Clinical costs were provided by Manchester Children's Hospital NHS Trust (funding for two of the therapists to train in the Webster-Stratton programme in Seattle, United States) and by Salford social services.

Competing interests: None declared.

	References

Top Abstract Introduction Participants and methods Results Discussion References

1.	Parry-Jones W. Adolescent psychiatric services: development and expansion. In: Harris Hendricks J, Black M, eds. Child and adolescent psychiatry: into the 1990s. London: Royal College of Psychiatrists, 1990:83-89.
2.	Kolvin I. Child and adolescent psychiatry: into the 1990s. In: Harris Hendricks J, Black M, eds. Child and adolescent psychiatry: into the 1990s. London: Royal College of Psychiatrists, 1990:113-116.
3.	Nicol R. Practice in nonmedical settings. In: Rutter M, Taylor E, Hersov L, eds. Child and adolescent psychiatry: modern approaches. 3rd ed. Oxford: Blackwell Scientific, 1994:1040-1054.
4.	Cunningham CE, Bremner R, Boyle M. Large group community-based parenting programmes for families of preschoolers at risk for disruptive behaviour disorders: utilization, cost effectiveness, and outcome. J Child Psychol Psychiatry 1995; 36: 1141-1159[Medline].
5.	World Health Organization. The ICD-10 classification of mental and behavioural disorders. Diagnostic criteria for research. Geneva: WHO, 1993.
6.	Webster-Stratton C. A randomized trial of two parent training programs for families with conduct disordered children. J Consult Clin Psychology 1984; 52: 666-678[CrossRef][Medline].
7.	Marks I. Cure and care of neuroses. New York: Wiley, 1981.
8.	Meltzer H, Gatward R, Goodman R, Ford T. Mental health of children and adolescents in Great Britain. London: Stationery Office, 2000.
9.	Eyberg SM, Ross AW. Assessment of child behavior problems: the validation of a new inventory. J Clin Child Psychol 1978; 7: 113-116.
10.	Ablow JC, Measelle JR, Kraemer HC, Harrington R, Luby J, Smider N, et al. The MacArthur three city outcome study: evaluating multi-informant measures of young children's symptomatology. J Am Acad Child Adolesc Psychiatry 1999; 38: 1580-1590[CrossRef][Medline].
11.	Magna A, Goldstein M, Karno M, Miklowitz D, Jenkins J, Falloon I. A brief method for assessing expressed emotion in relatives of psychiatric patients. Psychiatry Res 1986; 17: 203-212[CrossRef][Medline].
12.	Arnold D, O'Leary S, Wolf L, Acker M. The parenting scale: a measure of dysfunctional parenting in discipline situations. Psychol Assessment 1993; 5: 137-144[CrossRef].
13.	Beck AT. Depression: clinical, experimental and theoretical aspects. New York: Harper and Row, 1967.
14.	White C, Barrowclough C. Depressed and non-depressed mothers with problematic preschoolers: attributions for child behaviours. Br J Clin Psychol 1998; 37: 385-398.
15.	Byford S, Harrington RC, Torgerson D, Kerfoot M, Dyer E, Harrington V, et al. Cost-effectiveness analysis of a home-based social work intervention for children and adolescents who have deliberately poisoned themselves: the results of a randomized controlled trial. Br J Psychiatry 1999; 174: 56-62[Abstract/Free Full Text].
16.	Netten A, Dennett J, Knight J. Unit costs of health and social care. Canterbury, Kent: Personal Social Services Research Unit, University of Kent at Canterbury, 1999.
17.	Chartered Institute of Public Finance and Accountancy. The health service financial database 1998. London: CIPFA, 1998.
18.	Chartered Institute of Public Finance and Accountancy. The health service financial database 1997-98: actuals. London: CIPFA, 1998.
19.	Machin D, Campbell MJ. Statistical tables for the design of clinical trials. Oxford: Blackwell, 1987.
20.	Efron B, Gong G. A leisurely look at the bootstrap, the jack-knife and cross validation. Am Statistician 1983; 37: 36-48.
21.	Barber JA, Thompson SG. Analysis and interpretation of cost data in randomised controlled trials: review of published studies. BMJ 1998; 317: 1195-1200[Abstract/Free Full Text].
22.	Routh CP, Hill JW, Steele H, Elliot CE, Dewey ME. Maternal attachment status, psychosocial stressors and problem behaviour: follow-up after parent training courses for conduct disorder. J Child Psychol Psychiatry 1995; 36: 1179-1198[Medline].
23.	Webster-Stratton C. The effects of father involvment in parent training for conduct problem children. J Child Psychol Psychiatry 1985; 26: 801-810[CrossRef][Medline].
24.	Kazdin AE, Mazurick JL. Dropping out of child psychotherapy: distinguishing early and later dropouts over the course of treatment. J Consult Clin Psychol 1994; 62: 1069-1074[CrossRef][Medline].
25.	Hollis S, Campbell F. What is meant by intention to treat analysis? Survey of published randomised controlled trials. BMJ 1999; 319: 670-674[Abstract/Free Full Text].
26.	Davis H, Spurr P. Parent counselling: an evaluation of a community child mental health service. J Child Psychol Psychiatry 1998; 39: 365-376[CrossRef][Medline].
27.	Cox AD. Befriending young mothers. Br J Psychiatry 1993; 163: 6-18[Abstract/Free Full Text].

(Accepted 1 August 2000)