Parenting intervention in Sure Start services for children at risk of developing conduct disorder: pragmatic randomised controlled trial
BMJ 2007; 334 doi: https://doi.org/10.1136/bmj.39126.620799.55 (Published 29 March 2007) Cite this as: BMJ 2007;334:678
This article has a correction. Please see:
- Judy Hutchings, research director1,
- Tracey Bywater, project trial coordinator1,
- Dave Daley, senior research tutor1,
- Frances Gardner, reader in child and family psychology2,
- Chris Whitaker, statistician1,
- Karen Jones, research assistant1,
- Catrin Eames, research assistant1,
- Rhiannon T Edwards, senior research fellow in health economics3
- 1School of Psychology, University of Wales Bangor, Bangor, Gwynedd, LL57 2DG
- 2Department of Social Policy and Social Work, University of Oxford
- 3Centre for the Economics of Health, Institute of Medical and Social Care Research (IMSCaR), University of Wales Bangor
- Correspondence to: J Hutchings j.hutchings{at}bangor.ac.uk
- Accepted 23 January 2007
Abstract
Objective To evaluate the effectiveness of a parenting programme as a preventive intervention with parents of preschool children considered to be at risk of developing conduct disorder.
Design Pragmatic randomised controlled trial using a block design with allocation by area.
Setting Eleven Sure Start areas in north and mid-Wales.
Participants 153 parents from socially disadvantaged areas, with children aged 36-59 months at risk of conduct disorder defined by scoring over the clinical cut off on the Eyberg child behaviour inventory. Participants were randomised on a 2:1 basis, 104 to intervention and 49 to remaining on the wait listing (control). Twenty (13%) were lost to follow-up six months later, 18 from the intervention group.
Intervention The Webster-Stratton Incredible Years basic parenting programme, a 12 week group based intervention.
Main outcome measures Problem behaviour in children and parenting skills assessed by self reports from parents and by direct observation in the home. Parents' self reported parenting competence, stress, and depression. Standardised and well validated instruments were used throughout.
Results At follow-up, most of the measures of parenting and problem behaviour in children showed significant improvement in the intervention group. The intention to treat analysis for the primary outcome measure, the Eyberg child behaviour inventory, showed a mean difference between groups of 4.4 points (95% confidence interval 2.0 to 6.9, P<0.001) on the problem scale with an effect size of 0.63, and a mean difference of 25.1 (14.9 to 35.2, P<0.001) on the intensity scale with an effect size of 0.89.
Conclusion This community based study showed the effectiveness of an evidence based parenting intervention delivered with fidelity by regular Sure Start staff. It has influenced policy within Wales and provides lessons for England where, to date, Sure Start programmes have not been effective.
Trial registration ISRCTN46984318
Introduction
Antisocial behaviour in young people is a growing problem. In the United Kingdom and the United States about 5-10% of children aged 5-15 present with clinically important conduct disorders.1 2 Higher rates occur in single parent families and families with frequent changes of parental figures and parental substance misuse, psychopathology, marital problems, and poor parenting skills. Up to 20% of children in disadvantaged areas have conduct disorders.3
Early onset behavioural problems such as aggression and non-compliance are the best predictors of antisocial and criminal behaviour in adolescence and adulthood.4 Untreated, up to 40% of children with early difficulties develop subsequent conduct disorder, including drug misuse and criminal and violent behaviour.5
Early behavioural difficulties that predict long term problems are easily identifiable, and effective interventions prevent progression into more severe difficulties.6 There are severe financial costs if conduct disorder is not prevented. Use of health, social, education, and legal services is 10 times higher for this population,7 8 mostly borne by publicly funded services, especially in areas of social exclusion.7
Parenting behaviour contributes to the establishment of conduct disorder and many children learn, develop, or establish problem behaviours because parents lack, or inconsistently use, key parenting skills.9 When ineffective parenting is the problem, cognitive behaviourally based parenting programmes can provide an effective solution9 but are more effective with younger children. When problems are less well established parents can more easily influence their children's behaviour.10 One UK government strategy is the Sure Start early preventive parenting support for families of preschool children living in identified high risk, disadvantaged areas. Since its launch in 2001, £3100m (€4500m, $6000m) has been invested in the scheme.11 This funding was provided without direction from government about which services should be delivered. As a result, widely varying services were provided, many lacking evidence of effectiveness from randomised trials. The initial £20m (€30m, $39m) non-randomised, area based evaluation of the first three years in England found no significant effect in preventing or reducing conduct disorder.11 12
There is considerable evidence from randomised trials13 and systematic reviews14 that conduct problems can be prevented and treated with cognitive behavioural parenting interventions. Few trials, however, tested them in real world community settings delivered by existing staff as part of their everyday work. The Webster-Stratton Incredible Years basic parenting programme15 is one of the few “model” programmes for treatment and prevention of conduct disorder that incorporates all factors identified as improving parent training outcomes16 17 and can be used with disadvantaged, high risk families who either do not engage in or drop out of other programmes.6 Randomised trials in UK clinical and voluntary sector settings have shown the programme to be effective.13 18
The Incredible Years programme became established in north Wales through the provision of training, consultation, and support and, since 2001, 11 Sure Start services in north and mid-Wales began using the programme.19 This provided an opportunity for a pragmatic, service setting based trial of the programme as a preventive community intervention with parents of preschool children identified as high risk.
Methods
Study population
The trial took place in 11 Sure Start areas in north and mid-Wales. Health visitors administered the Eyberg child behaviour inventory20 to socially disadvantaged families, with a child aged 3 or 4 years. Families were eligible if the child lived with the primary carer, scored above the clinical cut off on either the Eyberg problem or intensity scale (11 or 127), and the primary carer was able to attend group times. In total 153 families were eligible and consented to take part; 104 were allocated to the intervention group and 49 to the control group. At follow-up 86 in the intervention group and 47 in the control group remained in the study (figure⇓). We included the 20 lost participants in the intention to treat analysis.
Intervention programme
A maximum of 12 parents attended each weekly session, which lasted for 2 to 2.5 hours, over a period of 12 weeks. Two trained leaders introduced a structured sequence of topics using a collaborative approach. All leaders had run at least one previous group. Throughout the 12 week programme leaders received three hours of supervision each week from a certified trainer.
Leaders had varied backgrounds and included social workers, family support workers, Barnardo's project workers, health visitors, and psychologists.
Outline of the intervention
The programme promotes positive parenting through:
Increasing positive child behaviour through praise and incentives
Improving parent-child interaction: relationship building
Setting clear expectations: limit setting and non-aversive management strategies for non-compliance
Applying consistent gentle consequences for problem behaviour
The programme uses a collaborative approach with methods such as:
Role play: acting out certain situations as parent or child
Helping parents to identify social learning principles
Modelling: learning positive behaviours by example
Discussion: with reference to parents' experiences and difficulties while acknowledging their feelings
Skills practise: practising new approaches during the session and through homework
Analysis of video material: viewing tapes of family behaviour for discussion
The programme addresses barriers to attendance by:
Offering transport
Providing lunch
Providing a crèche
The programme addresses fidelity of implementation by:
Providing all course materials, CDs, handouts, books, raffle prizes, etc
Ensuring leaders attended the accredited three day basic leader training and had previously run a group
Completion of peer and self evaluation questionnaires by group leaders to evaluate treatment exposure, adherence, and treatment delivery
Satisfaction questionnaires completed by parents
Completion of session specific checklists to monitor treatment integrity, participant responsiveness, treatment delivery, and treatment differentiation
Supervision to evaluate progress and delivery method—for example, reviewing videotapes during the three hour weekly supervision session with an incredible years trainer
Certification—evaluation of treatment fidelity based on observation of random videotapes by an independent incredible years trainer
Measures and procedures
Measures included questionnaires completed by parents and independent blind observation of parent-child interactions collected by the research team during two home visits, on entry to the trial and six months later. Parents in the intervention group attended sessions in the interim. The control families on the waiting list were offered the programme after follow-up.
Child problem behaviour reported by parents—The primary outcome measure was the Eyberg child behaviour inventory,20 which we used to assess the number and intensity of conduct problems. This was also administered to the sibling closest in age to the index child. Secondary outcome measures were the strengths and difficulties questionnaire21 to assess conduct and hyperactivity problems, the Conners abbreviated parent/teacher rating scale22 as a hyperactivity measure, and the Kendall self control rating scale.23
Parenting competencies, mood, and demographics reported by parents—We used the parenting stress index (short form)24 to assess stress levels, the parenting scale25 to measure parental competencies, the Beck depression inventory,26 and the personal data and health questionnaire27 to assess demographics and risk factors (see table 3⇓).
Observational measure—We used the Dyadic parent-child interaction coding system28 in a 30 minute home observation within three days of the administration of the questionnaires and have reported the summary variables of positive and critical parenting and deviant child behaviours. Inter-rater reliability was maintained through weekly training and reliability visits (20% of total visits). Observers were blind to allocation.
Randomisation
We had no prior knowledge regarding the relation between measures before and after the intervention or of possible differences between the Sure Start areas so we used a two samples independent t test on the change in response measure to estimate the sample size needed. We initially intended to evaluate seven Sure Start areas and expected to recruit 12 intervention and six control families in each area. After we allowed for expected drop out of about 18%, based on similar studies,6 13 we calculated that we would need 126 families to achieve an effect size of 0.8 for the primary outcome measure at the 5% significance level with a ratio of 2:1 intervention to control. After the start of the study it was clear that more areas were needed to achieve the required number of families as some areas failed to recruit 12 intervention and six control families. Ultimately we evaluated 12 groups in 11 areas with an initial total of 153 families.
We used a pragmatic randomised controlled trial design. Participants were block randomised by area. The unit of randomisation was the parent-index child pair. TB blindly and randomly allocated participants on a 2:1 basis, after stratification by sex and age, using a random number generator. This design allows evaluation of a larger intervention sample than a 1:1 ratio with only a small loss of statistical power29 and is a design favoured in this type of research.6 13
Masking
Allocation took place after baseline assessment. Researchers blind to allocation carried out the interviews and observations.
Analysis strategy
All families were included in the analysis irrespective of uptake of intervention. We carried out a strict intention to treat analysis and assumed no change from baseline assessment for those lost to follow-up. An initial analysis of the effects of area as a random effect, treatment as a fixed effect, their interaction, and baseline value showed no significant interaction between treatment and area for the different responses. We have presented the differences between the intervention and control conditions on follow-up scores from analysis of covariance (ANCOVA) of the response, taking account of area, treatment, and baseline response value. Effect sizes were calculated with Cohen's guidelines30 whereby a figure of 0.3 denotes a small but effective change, 0.5 denotes a medium effect size, and 0.8 and above denotes a large effect size. We measured inter-rater observation reliability with Kappa's coefficient.
Results
Baseline characteristics
Eligible families were recruited over 21 months between January 2003 and September 2004 and followed up every six months to March 2005. Tables 1 and 2 show the baseline characteristics of the families, including those lost to follow-up.⇓⇓ Most were socially and economically disadvantaged compared with the mean values for the UK.
Characteristics of children in the sample at baseline. Figures are means (SDs) unless stated otherwise
Family characteristics at baseline. Figures are numbers (percentages) unless stated otherwise
Programme attendance
Of the 104 parents allocated to the intervention condition, 86 completed assessments after the intervention, and, of these, 71 (83%) attended seven or more (58%) of the 12 sessions (Webster-Stratton6 had 88% attend six or more sessions). The overall mean attendance was 9.2 sessions (SD 3.2). From the 18 lost to follow-up in the intervention group, 14 did not attend any session, two parents attended one session, one attended four sessions, and one attended six sessions. The size of the intervention groups varied from five to 12 parents across areas, with an average of seven.
Main findings
Children in the intervention group had significantly reduced antisocial and hyperactive behaviour and increased self control compared with the control group children (table 3).⇓ The intention to treat analysis for the primary child outcome measure, the Eyberg child behaviour inventory, showed a mean difference of 4.4 points on the problem scale between groups at follow-up (95% confidence interval 2.0 to 6.9, P<0.001, effect size 0.63) and a mean difference of 25.1 on the intensity scale (14.9 to 35.2, P<0.001, effect size 0.89). This measure was also completed for sibling closest in age to the index child (range 2-15 years, intervention n=60, control n=29). Compared with parents in the control group, intervention group parents perceived intensity of problems in siblings as less severe at follow-up. For most of the remaining secondary outcome questionnaire measures the intervention families were above the level of clinical concern at baseline but below the level of clinical concern at follow-up. Observational results corroborated the questionnaire findings. There was a mean difference of 9.6 (3.7 to 15.5, P<0.002) between groups at follow-up for positive parenting behaviours with an effect size of 0.57 (table 4).⇓ Levels of parental criticism were reduced at follow-up for the intervention parents, although the difference between the groups was not significant in the intention to treat analyses. The intervention condition showed twice the reduction in observed child deviance than the control condition, although this was not significant. Kappa coefficients for the observational measure showed high reliability between raters (κ=0.91 averaged over the two time points) for the reliability visits (20% of total visits).
Summary of measures in children at baseline and follow-up
Summary of measure in parents at baseline and follow-up
Other secondary measures showed reductions in stress and depression levels, and improvements in parenting competencies in the intervention parents compared with the control parents. The same positive pattern of results was found for the per protocol analyses at follow-up between groups (table 4).⇑
Discussion
In this randomised controlled trial we have shown that the Incredible Years basic parenting programme can reduce key risk factors for the development of conduct disorder. Our findings replicate those of Webster-Stratton6 but in a different real world setting. Although the mean measures and demographic scores were similar at baseline for the two groups, medium to large effect sizes and significant differences at follow-up indicated that the programme had a positive effect on parent and child behaviour in the intervention group. These findings suggest that child behaviour may be mediated through parent behaviour and the learning of key parenting skills.
Strengths and weaknesses of the study
Our study is original in testing effectiveness of an evidence based intervention, delivered with fidelity across multiple community sites, with regular staff from multiple agencies.
We applied strict measures to reduce any possible bias and to ensure that parents were unaware of the group allocation until after the first assessment. Interviewers and observers were kept blind to allocation, and 20% of all observation visits were assessed for inter-rater reliability. Compared with in the control group we found increased positive parenting and reduced problem behaviour in the children in the intervention group at follow-up, not only through subjective parental report but also by more objective direct observation. This method reduces bias that could occur if we relied solely on questionnaire data. Furthermore, observations were coded “live” to enable a precise account of parent-child interaction as it happened and, because recording was in “real time,” observations were minimally interpretative.33 Although our study had a short follow-up (six months after baseline), this is typical in this type of research.
Comparison with other studies
Although our study was in a Welsh, predominantly rural sample, with about 30% bilingual participants, the results reflect findings in similar studies in Canada, the US, the UK, and Norway.6 13 18 34
Furthermore, we included problem behaviour outcomes reported by parents for the sibling closest in age to the index child. The positive effects of attending the parent programme also applied to this sibling, thereby suggesting additional benefits to the family and further possible reduced cost to society and services.
Meaning and implications of the study
These results are timely, particularly in the light of the recent appraisal from the National Institute for Health and Clinical Excellence that recommended the use of group parenting programmes for the treatment of conduct disorder in children35 and given the impact of conduct disorder on our communities. This study holds important lessons for the UK government because, unlike the disappointing results from the national evaluation of Sure Start, it shows that choosing an evidence based programme and delivering it with fidelity can achieve remarkable outcomes in high risk children whose parents generally fail to engage with services. These results have already had impact within Wales, where the Welsh Assembly government have funded training in the programme across Wales as part of its parenting action plan.36 In England in the new Pathfinder trial has funded the programme in six authorities as one of three evidence based programmes.37 The government must commission effective services for children at high risk of conduct disorders. They deserve evidence based programmes, as do the public, who pay a high price for services and for the other costs of antisocial behaviour.38
Questions and future research
It is important to establish whether the programme works equally well at all levels of severity of behaviour and depression and stress in parents. This will be explored through analysis of moderators of intervention effects in this sample. Adherence by leaders to the Incredible Years basic parenting programme protocol of delivery is currently being investigated in greater detail together with its relation to behavioural outcomes in children and parents to explore whether stricter adherence is associated with better outcomes.
What is already known on this topic
Children who show early persistent signs of antisocial behaviour are at greater risk of later juvenile delinquency and social exclusion, with high societal costs
What this study adds
The Incredible Years basic parenting programme effectively increases positive parenting practices and reduces antisocial behaviour in children at risk of developing conduct disorder
The programme works as a preventive intervention in highly disadvantaged community based settings when it is delivered by regular Sure Start staff with high levels of supervision and support to enable the programme to be delivered with fidelity
Footnotes
We thank all participating families; Sure Start staff, group leaders, and research and intervention staff for their help; Jess Eade for her help in the initial setting up of the project; Pam Dewis for arranging access to families; Dilys Willliams for her administrative support; Caroline White for assessing tapes of programme delivery; Carolyn Webster-Stratton and Ian Russell for their advice and encouragement; Carl Hughes for his support and attendance at steering group meetings, and all members of the research team who helped in the collection and scoring of data, especially Alan Ó'Céilleachair.
Contributors: JH had the idea for the study and obtained the grant, with assistance from FG. TB and JH managed the project and obtained further funding to extend the research. TB conducted randomisation and analyses. CW conducted additional analyses. TB and JH wrote the first draft of the paper. DD, RTE, and FG served on the steering group and helped write the manuscript. JH gave weekly supervision to group leaders. KJ and CE collected outcome measures and trained observers in the dyadic parent-child interaction coding system. JH is guarantor.
Funding: Research grant from the Health Foundation, grant No 1583/1566.
Competing interests: JH is paid by Incredible Years for running occasional training courses in the delivery of the parent programme and has served as an expert witness for the NICE appraisal on parenting and conduct disorder.
Ethical approval: North West Wales research ethics committee (ref No 02/12).