Papers

Evaluation of a structured test and a parent led method for screening for speech and language problems: prospective population based study

Gabrielle J Laing, consultant community paediatrician ^a, James Law, professor of language and communication science ^b, Abigail Levin, research speech and language therapist ^a, Stuart Logan, senior lecturer ^c. 

^a Child Health Department, St Leonard's Hospital, City and Hackney Primary Care Trust, London N1 5LZ, ^b City University, London EC1V OHB, ^c Institute of Health and Social Care Research, Peninsula Medical School, St Luke's Campus, Exeter EX1 2LU

Correspondence to: S Logan
S.Logan{at}pms.ac.uk

	Abstract

Top Abstract Introduction Methods Analysis Results Discussion References

Objective: To evaluate two methods for identifying speech and language problems in preschool children.
Design: Prospective population based study.
Setting: Inner London.
Participants and methods: 37 health visitors were randomly assigned to use a structured screening test (18) or a parent led method (19). Of 623 eligible children aged 30-36 months, the parents of 582 agreed to participate (353 using the structured test and 229 the parent led method).
Main outcome measures: Children were assessed by a speech and language therapist blinded to the test result, using the Reynell developmental language scales. Children were classified as having "severe language problems" if the Reynell score was below the third centile for receptive language and as "needing therapy" if the Reynell score was below the seventh centile for receptive or expressive language and clinical opinion.
Results: Reference assessments and usable scores were obtained for 458 (97%) of the 474 children screened. 98 (21%) children had severe language problems and 131 (29%) needed therapy. The sensitivity and specificity for the structured screening test were 66% (95% confidence interval 53% to 76%) and 89% (85% to 93%) respectively for severe language problems and 54% (43% to 65%) and 90% (85% to 93%) for those needing therapy. The sensitivity and specificity for referral by the parent led method were 56% (40% to 71%) and 85% (78% to 90%) for severe language problems and 58% (44% to 71%) and 90% (83% to 94%) for those needing speech and language therapy.
Conclusions: Both approaches failed to detect a substantial proportion of children with severe language problems and led to over-referral for diagnostic assessments. Screening is likely to be an ineffective approach to the management of speech and language problems in preschool children in this population.

What is already known on this topic Moderate to severe language difficulties in young children are predictive of long term problems affecting learning, school achievement, and behaviour Formal screening tests are widely used, but relying on parents' observations and health professionals' clinical judgment may be more effective in identifying children needing therapy What this study adds A commonly used screening test and an approach based on parents' observations and health visitors' judgment fail to identify a substantial proportion of children with serious language problems and lead to the over-referral of children without serious difficulties

	Introduction

Top Abstract Introduction Methods Analysis Results Discussion References

Moderate to severe language difficulties in young children, particularly those affecting language comprehension, are predictive of long term problems affecting learning, school achievement, and behaviour.^1-5 Young children presenting with language difficulties include those with specific impairment of language, autistic spectrum disorder, and general learning difficulties, as well as those whose difficulties are familial or associated with environmental factors. ^{6 7} For an individual child, the prognosis will depend on the cause of the problem and its responsiveness to intervention. ^{8 9}

The rate at which children acquire language varies greatly, and the definitions of "language delay" and "specific language impairment" lack clarity. The development of communication skills is closely linked with other areas of developmentsuch as cognitive, social, and behavioural skillswhich in turn influence language development. Estimates of the prevalence of speech and language impairment depend on the cut-off point chosen in a range across several component skills. Estimates of the prevalence of language difficulty in preschool children are between 1% and 19%.¹⁰

Most children with severe language difficulties are probably identified before they start school, but many may not be identified until they start formal education. Questions have been raised about the effectiveness of formal screening, although programmes exist in many developed countries and were common in the United Kingdom until recently.^11-14 Evidence suggests that parents are good observers of their child's development and that parents' reports may be as good as screening in detecting developmental difficulties. ^{15 16} As a result, many child health surveillance programmes in the United Kingdom now give advice about language development and rely on parents to identify children with problems in consultation with health visitors. ^{12 17}

The reliance on parents' observations and health visitors' clinical judgment is philosophically attractive, but the effectiveness of using this approach to identify children with serious speech and language difficulties is unclear. We compared the performance of a structured screening test with a method based on parental concerns and health professionals' clinical judgment against a reference assessment of speech and language in preschool children.

	Methods

Top Abstract Introduction Methods Analysis Results Discussion References

We invited all health visitors working in City and Hackney District, London, to participate in the study, and we assigned numbers to those who agreed. A statistician independent of the study used these numbers to randomly allocate health visitors in blocks of two (to ensure equal numbers) to either the structured screening or the parent led method. The health visitors received training in their allocated method. We asked the health visitors to invite parents of all children attending for their developmental check at age 21/2 years to participate in the study whether or not the child had a previously diagnosed developmental problem (confined to children aged 30-36 months seen as part of the routine child health promotion programme between February 1998 and June 1999). We obtained consent from the local research ethics committee.

We estimated the sensitivity of the formal screening test at 90%, that of the parent led method at 70%, and the prevalence of speech and language problems at 10%. From the number of eligible children and known uptake rates, we expected that health visitors would recruit 800 children in one year (400 in each group). This number would have implied a standard error of about 3.5% in the estimated sensitivity in the formal screening group and 5% in the parent led method. Because recruitment was lower than expected, we continued for a further five months, while funding was available.

The structured screening test contained 20 questions covering both expressive and receptive language skills and had been previously validated on the local population.¹⁸ A point was awarded for each question and a score of below 10 was regarded as a fail.

In the parent led method, we asked health visitors to discuss with parents whether they had any concerns about their children's language development. From this discussion and using clinical judgment based on history and observation, health visitors decided whether to refer the child, review the child at a later date, or classify the child as having no difficulties.

The assessment was carried out as part of the routine check, and parents were then asked to see one of two research speech and language therapists, either immediately or at an appointment within two weeks. The speech and language therapists carried out a diagnostic assessment based on the Reynell developmental language scales.¹⁹ This test is a comprehensive assessment of language skills and provides centiles standardised by age for expressive and receptive language. Each therapist saw children screened by both methods and were blinded to the outcome of the health visitor's assessment. Parents were asked not to tell the speech and language therapist the results of the screening assessment.

We classified children scoring less than the third centile on the Reynell receptive language scale as having severe language problems and children scoring below the seventh centile in either expressive or receptive language or who, in the opinion of the therapist, would benefit from referral as requiring speech and language therapy.

We obtained the number of children eligible for their developmental check at age 21/2 years and the proportion to be enrolled into each arm of the study from the Regional Child Health System.

	Analysis

Top Abstract Introduction Methods Analysis Results Discussion References

We entered data into EpiInfo.²⁰ We calculated sensitivity, specificity, and likelihood ratios for each method for the two diagnostic methods. For sensitivity and specificity we calculated 95% confidence intervals using the exact binomial method and for likelihood ratios we used a method recommended by Simel et al. ^{21 22} This study was originally designed as a cluster randomised controlled trial to compare the performance of the two screening methods. As neither performed adequately compared with the reference, it was decided that no statistical comparison should be made between the two methods.

	Results

Top Abstract Introduction Methods Analysis Results Discussion References

Of 47 health visitors, 37 agreed to participate. In all, 18 health visitors were allocated to the structured screening group and 19 to the parent led group. During the study period, health visitors did developmental checks on 1598 children aged 21/2 years, 40% of the total population. Of these, 308 (19%) were checked by a professional who was not taking part in the study. Similar proportions of children attending for checks with participating health visitors were enrolled into the study in each group. In the structured screening group, 376 children were invited to participate, of whom 353 (94%) agreed; 291 (77%) attended for screening and diagnostic assessment and usable results were obtained for 282 (75%). In the parent led group, 247 children were invited to participate, of whom 229 (93%) agreed, 183 (74%) attended for screening and diagnostic assessment, and usable results obtained for 176 (77%) (figure).

View larger version (34K): [in this window] [in a new window]   Flow of children through the study

The number of children recruited varied among health visitors (range 1-37; mean 16.2). The characteristics of the health visitors in each group and the children who agreed to participate and of those included in the analyses are given in table 1.

Of the 458 children who completed the reference assessment, 98 (21%) scored below the third centile and 131 (29%) were judged as "needing therapy." Performance in screening tests and reference assessments with likelihood ratios and 95% confidence intervals is given in tables 2 to 5.

The sensitivity and specificity for the structured screening test were 66% (95% confidence interval 53% to 76%) and 89% (85% to 93%) for children with severe language problems and 54% (43% to 65%) and 90% (85% to 93%) for children needing therapy. When children who scored more than 10/20 whose parents or health visitors were concerned were included in the fail category, the sensitivity increased slightly but the specificity decreased.

The sensitivity and specificity for referral by the parent led method were 56% (40% to 71%) and 85% (78% to 90%) for severe language problems and 58% (44% to 71%) and 90% (83% to 94%) for children needing therapy. We also combined the referral and review categories to reflect those children causing some concern. The resulting sensitivity and specificity were 77% (60% to 88%) and 68% (60% to 75%). for children with severe language problems and 79% (66% to 88%) and 74% (66% to 81%) for children needing therapy.

Analysis of the data including only the 387 (85%) children whose first language was English made virtually no difference to the results. Of these children, 72 (19%) scored below the third centile on the Reynell receptive language scale.

	Discussion

Top Abstract Introduction Methods Analysis Results Discussion References

In the United Kingdom, it has been argued that formal screening for speech and language difficulties in preschool children should be replaced by an approach in which health professionals respond to parental concerns.¹² The effectiveness of this approach has not been assessed previously. We asked health visitors using the parent led method to elicit parental concerns about their children's speech and language and to interpret these concerns in the light of their clinical knowledge. Health visitors allocated to this method attended refresher workshops to ensure that they felt comfortable with what would be considered normal language development in children of this age group. The results are disappointing; at least within this deprived inner city population, the approach would fail to identify a substantial proportion of children with serious language problems, while also leading to over-referral of children without difficulties.

Our results do not, however, imply that formal approaches to screening are best. The structured screening test performed as poorly as the parent led method. A previous assessment of this test in the same area had reported a higher sensitivity, but in that study only some children whose screen showed no language difficulties could be assessed.¹⁸

The sensitivity and specificity needed by a potential screening test depends on context and the balance between the consequences of correct and incorrect classification and the underlying prevalence of the condition. The performance of the approaches we tested is clearly less than satisfactory. We therefore decided that statistical comparison of performance between the tests would be inappropriate.

Applicability of results
The extent to which these results are applicable to other populations is unclear. The application of estimates of test performance from one population to another depends on the relation between the range of abnormality in the two groups. This is likely to differ between more and less deprived areas. Hackney is a deprived area where many people speak languages other than English. Restriction of the analyses to those children whose first language was English, however, made no difference to the results. The results apply most directly to children in other deprived areas.

A further difficulty is that the low uptake of the developmental check at age 21/2 years and attrition mean that we cannot assume our sample is representative of children in the district. Children whose parents were anxious about their language development may have been disproportionately likely to participate, and, similarly, health visitors may have preferentially enrolled children whom they were concerned about, which would accord with the high prevalence of serious language problems. This would have biased the study towards overestimating the performance of the screening tests.

Treating or preventing speech and language difficulties
The rationale for trying to identify children with speech and language difficulties is the belief that early intervention by speech and language therapists can improve outcome. A systematic review published in 1998 found that although there are many studies showing positive effects of interventions for speech and language therapy, the evidence was relatively limited.¹⁰ A recent randomised controlled trial suggested that therapy had little effect, although the trial was criticised because of the small amount of therapy provided to the intervention group. ^{23 24} This level of therapy is, however, likely to reflect the amount of therapy available in many NHS facilities in the United Kingdom. The effectiveness of therapy is likely to depend on the specific problem and on the presence of comorbidities and other contributory factors.

Early speech and language difficulties are strongly associated with later adverse outcomes.^1-5 Our results suggest that using formal screening or a parent led approach to identify children with speech and language difficulties is unlikely to be effective. Little consensus about alternative approaches, however, has been reached. The factors determining speech and language development are not understood, and children with problems include those with multiple developmental problems; those with specific possibly genetically determined, communication disorders; those where the problems are associated with poor environmental conditions or neglect; and many for whom the cause is unclear. ^{6 7} Retaining developmental screening may be useful in areas of social disadvantage because parents may be less likely to successfully obtain support services for children with impairments. A more effective instrument, however, than the screening test evaluated here would be needed.²⁵

Population based approaches to the promotion of language development are intuitively attractive, particularly in areas with a high prevalence of children with serious problems. In line with this approach, initiatives such as Sure Start in the United Kingdom have resulted in a wide range of innovative community based approaches designed to target speech and language development.²⁶ Unfortunately the evidence base for these proposed interventions is relatively weak. Better evaluation of such interventions should be a matter of priority.

	Acknowledgments

We thank the health visitors, parents, and children who participated; Julian Higgins, who did the randomisation; and Bernard Camilleri, who performed diagnostic assessments on some of the children.

Contributors: GJL and JL conceived the study and SL advised on the methods. AL assessed the subjects and collected and entered the data, supervised by GJL and JL. SL and GL did the analysis. SL wrote the first draft of the paper and all authors revised it. GL and SL are guarantors.

	Footnotes

Funding: The Inner City Research and Development Group of the former North Thames regional office of the NHS Executive.

Competing interests: None declared.

	References

Top Abstract Introduction Methods Analysis Results Discussion References

1.	Stothard SE, Snowling MJ, Bishop DVM, Chipchase BB, Kaplan CA. Language impaired pre-schoolers: a follow-up into adolescence. J Speech Lang Hear Res 1998; 41: 407-418[Abstract/Free Full Text].
2.	Johnson CJ, Beitchman JH, Young A, Escobar M, Atkinson L, Wilson B, et al. Fourteen-year follow-up of children with and without speech/language impairments: speech/language stability and outcomes. J Speech Lang Hear Res 1999; 42: 744-761[Abstract/Free Full Text].
3.	Conti-Ramsden G, Botting N, Simklin Z, Knox E. Follow-up of children attending infant language units: outcomes at 11 years of age. Int J Lang Commun Disord 2001; 36: 207-221[Web of Science][Medline].
4.	Aram DM, Nation JE. Preschool language disorders and subsequent academic difficulties. J Commun Disord 1980; 13: 150-159.
5.	Silva PA, Williams SM, Mcgee R. A longitudinal study of children with developmental language delay at age three: later intelligence, reading and behaviour problems. Dev Med Child Neurol 1987; 29: 630-640[Web of Science][Medline].
6.	Cohen NJ, Lipsett L. Recognised and unrecognised language impairment in psychologically disturbed children: child symptomology, maternal depression and family dysfunction: preliminary report. Can J Behav Sci 1991; 23: 376-389.
7.	Law J, Conway J. Child abuse and neglect: the effect on communication development, a review of the literature. London: Afasic, 1992.
8.	Paul R, Cohen D, Caparulo BK. A longitudinal study of patients with severe developmental disorders of language learning. J Am Acad Child Psychiatry 1983; 22: 525-534[CrossRef][Web of Science][Medline].
9.	McRae KM, Vickar E. Simple developmental speech delay: a follow-up study. Dev Med Child Neurol 1991; 33: 868-874[Web of Science][Medline].
10.	Law J, Boyle J, Harris F, Harkness A, Nye C. Screening for speech and language delay: a systematic review of the literature. Health Technol Assess 1998; 2: 1-184.
11.	Hall DMB. Health for all children. 2nd ed. Oxford: Oxford University Press, 1991.
12.	Hall DMB. Health for all children. 3rd ed. Oxford: Oxford University Press, 1996.
13.	American Academy of Pediatrics. Screening infants and young children for developmental disabilities. Pediatrics 1994; 93: 863-865[Abstract/Free Full Text].
14.	De Winter M, Balledux M, de Mare J, Burgmeijer R. Screening in child health care. Report of the Dutch working party on child health care. Oxford: Radcliffe Medical Press, 1995.
15.	Glascoe FP. Do parents discuss concerns about children's development with health care providers? Ambulatory Child Health 1997; 2: 349-356.
16.	Glascoe FP. Evidence-based approach to developmental and behavioural surveillance using parents' concerns. Child Care Health Dev 2000; 26: 137-149[CrossRef][Web of Science][Medline].
17.	Hall D, Elliman D. Health for all children. 4th ed. Oxford: Oxford University Press, 2002. www.health-for-all-children.co.uk/hfac4_draft.htm (accessed 13 Sep 2002).
18.	Law J. Early language screening in City and Hackney: the concurrent validity of a measure designed for use with 2.6 year olds. Child Care Health Dev 1994; 20: 295-308[CrossRef][Web of Science][Medline].
19.	Edwards S, Fletcher P, Garman M, Hughes A, Letts C, Sinka I. The Reynell developmental language scales III. Windsor: NFER-Nelson, 1997. (University of Reading edition.)
20.	Dean JA, Coulombier D, Smith DC, Brendel KA, Arner TG, Dean AG. EpiInfo v 6.03: a word processing, database and statistics programme for epidemiology on microcomputers. Atlanta: Centers for Disease Control and Prevention, 1995.
21.	Bryant T. Confidence interval analysis. Southampton: University of Southampton, 2000.
22.	Simel DL, Samsa GP, Matchar DB. Likelihood ratios with confidence: sample size estimation for diagnostic test studies. J Clin Epidemiol 1991; 44: 323-338.
23.	Glogowska M, Roulstone S, Enderby P, Peters TJ. Randomised controlled trial of community based speech and language therapy in preschool children. BMJ 2000; 321: 923[Abstract/Free Full Text].
24.	Law J, Conti-Ramsden G. Treating children with speech and language impairments. BMJ 2000; 321: 908-909[Free Full Text].
25.	Alberman E, Goldstein H. The "at risk" register: a statistical evaluation. Br J Prev Soc Med 1970; 24: 129-135[Web of Science][Medline].
26.	Law J, Harris F. Promoting speech and language development: guidance for Sure Start programmes 2001. Nottingham: Department for Education and Skills. www.surestart.gov.uk/infoDocs/18114spe.&_lang.pdf (accessed 28 Oct 2002).

(Accepted 12 June 2002)