Rapid Responses to:
|
|
Rapid Responses: Rapid Responses published:
-
![[Read Rapid Response]](/icons/misc/sectionDOWN.gif)
Therapy for young children who stutter: Efficacy is in the eye of the beholder
- Joseph Kalinowski, Chayadevie Nanjundeswaran, Vijaya K. Guntupalli, Jianliang Zhang. (24 August 2005)
-
The statistical fluctuation in the natural recovery rate between control and treatment group dilutes their results
- Tom Weidig (6 September 2005)
-
Re: The statistical fluctuation in the natural recovery rate between control and treatment group dilutes their results
- Tom Weidig (7 September 2005)
-
Stuttering children in the developing world
- Bolajoko. O Olusanya, Abayomi Somefun, Afolabi Lesi, Mohammed Ahmed-Danfulani (29 September 2005)
-
Reply to the rapid responses
- Mark A Jones, Mark Onslow, Ann Packman, Val Gebski (7 December 2005)
-
Re: Reply to the rapid responses
- Tom Weidig (9 January 2006)
|
|
|||
|
Joseph Kalinowski, Associate Professor Department of Communication Sciences and Disorders, East Carolina University, Greenville, NC, 27858, Chayadevie Nanjundeswaran, Vijaya K. Guntupalli, Jianliang Zhang.
Send response to journal:
|
Treating developmental stuttering in its incipient stages makes intuitive sense, especially considering the propensity for the disorder to cement itself within the central nervous system, making it highly resistant to treatment at later stages (i.e., in adolescents and adults). However, regardless of therapeutic intervention, Mother Nature appears to have her own recovery agenda, allowing 74% (by these authors admission) of children incipiently stuttering to complete recovery. For much of the first 50 years of speech pathology's professional development, the world followed Wendell Johnson's protocol, which simply advocated removing unwanted attention from stuttering behaviors1. Because of what we now understand about these natural recovery rates, we can clearly see how Johnson created an illusion of therapeutic success when 74% of his young patients were pre-ordained to recover. However, for those children that continued to stutter, implicit blame and guilt were cast a shadow upon the parents for labeling and reacting to their child behavior and drawing negative attention to their speech. Our understanding of stuttering has come a long way since the Johnsonian days and though the approach to treating children is now diametrically opposite (i.e., we now advocate direct intervention), it is still implemented with the assumption that taking advantage of early neural plasticity is the best recourse for halting the stuttering pathology. The question is, are the results achieved nowadays any better? The Lidcombe program provides a parent- based behavioral, response contingent approach to stuttering. In this therapy, parents attempt to reduce stuttering by presenting verbal positive affirmations for fluent speech behaviors and for reducing " bumpy speech”. The ability of these children to reduce their stuttering in front of clinicians is not new. Employing parents as clinicians and collecting data in multiple settings is a new approach in stuttering therapy and seems the signature of the Lidcombe program, yet still does not eliminate a potential Hawthorne effect during data and does not mean that any observed reduction or elimination of stuttering occurs across speaking environments and situations. Short-term reduction in stuttering frequency is easy to demonstrate with any kind of therapy2, whereas recovery appears to be difficult to achieve in persistent stutterers at any stage of development. Lidcombe behavioural therapy has been practiced for over a decade. Its popularity has spread across the globe. In fact, specialized manuals and training centers exist to help clinicians become proficient in implementing Lidcombe procedures. Now, in this day of "evidence-based" therapy, the first randomized trials by these authors should be applauded. They took substantial efforts to ensure that groups were adequately randomized and balanced, the treatment was administered according to protocol and the 'counting of stuttering' was conducted by observers who were blind to treatment allocation. However, knowing the natural recovery rates, we are unsure whether the advantages of employing an untreated control group outweigh the fact that a potential remedy was withheld from children during a critical time in the pathological development. Regardless, the authors suggest that the results support the 'efficacy' of the Lidcombe program. An important question in this study is how efficacy is defined. In this case, efficacy may merely be an acceleration of the natural healing process in those children prone to recover or a simple temporary reduction in stuttering that that can occur with any therapeutic program for incipient stuttering. For any program to be considered truly effective in the treatment of incipient stuttering it should be able to 'beat' Mother Nature and show complete recovery (i.e. forever free from all stuttering symptoms in most situations and the perception of being a person who stutters) rates in excess of 74%.3 However, since its inception, one of the main points of contention with this program, as well as others that claim efficacy in remedying childhood stuttering, has been the separation of therapeutic effects from those of natural 'spontaneous recovery'. In this study, 134 children were referred for research program but the results are only reported 47 children, although the other children may have not meet some research-imposed criteria it would have been very informative to see the natural progression of the disorder. As a group these 47 children demonstrate a mild to moderate pathology, the use of mild to moderate children is typical of this research group when they show results of less than 1 or 2% stuttering.4 Results show that after nine months the mean stuttering frequency in the treatment group was still 1.5%, with only 52% of children showing a stuttering frequency of less than 1%. Additionally, in an earlier investigation into the impact of Lidcombe program on early stuttering, the mean stuttering frequency following a 12 week treatment was 3.5 % syllables stuttered,5 indicating that traces of stuttering were most likely still evident in many and the pathology may be subject to 'balloon' again at a later date. These data show no evidence of complete recovery exceeding 74%. Therefore, this program cannot claim to heal any child that would not be otherwise remitted if simply left under the care of Mother Nature. Similarly, Franken et al. (2005)6 compared the Lidcombe approach to a Demands and Capacities approach and both treatments yielded similar positive outcomes. We suggest that almost all therapies can be helpful in alleviating some of the symptoms of childhood stuttering.7 The simple fact that children are learning some means of temporarily 'inhibiting' stuttering in nurturing environments (e.g., with parent therapists) can help ease the burden of stuttering and train coping mechanisms. For those who are likely to recover anyway, almost any type of therapeutic intervention may accelerate the process. Thus, it is doubtful that the Lidcombe or any other behaviorally based program is powerful enough to rewire neural networks and alter the epidemiology of stuttering. Treating children who stutter may be analogous to attempts to increase adult height by administering human growth hormone to children who are not growth hormone deficient. Though short periods of accelerated growth can be observed, the final height attained is often similar to what matched children achieve without the hormone treatment.8 In other words, regardless of therapy, Mother Nature still seems to have the final word on who recovers and her agenda is not fooled by temporary periods of accelerated gains. We are advocates for providing children who stutter with all the help in the world. Accelerating recovery and providing stuttering children with means of coping with the pathology remains a most noble cause. However, when employing the Lidcombe program and stuttering persists, we must also caution against falling into the Johnsonian snake pit of blaming parents for an involuntary childhood pathology whose full recovery is most likely beyond their control. References: 1. Johnson W. A study of the onset and development of stuttering. In Johnson W, Leutenegger RR, eds, Stutttering in Children and Adults. Minneapolis: University of Minnesota Press, 1955. 2. Bloodtein O. A Handbook on Stuttering. San Diego, CA: Singular Publishing Group, 1995. 3. Kalinowski J, Saltuklaroglu T, Dayalu V, & Guntupalli VK. Is it possible for speech therapy to improve upon natural recovery rates in children who stutter?. Int J Lang Comm Disord 2005;40:349-58. 4. Onslow M, Andrews C, Lincoln M. A control/experimental trial of an operant treatment for early stuttering. J Speech Hear Res 1994;37:1244-59. 5. Harris V, Onslow M, Packman A, Harrison E, Menzies R. An experimental investigation of the impact of the Lidcombe Program on early stuttering. J Fluency Disord 2002;27:203-14. 6. Franken MC, Schalk CJ, Boelens H. Experimental treatment of early stuttering: A preliminary study. J Fluency Disord, doi:10.1016/ j.jfludis.2005.05.002. 7. Kalinowski J, Dayalu VN, Saltuklaroglu, T. Cautionary notes on interpreting the efficacy of treatment programs for children who stutter. Int J Lang Comm Disord 2002;37:359-61. 8. Kawai M, Momoi T, Yorifuji T, Yamanaka C, Sasaki H, Furusho K. Unfavorable effects of growth hormone therapy on the final height of boys with short stature not caused by growth hormone deficiency. J Pediatr 1997;130:205-09. Joseph Kalinowski, PhD kalinowskij@mail.ecu.edu http://www.ecu.edu/cs-dhs/csd/stutt.cfm Competing interests: None declared |
|||
|
|
|||
|
Tom Weidig, Consultant QuantExperts 9, rue Donven 4084-Esch-Alzette Luxembourg
Send response to journal:
|
I believe that the authors of "Randomised controlled trial of the Lidcombe programe of early stuttering intervention" need to review their statistical analysis. My main argument is that the authors have failed to include the effect of the statistical fluctuations of the natural recovery rate within the two randomized samples. The inclusion could considerably reduce the statistical significance and effect size of their results. Especially in combination with other important but not here discussed methodological issues like possibility of relapse and different subtypes (sex and family history). However, I want to point out that my comments on the statistical study of the trial do not necessarily imply that the Lidcombe treatment by itself is not effective. I will first show why the natural recovery rate is important. Then, I will discuss the impact of the effect. Using a Monte Carlo simulation I will show that I can reproduce in 10% of the scenarios the authors' "minimum worthwhile difference between the two arms set at 1.0% syllables stuttered" by including the statistical fluctuation of the natural recovery rate without any treatment effect! Finally, I wonder whether this effect does not question the usefulness of a randomized controlled trial setup in this special case. Why does the statistical fluctuation of the natural recovery rate needs to be included in the statistical analysis? For simplicity and ignoring definition issues, I assume a natural recovery rate of 70%. 70% of all children with dysfluencies become fluent speakers. For a large sample of 1000 children with dysfluencies, about 200 would not recover. However, for small samples, the rates could fluctuate significantly. It is perfectly possible to by chance pick a sample with 20 children who will all naturally recover or 20 kids that will not recover. So drawing several samples, I will end up with a different rate for each sample, e.g. 61%, 92%, 100%, 78%, and so on. But only the average of the rates tends to 70%. And this is the problem. The authors have split the children into two groups, and performed a t-test to compare both samples. Implicitly, they therefore assume that both groups have the same natural recovery rate. However, this assumption is violated as the two groups might considerably differ, as mentioned above, the rate could be 61%, 92%, 100%, 78%, and so on. For example, they could by chance have started with the treatment group having an 92% recovery rate and the control group having a rate of 61%. The bigger the sample size, the closer both groups are around 70% recovery rate. To summarise, I argue that the authors need to discuss this effect in their statistical analysis, and at least say why the effect is irrelevant to their results. The existence of this effect does not necessarily mean that by including the effect their results are not significant. However, in this paragraph I argue that the inclusion of the effect most likely dilutes the significance of their results. First, the inclusion of statistical fluctuation can only decrease and not increase the significance. The question is by how much. Second, I can come up with realistic scenarios that have a significant impact. For example, let us assume that by chance the control group has an instrinsic rate of 50% and the treatment group of 70%. Further, I assume that all children stuttered at 5% before therapy, and the recovered ones have 0%. I also assume no treatment effect. So I end up with a dysfluency rate for the control group of ( 50% * 5% + 50% * 0 ) = 2.5% and for the treatment group of ( 30% * 5% + 70% * 0 ) = 1.5%. This 1% difference is not due to treatment effect but statistical fluctuation. Thus, I can construct scenarios that reproduce a clear difference without treatment effect. Third, the scenario is not only possible but is a probable scenario. I did a Monte Carlo simulation with the parameters of the trial. I randomly created thousands of pairs of control (20 children) and treatment (27 children) samples, and computed the sample difference between them. I used the same method to get the syllables stuttered as in the previous and second argument. I therefore have thousands of possible sample differences, and was able to plot a histogram with difference against probability. I find a 10% probability that a "minimum worthwhile difference between the two arms set at 1.0% syllables stuttered" occurs due to statistical fluctuation and not due to treatment effect. This strongly indicates that the true statistical significance and effect size of the treatment could be much lower with inclusion of the effect. To summarise, I believe that the authors need to include the natural recovery effect in their analysis, because I have shown that the effect exists and that it could impact the results. Finally, I am wondering whether it is even useful to use the randomized trial setup due to this effect and due to the fact that there is no need to filter out a placebo effect? Another setup could have been to only have a treatment group (which is then twice as large) and test whether this group has a higher rate of recovery (natural and treatment-wise) than the natural recovery rate while including the statistical fluctuation of the rate. But a more detailed analysis is needed to decide between both setups, and not the subject of this letter. EMAIL: tom.weidig AT physics.org BLOG: http://thestutteringbrain.blogspot.com/ Competing interests: None declared |
|||
|
|
|||
|
Tom Weidig, consultant Luxembourg
Send response to journal:
|
I just noticed a spelling mistake in my rapid response. The number 200 is wrong "For a large sample of 1000 children with dysfluencies, about 200 would not recover." Needs to be changed to "For a large sample of 1000 children with dysfluencies, about 300 would not recover." Competing interests: None declared |
|||
|
|
|||
|
Bolajoko. O Olusanya, Developmental Paediatrician Institute of Child Health and Primary Care, College of Medicine, University of Lagos, Nigeria, Abayomi Somefun, Afolabi Lesi, Mohammed Ahmed-Danfulani
Send response to journal:
|
The findings from this study conducted in New Zealand are also significant for the management of stuttering in the developing world. Stuttering in this region is generally regarded as self-limiting and not requiring professional intervention, a view possibly reinforced by the reported 74% natural recovery rate [1]. Consequently, most parents of children with this disorder rarely seek help. For instance from our recent work among children with communication disorders aged 6 months to 15 years that were referred to our audiology clinic, only 2.2% presented with stuttering compared to hearing impairment (65.2%), specific language impairment (23.9%) or central speech disorders (6.5%) [2]. We believe that many more did not seek medical intervention because it was also not considered as an unusual developmental phase in early childhood. Stutterers often devised individual coping strategies for interpersonal communication as they grow older. As a result stuttering has become least associated with social stigma (if any) compared to other communication disorders. Given the high proportion of those for whom intervention may not be necessary, it will be of interest to ascertain conditions that could facilitate or hinder natural recovery in any environment. However, identifying those who may follow this course is would be difficult in practice. Introducing an intervention programme in developing countries would therefore be worthwhile except that some practical challenges are foreseeable in the short-run. Firstly, a public health programme in preschool years is often difficult to implement without considerable investment in special awareness campaigns. A school entry programme may prove more cost-effective for ease of implementation provided this threshold is not too late for optimal intervention. Secondly, parental literacy is a key factor in this intervention plan and this may restrict its wide application. Additionally, the dearth of speech pathologists required to oversee the programme is a major constraint that may be difficult to resolve rapidly without some non-specialist training at community levels. However, these challenges are not insurmountable. The knowledge that stuttering can be effectively treated in early childhood and that early intervention is more efficacious than natural recovery should ultimately serve as an impetus for appropriate service development in this region. References 1. Yairi E, Ambrose N. Early childhood stuttering I: persistency and recovery rates. J Speech Lang Hear Res 1999;42:1097-12. 2. Somefun OA, Lesi FEA, Danfulani MA, Olusanya BO. Communication disorders in Nigerian children. Int J Pediatr Otorhinolaryngol (in press). Competing interests: None declared |
|||
|
|
|||
|
Mark A Jones, Biostatistician University of Queensland, Mark Onslow, Ann Packman, Val Gebski
Send response to journal:
|
The issue of spontaneous recovery has been a recurring theme in the rapid responses to our article. There have also been a number of other issues raised. Therefore we feel it is appropriate to respond to provide additional information that will hopefully clarify the issues raised in the rapid responses. The rate of natural recovery from stuttering in young children in the general population has been estimated at 74%(1), but in clinical populations with this condition this estimate is expected to be much lower. Children would normally present to the clinic only after stuttering has been apparent for some time and it is unlikely that those experiencing transient stuttering would seek help at a speech clinic. Evidence from a previous study(2) suggests that approximately 40% of children identified as stuttering had the condition for less than 6 months. Consequently, natural recovery rates for children who present to a speech clinic are likely to be significantly lower than the broader population recovery rates, although precise estimates of this rate are still unknown. In order to enrol in our study children needed to have been stuttering for at least 6 months. This RCT is the most recent of a series of peer reviewed scientific studies of outcomes of the Lidcombe Program (LP). Previous studies include earlier phase trials as well as retrospective file audits both in Australia and the United Kingdom(3). The combined data from these retrospective studies show that 93% of children were able to attain levels of stuttering of less than 1% of syllables stuttered (<1%SS). The “treatment failures” were due to the abandonment /non-compliance of the treatment due to reasons such as family relocation, family break-up, identification of other speech or language problems that needed to be addressed more urgently than the stuttering and behaviour issues with the children. In these studies, on two occasions treatment was abandoned due to problems with treatment delivery. A study examining the long-term effects of the LP showed treatment gains were maintained for between 2-7 years(4). Additionally, treatment success was not compromised, at least within the preschool years, when treatment was delayed for a year or more after onset of stuttering. In our study, children were followed for 9 months on the allocated treatment but we felt that it would be unethical to delay treatment in the control group after this time. Consequently, not all the children allocated to the LP had completed their treatment within the follow up period. The RCT design ensures as far as possible that the groups of subjects are alike in every respect and differences observed are those due to either the intervention or chance. In our randomisation procedure we stratified by age, gender, severity of stuttering, family history of recovery from stuttering, and treatment site. These factors were thought to possibly influence the outcome and we sought to maintain reasonable treatment balance within these subgroups and hence the rate of natural recovery would have been similar in the two groups. As to the comment that only children with mild to moderate levels of stuttering were recruited to the study, our only exclusion criterion based on severity was mild stuttering (<2%SS) and children with stuttering levels as high as 20%SS or more were recruited. Hence the study population comprised moderate to severe early stutterers. Another comment referred to the sampling of children’s speech. We point out that speech samples included the child speaking to a family member at home, to a non-family member at home and to a non-family member away from home, reflecting a variety of speaking situations. More widely, children do not move to Stage 2 of the program until the "daily global severity ratings made by parents" drop to a very low level. References 1. Yairi, E., & Ambrose, N. (1999). Early childhood stuttering I: Persistency and recovery rates. Journal of Speech, Language, & Hearing Research, 42, 1097-1112. 2. Andrews, G., & Harris, M. (1964). The Syndrome of Stuttering. Clinics in Developmental Medicine, No. 17. London: Spastics Society Medical Education and Information Unit in association with Wm. Heinemann Medical Books. 3. Onslow, M., Packman, A., & Harrison, E. (Eds.) (2003). The Lidcombe Program of early stuttering intervention: A clinician’s guide. Austin, TX: Pro-Ed. 4. Lincoln, M., & Onslow, M. (1997). Long-term outcome of an early intervention for stuttering. American Journal of Speech-Language Pathology, 6, 51-58. Competing interests: None declared |
|||
|
|
|||
|
Tom Weidig, Scientist-turned-Consultant thestutteringbrain.blogspot.com
Send response to journal:
|
Jones et al. have not directly responded to the arguments put forward in my rapid response. For example, they say that "... spontaneous recovery has been a recurring theme in the rapid responses to our article... The rate of natural recovery from stuttering in young children in the general population has been estimated at 74%(1), but in clinical populations with this condition this estimate is expected to be much lower...Evidence from a previous study(2) suggests that approximately 40% of children identified as stuttering had the condition for less than 6 months." However, they do not directly relate this or other statements to my arguments to disprove my case, so it is difficult for me to respond in a constructive way. My guess is that they imply that any effect due to the spontaneous recovery rate on the statistics is very small because the rate in the clinical setting is only about 40% and not about 75%. But this argument is not correct. The great irony is that the closer the recovery rate is to 50%, the greater the impact on the statistics!! So by claiming a 40% rate instead of a 74%, they have actually strengthened my argument that they need to revise the statistical significance and effect size of their study to take into account the natural recovery rate! The details of my arguments are described in my first rapid response, and below I will explain why a lower natural recovery closer to 50% makes the need for a correction greater. In a RCT, the subjects are split into two groups in such a way that both groups can be considered to start from the same baseline. This is standard procedure. However, this condition is violated in their trial, because they cannot control for the fluctuation in the inherent natural recovery rate of children who stutter. Two groups are in general NOT equal, because they do not have the same inherent natural recovery rate. The rate fluctuates in each group. For example, by chance one group might have a recovery rate of 80% and the other group of 40%. So both groups in general do not start out from the same baseline. And they need to correct their statistical measures like effect size or statistical significance to include this effect. Failure to include the effect leads to exagerated effect size and significance, e.g. an observed treatment effect even if there is not treatment effect! (see my first post) The interesting observation is that this effect is greatest for a recovery rate of 50%, because at 50% the average difference of the two groups in the natural recovery rate is greatest. The strength of the effect declines and goes to zero for a rate of zero or one, because for these limiting cases the rate is the same in both groups. So ironically claiming that the recovery rate is closer to 40% than to 75% increases the effect, as 40% is closer to 50% than 75%! To conclude, I re-emphasise that the statistical measures are not correct, and that they need to revise their statistics. This correction is not straightforward, but can be done using Monte Carlo simulation techniques. I am happy to cooperate. Again, I point out that I do not question the study as a whole, and I admire the efforts they put in. But they need to realise that they cannot just apply the standard RCT framework. They need to correct for the natural recovery rate, which is not present in a standard RCT setting. Competing interests: None declared |
|||