Why didn’t patients use it? Engagement is the real story in Gilbody et al. (2015), not effectiveness.
CCBYOpen access
Rapid response to:
Research
Computerised cognitive behaviour therapy (cCBT) as treatment for depression in primary care (REEACT trial): large scale pragmatic randomised controlled trial
Why didn’t patients use it? Engagement is the real story in Gilbody et al. (2015), not effectiveness.
We congratulate the researchers on their impressive study. There is a lesson to learn from Gilbody et al., except it is not about effectiveness, it is about engagement and adherence.
Gilbody et al. (2015) reported “no additional improvement in depression compared with usual GP care at four months” for participants who were offered computer-delivered CBT (cCBT).(1) Offered is the operative word because Gilbody et al. found that participants used only 1-2 sessions of the programs. This level of adherence makes it impossible to conclude anything about effectiveness. This shows poor adherence. This is akin to saying that a pharmaceutical drug that reduces depression doesn’t work when the real issue is that people don’t want to take it because of side effects or poor explanation from their doctor.
Analyses addressing predictors of adherence are missing from this paper. The adjunct cCBT programmes may be ineffective or participants may have received sufficient usual care to reduce cCBT’s relevance. To properly test this requires a dismantling trial in which all participants received the same dose of treatment. Otherwise, this turns into testing a phantom.
The overlap between experimental and control groups obfuscates outcome.
This was a pragmatic trial, meaning that there were no restrictions on access to other treatment. This is an important test to examine real-world usage patterns of various treatment modalities. It is not appropriate for determining the efficacy of a single intervention. The authors reported that 19% of the GP care group had also actively used cCBT. When a substantial proportion of participants in the control group receive a similar intervention to those in the active experimental groups, it is difficult to make a conclusion about the superiority of one versus the others. Moreover, between 77% (MoodGYM) and 84% (GP treatment only group) received also pharmacological treatment for depression. Meta-analytic evidence indicates that the additional effect of (face-to-face) psychotherapy to pharmacological treatment only is small.(2) If a substantial proportion of all groups in addition to the pharmacological treatment also receive psychological treatment for depression in secondary care (17% in the MoodGYM group, to 28% in the GP group) it is no wonder that the authors can´t find large differences between the groups.
The general design of the trial is properly chosen under the hypothesis that cCBT was more effective than GP Care. Nevertheless, the conclusion that cCBT is not superior to GP care cannot be drawn based on this trial design. The lack of significance of a statistical test for superiority does not prove equivalence (or non-inferiority). The fact that both arms of adjunct treatment do not show superiority over GP care supports the impression of a lack of assay sensitivity, meaning, the ability to distinguish effective treatments from less effective or ineffective treatments.(3) The International Conference on Harmonization (ICH) lists a number of factors that can reduce assay sensitivity.(4) These include poor compliance with the study medication/intervention as observed in the current trial.
Therefore, the question of whether cCBT is really not superior to GP care in a pragmatic setting remains unanswered. This question could only be answered in a prospective confirmatory non-inferiority or equivalence trial. While hypothetically, it would be possible to change the presented analysis from superiority to a non-inferiority analysis there are several requirements for this decision (e.g., in the trial protocol predefined or otherwise justified non-inferiority margins which are not met in the current trial.(5)
For the presented non-inferiority comparison between Beating the Blues and MoodGYM the analysis should be repeated in the per-protocol collective to increase confidence in the results.(6)
The investigated interventions do not represent the standard for cCBT programmes.
There are numerous meta-analyses showing that guided self-help cCBT programmes have superior adherence and outcomes to self-guided programmes. MoodGYM is used as an “open access” and unguided intervention, which is not as effective as guided self-help. The adherence and engagement found in the Gilbody et al. study is worse than what has been reported in meta-analyses for other self-guided depression programmes which show completion rates of approximately 80%.(7)
The nature of the “support” provided in REEACT would not likely meet current standards for guidance in cCBT. Evidence suggests that specific guidance or “support” behaviors predict better outcomes: individualized feedback with task reinforcement, task prompting, and empathetic communication.(8) REEACT described only task prompting: reminders to complete sessions. While others have shown that online CBT with guidance from a technician can be equally effective to the same intervention provided by clinicians (e.g., Titov et al.), the quality of guidance provided by the technicians in REEACT is a key open question.(9)
How these programmes are implemented and introduced to participants is critical. The way a treatment is presented has substantial effects on adherence and, relatedly, treatment outcome expectancy.(10) Context matters probably most in such treatments, as does usability. Improved design elements such as mobile-first technology and personalisation can likely further boost engagement.
Zooming out: Using stepped care to reduce mental health care disparities
The objective of cCBT developers and researchers is not to eliminate in person care. It is to create more options within the mental health care ecosystem to improve access to effective care. The fact remains that while usual care is effective– the wait times and cost of accessing specialist or GP services are often prohibitive. Digital mental health technology offers a promising path to bridge this gap, assisting individuals whose needs can be effectively met with a lower intensity approach. Moreover, only a limited percentage of those in need of mental health treatment actually utilise it. Research indicates that this is not only a problem of evidence-based treatment availability, but also of poor utilization due to other barriers, such as stigma.(11) Digital programmes offer a low-threshold and highly acceptable (to many) way of investing in one’s own emotional wellbeing which for many, might be the only care they can receive.
Megan Jones, Lantern, San Francisco, United States and Stanford University School of Medicine, Stanford, United States
David Daniel Ebert, Friedrich-Alexander University Erlangen-Nuremberg, Nuremberg, Germany
Corinna Jacobi, Technische Universität Dresden, Dresden, Germany
Ina Beintner, Technische Universität Dresden, Dresden, Germany
Thomas Berger, Universität Bern, Switzerland
Dennis Görlich, Westfälische Wilhelms-Universität Münster, Germany
Michael P. Schaub, Swiss Research Institute for Public Health and Addiction, WHO Collaborating Center, University of Zurich, Zurich, Switzerland
Heleen Riper, VU University Amsterdam, Amsterdam, The Netherlands
Ulrike Schmidt, Kings College London, United Kingdom
Rosa Baños, Universidad de Valencia, Spain
Cristina Botella, Universitat Jaume I, Spain
On Behalf of the ICare Investigators
References
1. Gilbody S, Littlewood E, Hewitt C et al. Computerised cognitive behaviour therapy (cCBT) as treatment for depression in primary care (REEACT trial):large scale pragmatic randomised controlled trial. BMJ 2015;351:h5627.
2. Cuijpers P, Dekker J, Hollon SD, et al. Adding psychotherapy to pharmacotherapy in the treatment of depressive disorders in adults: a meta-analysis. Journal of clinical psychiatry 2009;70(9),1219-1229.
3. Committee for Proprietary Medicinal Products. "Points to consider on multiplicity issues in clinical trials." London: The European agency for the evaluation of medicinal products 2002.
4. International Conference on Harmonization: Choice of control group in clinical trials. Federal register. 1999;64:51767–51780.
5. Committee for Proprietary Medicinal Products (CPMP) Points to consider on switching between superiority and non-inferiority. British journal of clinical pharmacology 2001;52,221.
6. Piaggio G, Elbourne DR, Pocock SJ, Evans SJW, Altman DG, for the CONSORT Group. Reporting of noninferiority and equivalence randomized trials. Extension of the CONSORT 2010 statement. JAMA 2012;308(24):2594-2604. doi:10.1001/jama.2012.87802.
7. Van Ballegooijen, W., Cuijpers, P., van Straten, A., Karyotaki, E., Andersson, G., Smit, J. H., & Riper, H. Adherence to Internet-based and face-to-face cognitive behavioural therapy for depression: a meta-analysis. PloS One 2014;9(7),e100674. http://doi.org/10.1371/journal.pone.0100674.
8. Paxling B, Lundgren S, Norman A, et al. Therapist behaviours in internet-delivered cognitive behaviour therapy: analyses of e-mail correspondence in the treatment of generalized anxiety disorder. Behavioural and cognitive psychotherapy 2013;41(03):280-89.
9. Titov N, Andrews G, Davies M, et al. Internet treatment for depression: A randomized controlled trial comparing clinician vs. technician assistance. PLoS ONE 2010;5(6):e10939.
10. Ebert DD, Berking M, Cuijpers P, et al. Increasing the acceptance of internet-based mental health interventions in primary care patients with depressive symptoms. A randomized controlled trial. Journal of affective disorders 2015;176:9-17.
11. Andrade, L. H., Alonso, J., Mneimneh, Z., et al. Barriers to mental health treatment: results from the WHO World Mental Health surveys. Psychological Medicine 2014;44(6),1303-1317. http://doi.org/10.1017/S0033291713001943.
Competing interests:
No competing interests
18 November 2015
Megan Jones
Psychologist and Researcher
David Daniel Ebert, Friedrich-Alexander University Erlangen-Nuremberg, Nuremberg, Germany Corinna Jacobi, Technische Universität Dresden, Dresden, Germany Ina Beintner, Technische Universität Dresden, Dresden, Germany Thomas Berger, Universität Bern, Switzerland Dennis Görlich, Westfälische Wilhelms-Universität Münster, Germany Michael P. Schaub, Swiss Research Institute for Public Health and Addiction, WHO Collaborating Center, University of Zurich, Zurich, Switzerland Heleen Riper, VU University Amsterdam, Amsterdam, The Netherlands Ulrike Schmidt, Kings College London, United Kingdom Rosa Baños, Universidad de Valencia, Spain Cristina Botella, Universitat Jaume I, Spain On Behalf of the ICare Investigators
Stanford University School of Medicine and Lantern
Rapid Response:
Why didn’t patients use it? Engagement is the real story in Gilbody et al. (2015), not effectiveness.
We congratulate the researchers on their impressive study. There is a lesson to learn from Gilbody et al., except it is not about effectiveness, it is about engagement and adherence.
Gilbody et al. (2015) reported “no additional improvement in depression compared with usual GP care at four months” for participants who were offered computer-delivered CBT (cCBT).(1) Offered is the operative word because Gilbody et al. found that participants used only 1-2 sessions of the programs. This level of adherence makes it impossible to conclude anything about effectiveness. This shows poor adherence. This is akin to saying that a pharmaceutical drug that reduces depression doesn’t work when the real issue is that people don’t want to take it because of side effects or poor explanation from their doctor.
Analyses addressing predictors of adherence are missing from this paper. The adjunct cCBT programmes may be ineffective or participants may have received sufficient usual care to reduce cCBT’s relevance. To properly test this requires a dismantling trial in which all participants received the same dose of treatment. Otherwise, this turns into testing a phantom.
The overlap between experimental and control groups obfuscates outcome.
This was a pragmatic trial, meaning that there were no restrictions on access to other treatment. This is an important test to examine real-world usage patterns of various treatment modalities. It is not appropriate for determining the efficacy of a single intervention. The authors reported that 19% of the GP care group had also actively used cCBT. When a substantial proportion of participants in the control group receive a similar intervention to those in the active experimental groups, it is difficult to make a conclusion about the superiority of one versus the others. Moreover, between 77% (MoodGYM) and 84% (GP treatment only group) received also pharmacological treatment for depression. Meta-analytic evidence indicates that the additional effect of (face-to-face) psychotherapy to pharmacological treatment only is small.(2) If a substantial proportion of all groups in addition to the pharmacological treatment also receive psychological treatment for depression in secondary care (17% in the MoodGYM group, to 28% in the GP group) it is no wonder that the authors can´t find large differences between the groups.
The general design of the trial is properly chosen under the hypothesis that cCBT was more effective than GP Care. Nevertheless, the conclusion that cCBT is not superior to GP care cannot be drawn based on this trial design. The lack of significance of a statistical test for superiority does not prove equivalence (or non-inferiority). The fact that both arms of adjunct treatment do not show superiority over GP care supports the impression of a lack of assay sensitivity, meaning, the ability to distinguish effective treatments from less effective or ineffective treatments.(3) The International Conference on Harmonization (ICH) lists a number of factors that can reduce assay sensitivity.(4) These include poor compliance with the study medication/intervention as observed in the current trial.
Therefore, the question of whether cCBT is really not superior to GP care in a pragmatic setting remains unanswered. This question could only be answered in a prospective confirmatory non-inferiority or equivalence trial. While hypothetically, it would be possible to change the presented analysis from superiority to a non-inferiority analysis there are several requirements for this decision (e.g., in the trial protocol predefined or otherwise justified non-inferiority margins which are not met in the current trial.(5)
For the presented non-inferiority comparison between Beating the Blues and MoodGYM the analysis should be repeated in the per-protocol collective to increase confidence in the results.(6)
The investigated interventions do not represent the standard for cCBT programmes.
There are numerous meta-analyses showing that guided self-help cCBT programmes have superior adherence and outcomes to self-guided programmes. MoodGYM is used as an “open access” and unguided intervention, which is not as effective as guided self-help. The adherence and engagement found in the Gilbody et al. study is worse than what has been reported in meta-analyses for other self-guided depression programmes which show completion rates of approximately 80%.(7)
The nature of the “support” provided in REEACT would not likely meet current standards for guidance in cCBT. Evidence suggests that specific guidance or “support” behaviors predict better outcomes: individualized feedback with task reinforcement, task prompting, and empathetic communication.(8) REEACT described only task prompting: reminders to complete sessions. While others have shown that online CBT with guidance from a technician can be equally effective to the same intervention provided by clinicians (e.g., Titov et al.), the quality of guidance provided by the technicians in REEACT is a key open question.(9)
How these programmes are implemented and introduced to participants is critical. The way a treatment is presented has substantial effects on adherence and, relatedly, treatment outcome expectancy.(10) Context matters probably most in such treatments, as does usability. Improved design elements such as mobile-first technology and personalisation can likely further boost engagement.
Zooming out: Using stepped care to reduce mental health care disparities
The objective of cCBT developers and researchers is not to eliminate in person care. It is to create more options within the mental health care ecosystem to improve access to effective care. The fact remains that while usual care is effective– the wait times and cost of accessing specialist or GP services are often prohibitive. Digital mental health technology offers a promising path to bridge this gap, assisting individuals whose needs can be effectively met with a lower intensity approach. Moreover, only a limited percentage of those in need of mental health treatment actually utilise it. Research indicates that this is not only a problem of evidence-based treatment availability, but also of poor utilization due to other barriers, such as stigma.(11) Digital programmes offer a low-threshold and highly acceptable (to many) way of investing in one’s own emotional wellbeing which for many, might be the only care they can receive.
Megan Jones, Lantern, San Francisco, United States and Stanford University School of Medicine, Stanford, United States
David Daniel Ebert, Friedrich-Alexander University Erlangen-Nuremberg, Nuremberg, Germany
Corinna Jacobi, Technische Universität Dresden, Dresden, Germany
Ina Beintner, Technische Universität Dresden, Dresden, Germany
Thomas Berger, Universität Bern, Switzerland
Dennis Görlich, Westfälische Wilhelms-Universität Münster, Germany
Michael P. Schaub, Swiss Research Institute for Public Health and Addiction, WHO Collaborating Center, University of Zurich, Zurich, Switzerland
Heleen Riper, VU University Amsterdam, Amsterdam, The Netherlands
Ulrike Schmidt, Kings College London, United Kingdom
Rosa Baños, Universidad de Valencia, Spain
Cristina Botella, Universitat Jaume I, Spain
On Behalf of the ICare Investigators
References
1. Gilbody S, Littlewood E, Hewitt C et al. Computerised cognitive behaviour therapy (cCBT) as treatment for depression in primary care (REEACT trial):large scale pragmatic randomised controlled trial. BMJ 2015;351:h5627.
2. Cuijpers P, Dekker J, Hollon SD, et al. Adding psychotherapy to pharmacotherapy in the treatment of depressive disorders in adults: a meta-analysis. Journal of clinical psychiatry 2009;70(9),1219-1229.
3. Committee for Proprietary Medicinal Products. "Points to consider on multiplicity issues in clinical trials." London: The European agency for the evaluation of medicinal products 2002.
4. International Conference on Harmonization: Choice of control group in clinical trials. Federal register. 1999;64:51767–51780.
5. Committee for Proprietary Medicinal Products (CPMP) Points to consider on switching between superiority and non-inferiority. British journal of clinical pharmacology 2001;52,221.
6. Piaggio G, Elbourne DR, Pocock SJ, Evans SJW, Altman DG, for the CONSORT Group. Reporting of noninferiority and equivalence randomized trials. Extension of the CONSORT 2010 statement. JAMA 2012;308(24):2594-2604. doi:10.1001/jama.2012.87802.
7. Van Ballegooijen, W., Cuijpers, P., van Straten, A., Karyotaki, E., Andersson, G., Smit, J. H., & Riper, H. Adherence to Internet-based and face-to-face cognitive behavioural therapy for depression: a meta-analysis. PloS One 2014;9(7),e100674. http://doi.org/10.1371/journal.pone.0100674.
8. Paxling B, Lundgren S, Norman A, et al. Therapist behaviours in internet-delivered cognitive behaviour therapy: analyses of e-mail correspondence in the treatment of generalized anxiety disorder. Behavioural and cognitive psychotherapy 2013;41(03):280-89.
9. Titov N, Andrews G, Davies M, et al. Internet treatment for depression: A randomized controlled trial comparing clinician vs. technician assistance. PLoS ONE 2010;5(6):e10939.
10. Ebert DD, Berking M, Cuijpers P, et al. Increasing the acceptance of internet-based mental health interventions in primary care patients with depressive symptoms. A randomized controlled trial. Journal of affective disorders 2015;176:9-17.
11. Andrade, L. H., Alonso, J., Mneimneh, Z., et al. Barriers to mental health treatment: results from the WHO World Mental Health surveys. Psychological Medicine 2014;44(6),1303-1317. http://doi.org/10.1017/S0033291713001943.
Competing interests: No competing interests