Re: Clinical effectiveness of an online supervised group physical and mental health rehabilitation programme for adults with post-covid-19 condition (REGAIN study): multicentre randomised controlled trial
CCBYOpen access
Rapid response to:
Research
Clinical effectiveness of an online supervised group physical and mental health rehabilitation programme for adults with post-covid-19 condition (REGAIN study): multicentre randomised controlled trial
Re: Clinical effectiveness of an online supervised group physical and mental health rehabilitation programme for adults with post-covid-19 condition (REGAIN study): multicentre randomised controlled trial
Dear Editor
Correspondents have raised some queries about the results of REGAIN trial (1).
1. That we make sweeping conclusions by extrapolating findings from our population to all people with long-covid.
This is incorrect. We clearly define our population of interest throughout the article, in the abstract, introduction, methods, discussion and conclusion. We do not extrapolate our findings to all people with post-covid-19 condition. Our conclusion states:
“Among adults with post-covid-19 condition at least three months after hospital discharge for covid-19, an individualised online, group physical and mental health rehabilitation intervention improved overall heath related quality of life more than usual care at three- and 12-months post-randomisation.”
2. That the trial is unblinded.
It is not possible to blind participants to their treatment allocation in trials of complex interventions. However, as reported in the article, we did maintain blinding of any study team members involved in collecting outcome data.
3. That the trial relied solely on subjective outcomes.
Our patient-partners with post-covid-19 condition, who helped us design the trial, unequivocally identified that improvement in overall quality of life was their top priority over and above objective measures of physical function. The core outcome set for post-covid-19 condition, developed by the COSMIN international group, did not include any objective outcomes of physical function (2). Regardless of any potential merit from collecting objective data such as six-minute walk test, this was clearly impossible during the height of the covid-19 epidemic in the UK. We did, however, collect data on employment status for our forthcoming health economic analysis. The number of participants who reported being unable to work at least one day in the previous three months for intervention vs usual care was 19% (41/216) vs 25% (56/20) at 3 months, 20% (43/216) vs 22% (48/ 223), at six months, and 18% (38/211) vs 28% (61/216) at 12 months.
4. That the REGAIN intervention puts people at risk of post exertional symptom exacerbation and that incomplete compliance in the intervention group may be an indication of this.
On enrolment we did not exclude people who described sequalae consistent with post-exertional symptom exacerbation. Throughout the trial we prospectively monitored for post exertional symptom exacerbation during and after every exercise and psychological support session. This included full and partial compliers. We did not identify any instances in over 1,000 hours of exercise and psychological support sessions.
Attention has been drawn to the difference in full compliance rate between the intervention and control group (90% vs 47%). Compliance will, inevitably, be lower in the intervention group, in a trial of this nature. To fully comply with the control intervention participants needed to attend just one ‘best-practice usual care session’. To fully comply with the REGAIN intervention participants needed to attend an initial one-to-one consultation session, ≥4/6 support sessions, and ≥5/8 exercise sessions. Given the nature of the disorder, it is a testament to the value participants gave to the treatment sessions that nearly half were able to attend both the initial one-to-one session and nine or more group sessions. To suggest that post-exertional symptom exacerbation might have contributed to less than full compliance, in the population recruited this trial, is pure speculation with no supportive evidence.
5. That we have wrongly concluded that the REGAIN intervention is effective because the observed effect size is small.
This criticism is largely a reprise of material from our discussion where we sought to put the observed effect size in context. As the correspondent notes, our observed effect size 0.03 (95% confidence interval 0.01 to 0.05) is at the lower end of what might be considered a ‘minimally clinical important difference’ for preference-based measures (0.03 to 0.05). The currently suggested ‘minimally clinical important difference’ of 0.04 for the PROPr score is from 'work-in-progress', and therefore, may be subject to change (3). Further, it falls into the 95% confidence interval of the effect size observed in our trial.
Important limitations in using uniform ‘minimally clinically important differences’ to define if interventions are worthwhile include that they are not context specific, and they do not take the patient’s perspective into account (4). In our protocol, and methods, we clearly state that what might be a worthwhile benefit has not been defined for an intervention of this nature in this population.
The correspondents have ignored the additional information we present to help patients decide, with support from their clinicians, if this is a treatment suitable for them: numbers needed to treat for participants to report feeling ‘much better now’ (11.9) and at least ‘somewhat better now’ (5.4).
6. That we are basing our conclusions on the treatment effect in the fully compliant sub-set.
Our conclusions are based on the intention to treat analysis. We presented a complier average causal effect analysis to aid interpretation. Complier average causal effect analysis does not simply measure the effect in a sub-set of participants, rather it estimates the treatment effect in people randomly assigned to the intervention who actually received it by comparing participants who fully or partially adhered in the intervention group with participants in the usual care group who would have been classed as adherent had they been allocated to the intervention group. The fully compliant analysis does show a larger effect (0.05 [95% CI; 0.01 to 0.09]) when compared to the intention to treat analysis (0.03 [0.01 to 0.05]). We agree that that such analyses should be interpreted with caution and caveated our interpretation accordingly in the article ‘…suggesting the true effect, in those fully complying with the intervention, might exceed this threshold [0.04]’.
Correspondents have ignored another reason why the true effect size, when compared to no treatment, might possibly be larger than that observed. Our usual care group received more intervention than might have been offered in clinical practice which may have masked the true effect of the REGAIN intervention.
Taking our findings overall we have clearly demonstrated statistically significant clinical benefits, for a treatment with few observed harms, of a magnitude that individual patients might find attractive for a disorder for which, to the best of our knowledge, there is no other treatment of proven effectiveness.
1. McGregor G, Sandhu H, Bruce J, Sheehan B, McWilliams D, Yeung J, et al. Clinical effectiveness of an online supervised group physical and mental health rehabilitation programme for adults with post-covid-19 condition (REGAIN study): multicentre randomised controlled trial. BMJ. 2024;384:e076506.
2. Gorst SL, Seylanova N, Dodd SR, Harman NL, O'Hara M, Terwee CB, et al. Core outcome measurement instruments for use in clinical and research settings for adults with post-COVID-19 condition: an international Delphi consensus study. The Lancet Respiratory Medicine. 2023;11(12):1101-14.
3. National Institute for Health. What it this minimally important difference for PROPr? Available at: https://www.proprscore.com/faqs/2023.
4. Goyal M, McDonough R, Fisher M, Ospel J. The Challenge of Designing Stroke Trials That Change Practice: MCID vs. Sample Size and Pragmatism. J Stroke. 2022;24(1):49-56.
Competing interests:
GM, MU, JB, HS are chief or co-investigator on multiple previous and current research grants from the UK National Institute for Health Research.
GM receives payment as a director of Atrium Health Ltd, a non-profit cardiopulmonary rehabilitation provider, which provided the treatment hub for the REGAIN trial.
HS is a director of Health Psychology Services Ltd, a private health psychology provider.
MU is a current or recent co-investigator on Arthritis Research UK, Australian NHMRC, and Norwegian MRC grants. He is a director and shareholder of Clinvivo Ltd that provides electronic data collection for health services research. He receives some salary support from University Hospitals Coventry and Warwickshire NHS Trust. He is a co-investigator on two current and one recent NIHR funded studies receiving additional support from Stryker Ltd.
JB is supported by NIHR Research Capability Funding via university Hospitals Coventry and Warwickshire NHS Trust.
11 April 2024
Gordon S McGregor
Professor
Julie Bruce, Harbinder Sandhu and Martin Underwood, on behalf of the REGAIN trial investigators
University Hospitals Coventry & Warwickshire NHS Trust, Coventry, UK; Warwick Clinical Trials Unit, Warwick Medical School, University of Warwick, Coventry, UK; Research Institute for Health & Wellbeing, Coventry University, Coventry, UK
Rapid Response:
Re: Clinical effectiveness of an online supervised group physical and mental health rehabilitation programme for adults with post-covid-19 condition (REGAIN study): multicentre randomised controlled trial
Dear Editor
Correspondents have raised some queries about the results of REGAIN trial (1).
1. That we make sweeping conclusions by extrapolating findings from our population to all people with long-covid.
This is incorrect. We clearly define our population of interest throughout the article, in the abstract, introduction, methods, discussion and conclusion. We do not extrapolate our findings to all people with post-covid-19 condition. Our conclusion states:
“Among adults with post-covid-19 condition at least three months after hospital discharge for covid-19, an individualised online, group physical and mental health rehabilitation intervention improved overall heath related quality of life more than usual care at three- and 12-months post-randomisation.”
2. That the trial is unblinded.
It is not possible to blind participants to their treatment allocation in trials of complex interventions. However, as reported in the article, we did maintain blinding of any study team members involved in collecting outcome data.
3. That the trial relied solely on subjective outcomes.
Our patient-partners with post-covid-19 condition, who helped us design the trial, unequivocally identified that improvement in overall quality of life was their top priority over and above objective measures of physical function. The core outcome set for post-covid-19 condition, developed by the COSMIN international group, did not include any objective outcomes of physical function (2). Regardless of any potential merit from collecting objective data such as six-minute walk test, this was clearly impossible during the height of the covid-19 epidemic in the UK. We did, however, collect data on employment status for our forthcoming health economic analysis. The number of participants who reported being unable to work at least one day in the previous three months for intervention vs usual care was 19% (41/216) vs 25% (56/20) at 3 months, 20% (43/216) vs 22% (48/ 223), at six months, and 18% (38/211) vs 28% (61/216) at 12 months.
4. That the REGAIN intervention puts people at risk of post exertional symptom exacerbation and that incomplete compliance in the intervention group may be an indication of this.
On enrolment we did not exclude people who described sequalae consistent with post-exertional symptom exacerbation. Throughout the trial we prospectively monitored for post exertional symptom exacerbation during and after every exercise and psychological support session. This included full and partial compliers. We did not identify any instances in over 1,000 hours of exercise and psychological support sessions.
Attention has been drawn to the difference in full compliance rate between the intervention and control group (90% vs 47%). Compliance will, inevitably, be lower in the intervention group, in a trial of this nature. To fully comply with the control intervention participants needed to attend just one ‘best-practice usual care session’. To fully comply with the REGAIN intervention participants needed to attend an initial one-to-one consultation session, ≥4/6 support sessions, and ≥5/8 exercise sessions. Given the nature of the disorder, it is a testament to the value participants gave to the treatment sessions that nearly half were able to attend both the initial one-to-one session and nine or more group sessions. To suggest that post-exertional symptom exacerbation might have contributed to less than full compliance, in the population recruited this trial, is pure speculation with no supportive evidence.
5. That we have wrongly concluded that the REGAIN intervention is effective because the observed effect size is small.
This criticism is largely a reprise of material from our discussion where we sought to put the observed effect size in context. As the correspondent notes, our observed effect size 0.03 (95% confidence interval 0.01 to 0.05) is at the lower end of what might be considered a ‘minimally clinical important difference’ for preference-based measures (0.03 to 0.05). The currently suggested ‘minimally clinical important difference’ of 0.04 for the PROPr score is from 'work-in-progress', and therefore, may be subject to change (3). Further, it falls into the 95% confidence interval of the effect size observed in our trial.
Important limitations in using uniform ‘minimally clinically important differences’ to define if interventions are worthwhile include that they are not context specific, and they do not take the patient’s perspective into account (4). In our protocol, and methods, we clearly state that what might be a worthwhile benefit has not been defined for an intervention of this nature in this population.
The correspondents have ignored the additional information we present to help patients decide, with support from their clinicians, if this is a treatment suitable for them: numbers needed to treat for participants to report feeling ‘much better now’ (11.9) and at least ‘somewhat better now’ (5.4).
6. That we are basing our conclusions on the treatment effect in the fully compliant sub-set.
Our conclusions are based on the intention to treat analysis. We presented a complier average causal effect analysis to aid interpretation. Complier average causal effect analysis does not simply measure the effect in a sub-set of participants, rather it estimates the treatment effect in people randomly assigned to the intervention who actually received it by comparing participants who fully or partially adhered in the intervention group with participants in the usual care group who would have been classed as adherent had they been allocated to the intervention group. The fully compliant analysis does show a larger effect (0.05 [95% CI; 0.01 to 0.09]) when compared to the intention to treat analysis (0.03 [0.01 to 0.05]). We agree that that such analyses should be interpreted with caution and caveated our interpretation accordingly in the article ‘…suggesting the true effect, in those fully complying with the intervention, might exceed this threshold [0.04]’.
Correspondents have ignored another reason why the true effect size, when compared to no treatment, might possibly be larger than that observed. Our usual care group received more intervention than might have been offered in clinical practice which may have masked the true effect of the REGAIN intervention.
Taking our findings overall we have clearly demonstrated statistically significant clinical benefits, for a treatment with few observed harms, of a magnitude that individual patients might find attractive for a disorder for which, to the best of our knowledge, there is no other treatment of proven effectiveness.
1. McGregor G, Sandhu H, Bruce J, Sheehan B, McWilliams D, Yeung J, et al. Clinical effectiveness of an online supervised group physical and mental health rehabilitation programme for adults with post-covid-19 condition (REGAIN study): multicentre randomised controlled trial. BMJ. 2024;384:e076506.
2. Gorst SL, Seylanova N, Dodd SR, Harman NL, O'Hara M, Terwee CB, et al. Core outcome measurement instruments for use in clinical and research settings for adults with post-COVID-19 condition: an international Delphi consensus study. The Lancet Respiratory Medicine. 2023;11(12):1101-14.
3. National Institute for Health. What it this minimally important difference for PROPr? Available at: https://www.proprscore.com/faqs/2023.
4. Goyal M, McDonough R, Fisher M, Ospel J. The Challenge of Designing Stroke Trials That Change Practice: MCID vs. Sample Size and Pragmatism. J Stroke. 2022;24(1):49-56.
Competing interests: GM, MU, JB, HS are chief or co-investigator on multiple previous and current research grants from the UK National Institute for Health Research. GM receives payment as a director of Atrium Health Ltd, a non-profit cardiopulmonary rehabilitation provider, which provided the treatment hub for the REGAIN trial. HS is a director of Health Psychology Services Ltd, a private health psychology provider. MU is a current or recent co-investigator on Arthritis Research UK, Australian NHMRC, and Norwegian MRC grants. He is a director and shareholder of Clinvivo Ltd that provides electronic data collection for health services research. He receives some salary support from University Hospitals Coventry and Warwickshire NHS Trust. He is a co-investigator on two current and one recent NIHR funded studies receiving additional support from Stryker Ltd. JB is supported by NIHR Research Capability Funding via university Hospitals Coventry and Warwickshire NHS Trust.