In their letter [1], Keller and co-authors of the Study 329 report published in Journal of the American Academy of Child and Adolescent Psychiatry (JAACAP) in 2001[2] challenge the validity of the 2015 re-analysis published in The BMJ by Le Noury et al. [3], in part because of “substantial problems with RIATT [sic] methodology.” Keller et al. note that the RIAT approach lacks detailed protocols or other documents explaining RIAT methodology, and consider this problematic.
As lead and senior authors of the Restoring Invisible and Abandoned Trials (RIAT) declaration [4], we disagree with Keller et al.’s criticism, and use this opportunity to explain our position.
RIAT is a conceptual framework for bringing corrective action to the scientific literature by publishing unpublished trials and re-publishing published-but-misreported clinical trials. The 2015 publication by Le Noury et al. used the RIAT framework to republish Study 329. The basis for this was the misreporting of the study by Keller et al. in their 2001 publication in JAACAP.[2] This publication was a key piece of evidence in the US Department of Justice criminal lawsuit against GlaxoSmithKline which ultimately settled for US $3 billion in 2012.[5] Other researchers have used the RIAT framework to publish an unpublished trial on colorectal surgery.[6,7]
The RIAT declaration [4] outlines a number of steps that “restorative authors” (here, Le Noury et al.) should use to enable an ethical primary publication of a clinical trial. A key one is that RIAT papers must report the clinical trial according to the original protocol of the original trial. Any analyses conducted that were not pre-specified in the original protocol must be clearly marked as such. (We wrote: “RIAT analyses should follow the analyses specified in the protocol (including any specified in amendments). Any other analyses are discouraged, but if done must be clearly noted as exploratory and not prespecified. At the same time, RIAT authors may wish to critically appraise the trials they report. This can be useful, but the critique should be clearly identifiable and placed in the discussion section.”[4])
PD served as one of the formal peer reviewers for the Le Noury paper and as far as he can tell, the authors followed this guidance.
We and our co-authors specifically intended RIAT to be a living concept, open to suggestions for improvement. While Keller et al. express concern over a lack of “detailed methodology” for RIAT, they do not cite the RIAT declaration [4] nor mention any details of what is actually lacking in the current process. We invite them to read the RIAT declaration.
We welcome all thoughts on how to ensure the most robust RIAT papers possible.
Peter Doshi and Tom Jefferson
References:
[1] Keller M, Birmaher B, Carlson GA, Clarke GN, Emslie GJ, Koplewicz H, et al. Re: Restoring Study 329: efficacy and harms of paroxetine and imipramine in treatment of major depression in adolescence. Response from the authors of the original Study 329 [Internet]. 2016 [cited 2016 Jan 20]. Available from: http://www.bmj.com/content/351/bmj.h4320/rr-27
[2] Keller MB, Ryan ND, Strober M, et al. Efficacy of paroxetine in the treatment of adolescent major depression: a randomized, controlled trial. J Am Acad Child Adolesc Psychiatry 2001;40:762-72. http://www.ncbi.nlm.nih.gov/pubmed/11437014
[3] Le Noury J, Nardo JM, Healy D, Jureidini J, Raven M, Tufanaru C, et al. Restoring Study 329: efficacy and harms of paroxetine and imipramine in treatment of major depression in adolescence. The BMJ. 2015 Sep 16;351:h4320. http://www.bmj.com/content/351/bmj.h4320
[4] Doshi P, Dickersin K, Healy D, Vedula SS, Jefferson T. Restoring invisible and abandoned trials: a call for people to publish the findings. BMJ 2013;346:f2865. http://www.bmj.com/cgi/doi/10.1136/bmj.f2865
[5] Doshi P. No correction, no retraction, no apology, no comment: paroxetine trial reanalysis raises questions about institutional responsibility. The BMJ. 2015 Sep 16;351:h4629. http://www.bmj.com/content/351/bmj.h4629
[6] Treasure T, Monson K, Fiorentino F, Russell C. The CEA Second-Look Trial: a randomised controlled trial of carcinoembryonic antigen prompted reoperation for recurrent colorectal cancer. BMJ Open. 2014 May 1;4(5):e004385. http://bmjopen.bmj.com/content/4/5/e004385
[7] Treasure T, Monson K, Fiorentino F, Russell C. Operating to remove recurrent colorectal cancer: have we got it right? BMJ. 2014 May 13;348(may13 2):g2085. http://www.bmj.com/content/348/bmj.g2085
Competing interests:
We are the first and senior authors of the RIAT declaration, which was coauthored by David Healy, who is part of the group that reanalysed Study 329. PD served as one of the formal peer reviewer for the reanalysis manuscript and provided the Jureidini team with unpaid advice on the RIAT process before the paper was submitted and while it was under review. PD is also a graduate of Brown University, where Professor Keller is Professor Emeritus of Psychiatry and Human Behavior. PD initiated an inquiry in 2012 that resulted in additional information from clinical study reports of Study 329 and eight other studies being posted on GSK’s website. In addition, PD received €1500 from the European Respiratory Society in support of his travel to the society’s September 2012 annual congress in Vienna, where he gave an invited talk on oseltamivir. PD and TJ were co-recipients of a UK National Institute for Health Research grant (HTA – 10/80/01 Update and amalgamation of two Cochrane Reviews: neuraminidase inhibitors for preventing and treating influenza in healthy adults and children: http://www.nets.nihr.ac.uk/projects/hta/108001). This review relied on clinical study reports provided by GSK for zanamivir. TJ receives royalties from his books published by Blackwells and Il Pensiero Scientifico Editore, Rome. TJ is occasionally interviewed by market research companies for anonymous interviews about Phase 1 or 2 pharmaceutical products. In 2011-2013, TJ acted as an expert witness in a litigation case related to oseltamivir phosphate; Tamiflu [Roche] and in a labour case on influenza vaccines in healthcare workers in Canada. In 1997-99 TJ acted as a consultant for Roche, in 2001-2 for GSK, and in 2003 for Sanofi-Synthelabo for pleconaril (an anti-rhinoviral, which did not get approval from the Food and Drug Administration). TJ was a consultant for IMS Health in 2013, and in 2014 was retained as a scientific adviser to a legal team acting on the drug Tamiflu (oseltamivir, Roche). In 2014-15 TJ was a member of two advisory boards for Boerhinger and is in receipt of a Cochrane Methods Innovations Fund grant to develop guidance on the use of regulatory data in Cochrane reviews. TJ has a potential financial conflict of interest in the investigation of the drug oseltamivir. TJ is acting as an expert witness in a legal case involving the drug oseltamivir (Roche). TJ is a member of an Independent Data Monitoring Committee for a Sanofi Pasteur clinical trial.
Rapid Response:
Response to Keller et al. re: RIAT
In their letter [1], Keller and co-authors of the Study 329 report published in Journal of the American Academy of Child and Adolescent Psychiatry (JAACAP) in 2001[2] challenge the validity of the 2015 re-analysis published in The BMJ by Le Noury et al. [3], in part because of “substantial problems with RIATT [sic] methodology.” Keller et al. note that the RIAT approach lacks detailed protocols or other documents explaining RIAT methodology, and consider this problematic.
As lead and senior authors of the Restoring Invisible and Abandoned Trials (RIAT) declaration [4], we disagree with Keller et al.’s criticism, and use this opportunity to explain our position.
RIAT is a conceptual framework for bringing corrective action to the scientific literature by publishing unpublished trials and re-publishing published-but-misreported clinical trials. The 2015 publication by Le Noury et al. used the RIAT framework to republish Study 329. The basis for this was the misreporting of the study by Keller et al. in their 2001 publication in JAACAP.[2] This publication was a key piece of evidence in the US Department of Justice criminal lawsuit against GlaxoSmithKline which ultimately settled for US $3 billion in 2012.[5] Other researchers have used the RIAT framework to publish an unpublished trial on colorectal surgery.[6,7]
The RIAT declaration [4] outlines a number of steps that “restorative authors” (here, Le Noury et al.) should use to enable an ethical primary publication of a clinical trial. A key one is that RIAT papers must report the clinical trial according to the original protocol of the original trial. Any analyses conducted that were not pre-specified in the original protocol must be clearly marked as such. (We wrote: “RIAT analyses should follow the analyses specified in the protocol (including any specified in amendments). Any other analyses are discouraged, but if done must be clearly noted as exploratory and not prespecified. At the same time, RIAT authors may wish to critically appraise the trials they report. This can be useful, but the critique should be clearly identifiable and placed in the discussion section.”[4])
PD served as one of the formal peer reviewers for the Le Noury paper and as far as he can tell, the authors followed this guidance.
We and our co-authors specifically intended RIAT to be a living concept, open to suggestions for improvement. While Keller et al. express concern over a lack of “detailed methodology” for RIAT, they do not cite the RIAT declaration [4] nor mention any details of what is actually lacking in the current process. We invite them to read the RIAT declaration.
We welcome all thoughts on how to ensure the most robust RIAT papers possible.
Peter Doshi and Tom Jefferson
References:
[1] Keller M, Birmaher B, Carlson GA, Clarke GN, Emslie GJ, Koplewicz H, et al. Re: Restoring Study 329: efficacy and harms of paroxetine and imipramine in treatment of major depression in adolescence. Response from the authors of the original Study 329 [Internet]. 2016 [cited 2016 Jan 20]. Available from: http://www.bmj.com/content/351/bmj.h4320/rr-27
[2] Keller MB, Ryan ND, Strober M, et al. Efficacy of paroxetine in the treatment of adolescent major depression: a randomized, controlled trial. J Am Acad Child Adolesc Psychiatry 2001;40:762-72. http://www.ncbi.nlm.nih.gov/pubmed/11437014
[3] Le Noury J, Nardo JM, Healy D, Jureidini J, Raven M, Tufanaru C, et al. Restoring Study 329: efficacy and harms of paroxetine and imipramine in treatment of major depression in adolescence. The BMJ. 2015 Sep 16;351:h4320. http://www.bmj.com/content/351/bmj.h4320
[4] Doshi P, Dickersin K, Healy D, Vedula SS, Jefferson T. Restoring invisible and abandoned trials: a call for people to publish the findings. BMJ 2013;346:f2865. http://www.bmj.com/cgi/doi/10.1136/bmj.f2865
[5] Doshi P. No correction, no retraction, no apology, no comment: paroxetine trial reanalysis raises questions about institutional responsibility. The BMJ. 2015 Sep 16;351:h4629. http://www.bmj.com/content/351/bmj.h4629
[6] Treasure T, Monson K, Fiorentino F, Russell C. The CEA Second-Look Trial: a randomised controlled trial of carcinoembryonic antigen prompted reoperation for recurrent colorectal cancer. BMJ Open. 2014 May 1;4(5):e004385. http://bmjopen.bmj.com/content/4/5/e004385
[7] Treasure T, Monson K, Fiorentino F, Russell C. Operating to remove recurrent colorectal cancer: have we got it right? BMJ. 2014 May 13;348(may13 2):g2085. http://www.bmj.com/content/348/bmj.g2085
Competing interests: We are the first and senior authors of the RIAT declaration, which was coauthored by David Healy, who is part of the group that reanalysed Study 329. PD served as one of the formal peer reviewer for the reanalysis manuscript and provided the Jureidini team with unpaid advice on the RIAT process before the paper was submitted and while it was under review. PD is also a graduate of Brown University, where Professor Keller is Professor Emeritus of Psychiatry and Human Behavior. PD initiated an inquiry in 2012 that resulted in additional information from clinical study reports of Study 329 and eight other studies being posted on GSK’s website. In addition, PD received €1500 from the European Respiratory Society in support of his travel to the society’s September 2012 annual congress in Vienna, where he gave an invited talk on oseltamivir. PD and TJ were co-recipients of a UK National Institute for Health Research grant (HTA – 10/80/01 Update and amalgamation of two Cochrane Reviews: neuraminidase inhibitors for preventing and treating influenza in healthy adults and children: http://www.nets.nihr.ac.uk/projects/hta/108001). This review relied on clinical study reports provided by GSK for zanamivir. TJ receives royalties from his books published by Blackwells and Il Pensiero Scientifico Editore, Rome. TJ is occasionally interviewed by market research companies for anonymous interviews about Phase 1 or 2 pharmaceutical products. In 2011-2013, TJ acted as an expert witness in a litigation case related to oseltamivir phosphate; Tamiflu [Roche] and in a labour case on influenza vaccines in healthcare workers in Canada. In 1997-99 TJ acted as a consultant for Roche, in 2001-2 for GSK, and in 2003 for Sanofi-Synthelabo for pleconaril (an anti-rhinoviral, which did not get approval from the Food and Drug Administration). TJ was a consultant for IMS Health in 2013, and in 2014 was retained as a scientific adviser to a legal team acting on the drug Tamiflu (oseltamivir, Roche). In 2014-15 TJ was a member of two advisory boards for Boerhinger and is in receipt of a Cochrane Methods Innovations Fund grant to develop guidance on the use of regulatory data in Cochrane reviews. TJ has a potential financial conflict of interest in the investigation of the drug oseltamivir. TJ is acting as an expert witness in a legal case involving the drug oseltamivir (Roche). TJ is a member of an Independent Data Monitoring Committee for a Sanofi Pasteur clinical trial.