Cochrane's efforts to improve access to CSRs and provide guidance on when and how to use them
Owen Dyer highlights many of the challenges researchers face in accessing and making use of clinical study reports (CSRs) to assess healthcare interventions. The feature is of great interest to those involved in Cochrane and brings into stark focus the fundamental role CSRs can play in the completeness and trustworthiness of systematic reviews.
Like The BMJ, Cochrane has long been a passionate supporter of the AllTrials initiative and advocates for full, open-access CSRs. Examples include an open letter written in response to the FDA’s Public Comment Request on the Clinical Data Summary Report Pilot,[1] and a statement in support of the EMA’s clinical study report transparency policy following the recent legal challenge.[2] Campaigning for access to CSRs will continue to be a key focus of Cochrane's advocacy work and the organisation is in discussion with multiple data sharing platforms to facilitate access, including Vivli and the EMA. We would like to respond to the statement that Cochrane reviewers have “not embraced clinical study reports”[4] by outlining a range of initiatives that are underway to help Cochrane authors decide when to include CSRs, and to support them in doing so.
Shortly after the Cochrane Review of Tamiflu was published,[3] Cochrane funded a project to identify when to use CSRs in Cochrane Reviews. The project involved two of the review's authors, Thomas Jefferson and Peter Doshi, who are correctly posited in Dyer’s article as trailblazers in the campaign to improve access. The report was subsequently published in The BMJ and identified 18 criteria to guide when to adopt this approach.[5] Based on the results of the project, the Cochrane Scientific Committee acknowledged that CSRs were likely to be of huge importance for some, but not all, Cochrane reviews and recommended that their use be optional while the necessary methods tools and guidance for authors and editorial teams were put in place.[6]
In May 2019, Cochrane hosted a consultation meeting with clinical researchers, methodologists and Cochrane editors to discuss the future of searching for, assessing and incorporating data from CSRs into Cochrane Reviews where it would be appropriate to do so. A post about this meeting, including the slides and minutes, was subsequently made available on the Cochrane Methods website.[7] Challenges and obstacles for Cochrane Review Groups were discussed in detail by those with experience dealing with CSRs, which led to the identification of five priority areas for further work and formation of a CSR Working Group. It is important to note that Dyer's reference to the 2018 survey finding that 83% of Cochrane authors had not considered using CSRs[8] was sent to all Cochrane authors, and does not account for Jefferson and Doshi's criteria for assessing whether it is appropriate to do so.[5] The survey also investigated barriers to accessing and using CSRs and those who responded (which included both authors who had or had not considered using CSRs) highlighted many of the same challenges identified in the Cochrane consultation meeting (limited access, time and resources required, and a lack of guidance).
An update on progress was given in November 2019 as part of the Virtual Cochrane Colloquium 2019[9] and the recently formed CSR Working Group have since developed a draft starter pack to help Cochrane Groups to support author teams wanting to include CSRs in their reviews. The starter pack brings together available guidance, training, tools and support and identifies gaps where resources need to be developed. Cochrane is supporting the teams producing five ongoing Cochrane Reviews in their use of CSRs to identify the issues first-hand and help concentrate efforts to support their use more widely. There is ongoing research in Cochrane looking at risk of bias assessments for journal articles and CSRs which will further contribute to understanding the usability of CSRs within Cochrane Reviews.
Cochrane fully supports the use of CSRs in systematic reviews and is undertaking all of the initiatives described to alleviate the challenges they pose for Cochrane Review Groups and volunteer author teams. We hope that as the infrastructure and guidance develops that more Cochrane Review Groups will feel confident and supported to use CSRs within their reviews when it is relevant to the research question.
Rapid Response:
Cochrane's efforts to improve access to CSRs and provide guidance on when and how to use them
Owen Dyer highlights many of the challenges researchers face in accessing and making use of clinical study reports (CSRs) to assess healthcare interventions. The feature is of great interest to those involved in Cochrane and brings into stark focus the fundamental role CSRs can play in the completeness and trustworthiness of systematic reviews.
Like The BMJ, Cochrane has long been a passionate supporter of the AllTrials initiative and advocates for full, open-access CSRs. Examples include an open letter written in response to the FDA’s Public Comment Request on the Clinical Data Summary Report Pilot,[1] and a statement in support of the EMA’s clinical study report transparency policy following the recent legal challenge.[2] Campaigning for access to CSRs will continue to be a key focus of Cochrane's advocacy work and the organisation is in discussion with multiple data sharing platforms to facilitate access, including Vivli and the EMA. We would like to respond to the statement that Cochrane reviewers have “not embraced clinical study reports”[4] by outlining a range of initiatives that are underway to help Cochrane authors decide when to include CSRs, and to support them in doing so.
Shortly after the Cochrane Review of Tamiflu was published,[3] Cochrane funded a project to identify when to use CSRs in Cochrane Reviews. The project involved two of the review's authors, Thomas Jefferson and Peter Doshi, who are correctly posited in Dyer’s article as trailblazers in the campaign to improve access. The report was subsequently published in The BMJ and identified 18 criteria to guide when to adopt this approach.[5] Based on the results of the project, the Cochrane Scientific Committee acknowledged that CSRs were likely to be of huge importance for some, but not all, Cochrane reviews and recommended that their use be optional while the necessary methods tools and guidance for authors and editorial teams were put in place.[6]
In May 2019, Cochrane hosted a consultation meeting with clinical researchers, methodologists and Cochrane editors to discuss the future of searching for, assessing and incorporating data from CSRs into Cochrane Reviews where it would be appropriate to do so. A post about this meeting, including the slides and minutes, was subsequently made available on the Cochrane Methods website.[7] Challenges and obstacles for Cochrane Review Groups were discussed in detail by those with experience dealing with CSRs, which led to the identification of five priority areas for further work and formation of a CSR Working Group. It is important to note that Dyer's reference to the 2018 survey finding that 83% of Cochrane authors had not considered using CSRs[8] was sent to all Cochrane authors, and does not account for Jefferson and Doshi's criteria for assessing whether it is appropriate to do so.[5] The survey also investigated barriers to accessing and using CSRs and those who responded (which included both authors who had or had not considered using CSRs) highlighted many of the same challenges identified in the Cochrane consultation meeting (limited access, time and resources required, and a lack of guidance).
An update on progress was given in November 2019 as part of the Virtual Cochrane Colloquium 2019[9] and the recently formed CSR Working Group have since developed a draft starter pack to help Cochrane Groups to support author teams wanting to include CSRs in their reviews. The starter pack brings together available guidance, training, tools and support and identifies gaps where resources need to be developed. Cochrane is supporting the teams producing five ongoing Cochrane Reviews in their use of CSRs to identify the issues first-hand and help concentrate efforts to support their use more widely. There is ongoing research in Cochrane looking at risk of bias assessments for journal articles and CSRs which will further contribute to understanding the usability of CSRs within Cochrane Reviews.
Cochrane fully supports the use of CSRs in systematic reviews and is undertaking all of the initiatives described to alleviate the challenges they pose for Cochrane Review Groups and volunteer author teams. We hope that as the infrastructure and guidance develops that more Cochrane Review Groups will feel confident and supported to use CSRs within their reviews when it is relevant to the research question.
[1] https://methods.cochrane.org/news/open-response-FDA-Clinical-Data-Summar...
[2] https://www.cochrane.org/news/statement-support-emas-clinical-study-repo...
[3] https://doi.org/10.1002/14651858.CD008965.pub4
[4] BMJ 2020;368:m626 doi: 10.1136/bmj.m626
[5] https://ebm.bmj.com/content/23/6/210.long
[6] https://methods.cochrane.org/sites/default/files/public/uploads/image_fi...
[7] https://community.cochrane.org/news/using-clinical-study-reports-data-so...
[8] https://europepmc.org/article/PMC/6083614
[9] https://www.youtube.com/watch?v=pIY7Y0asMEE
Competing interests: Both authors are employed by Cochrane as members of the Editorial and Methods Department