It is an outrage that 85% of research is wasted because "it asks the wrong questions, is badly designed, not published or poorly reported" ( http://rewardalliance.net ). Much remains to be done by funders, regulatory authorities, research sponsors and the scientific community. But this observation also shows that it is very difficult to do good research. Indeed no trial is perfect and if examined closely enough criticisms can be found and emphasised for almost any piece of research.
The PACE trial represents work in a clearly contentious area where there is significant disagreement between some medical/research professionals and some patient groups. But, with peer reviewed publications, public access to (critically reviewed) protocols, and access to some original data, this research does not seem to fall into the 85% of research waste. It was not a perfect trial, but whenever it has been scrutinised by organisations experienced in judging research (initial research council review, Cochrane review, and most recently the HRA) the process, methodology, analysis and the research team have been found to be well within, and acting within, the accepted norms of clinical research. Unfortunately the social context of this work means that it is unlikely that any trial on this topic could have been carried out without magnified criticism. That there is criticism (or at least still scientific questions that need answering) is not indicative that something has gone wrong. Indeed there is a famous quote (attributed alternatively to Asimov, Kekule and Flemming) that goes along the lines of: "The most exciting phrase to hear in science, the one that heralds new discoveries, is not “Eureka!” but “That’s funny …”". The community needs to move on from PACE and focus efforts on designing and conducting the next research.
When reviewing research, ethics committees do not try to judge whether a trial is perfect. Instead (and once any legal review obligations are considered for certain types of studies) we try to form an opinion on the proposed research based around the principles of Autonomy, Beneficence, Non-maleficence and Justice. The reason why a committee is needed is because these decisions cannot be reduced to an algorithm as they are necessarily complex, necessarily grey. Research Ethics Committees can be held accountable for not following process, or being inconsistent with previous decisions (or decisions made by similarly constituted committees), but the nature of the "opinion" is to provide confidence, call it a safeguard, that "outside eyes" have reviewed the research. While definitely not infallible, this system has shown itself to be sufficient, certainly to the task of protecting participants and ensuring research is publicly acceptable. What is now needed is more critical analysis of how research ethics committees can help to reduce research waste. But, as the PACE example perhaps shows, any such efforts must always be tempered by the reminder that no study, no matter how well funded or designed, is likely to be perfect in every way. While more can certainly be done to address research waste, we must not let the mythical concept of "perfect research" get in the way of the practical reality of "good research".
Competing interests:
I am chair of two UKECA recognised research ethics committees, an academic with a research focus on the role and function of research ethics committees, and involved with the REWARD Alliance Regulation and Governance Working Group.
07 February 2019
Simon E Kolstoe
Senior Lecturer
University of Portsmouth
School of Health Sciences and Social Work, University of Portsmouth, Portsmouth, UK
Rapid Response:
Perfect Research is a mythical ideal
It is an outrage that 85% of research is wasted because "it asks the wrong questions, is badly designed, not published or poorly reported" ( http://rewardalliance.net ). Much remains to be done by funders, regulatory authorities, research sponsors and the scientific community. But this observation also shows that it is very difficult to do good research. Indeed no trial is perfect and if examined closely enough criticisms can be found and emphasised for almost any piece of research.
The PACE trial represents work in a clearly contentious area where there is significant disagreement between some medical/research professionals and some patient groups. But, with peer reviewed publications, public access to (critically reviewed) protocols, and access to some original data, this research does not seem to fall into the 85% of research waste. It was not a perfect trial, but whenever it has been scrutinised by organisations experienced in judging research (initial research council review, Cochrane review, and most recently the HRA) the process, methodology, analysis and the research team have been found to be well within, and acting within, the accepted norms of clinical research. Unfortunately the social context of this work means that it is unlikely that any trial on this topic could have been carried out without magnified criticism. That there is criticism (or at least still scientific questions that need answering) is not indicative that something has gone wrong. Indeed there is a famous quote (attributed alternatively to Asimov, Kekule and Flemming) that goes along the lines of: "The most exciting phrase to hear in science, the one that heralds new discoveries, is not “Eureka!” but “That’s funny …”". The community needs to move on from PACE and focus efforts on designing and conducting the next research.
When reviewing research, ethics committees do not try to judge whether a trial is perfect. Instead (and once any legal review obligations are considered for certain types of studies) we try to form an opinion on the proposed research based around the principles of Autonomy, Beneficence, Non-maleficence and Justice. The reason why a committee is needed is because these decisions cannot be reduced to an algorithm as they are necessarily complex, necessarily grey. Research Ethics Committees can be held accountable for not following process, or being inconsistent with previous decisions (or decisions made by similarly constituted committees), but the nature of the "opinion" is to provide confidence, call it a safeguard, that "outside eyes" have reviewed the research. While definitely not infallible, this system has shown itself to be sufficient, certainly to the task of protecting participants and ensuring research is publicly acceptable. What is now needed is more critical analysis of how research ethics committees can help to reduce research waste. But, as the PACE example perhaps shows, any such efforts must always be tempered by the reminder that no study, no matter how well funded or designed, is likely to be perfect in every way. While more can certainly be done to address research waste, we must not let the mythical concept of "perfect research" get in the way of the practical reality of "good research".
Competing interests: I am chair of two UKECA recognised research ethics committees, an academic with a research focus on the role and function of research ethics committees, and involved with the REWARD Alliance Regulation and Governance Working Group.