Intended for healthcare professionals

Research Methods & Reporting

How to estimate the health benefits of additional research and changing clinical practice

BMJ 2015; 351 doi: https://doi.org/10.1136/bmj.h5987 (Published 25 November 2015) Cite this as: BMJ 2015;351:h5987

Re: How to estimate the health benefits of additional research and changing clinical practice

Dear Editor,

We thank Tuffaha and Scuffham for their considered response to our article.

We of course agree that in order to utilise the results of meta-analysis it is necessary to account for the quality of the included studies, the risk of bias and the relevance of the pooled result to the decision problem.

Tuffaha and Scuffham expand further our discussion of the other aspects of outcome to consider. More fully characterising the uncertain aspects of the decision problem would indeed alter the estimated consequences of uncertainty; any resultant maximum value of research will pertain to the specified final outcome, the parameters successfully quantified within the analysis, and the modelled relationships between parameters. Increasing the complexity of the model allows for the incorporation of more aspects of outcome, but it is unlikely that any model could truly quantify all aspects of decision uncertainty. Ultimately, similar considerations are required in utilising the results of this type analysis as with the use of the results of the underlying meta-analysis. That is, what aspects of outcomes are included in the model, what is the risk of bias, and what is the relevance of the result to the decision problem? Our suggested approach relies upon a simple model with few parameters, which has the advantages of being easy to implement in terms of both time and level of expertise required and in making clear precisely what aspects of outcome are captured. The resultant maximum value of additional research is a useful piece of information, and represents a distinct improvement over considerations of statistical significance, both in terms of prioritising further research and in prioritising the use of more resource demanding methods for estimating the expected net improvement in health benefits that could be provided by alternative study designs.

Karl Claxton, Susan Griffin, Hendrik Koffijberg, Claire Rothery (nee McKenna)

Competing interests: No competing interests

05 February 2016
Susan Griffin
Senior Research Fellow
Karl Claxton, Hendrik Koffijberg, Claire Rothery
Centre for Health Economics, University of York
Alcuin A Block, University of York, Heslington West, YO10 5DD