How to formulate research recommendationsBMJ 2006; 333 doi: https://doi.org/10.1136/bmj.38987.492014.94 (Published 12 October 2006) Cite this as: BMJ 2006;333:804
- Polly Brown (), publishing manager1,
- Klara Brunnhuber, clinical editor1,
- Kalipso Chalkidou, associate director, research and development2,
- Iain Chalmers, director3,
- Mike Clarke, director4,
- Mark Fenton, editor3,
- Carol Forbes, reviews manager5,
- Julie Glanville, associate director/information service manager5,
- Nicholas J Hicks, consultant in public health medicine6,
- Janet Moody, identification and prioritisation manager6,
- Sara Twaddle, director7,
- Hazim Timimi, systems developer8,
- Pamela Young, senior programme manager6
- 1 BMJ Publishing Group, London WC1H 9JR,
- 2National Institute for Health and Clinical Excellence, London WC1V 6NA,
- 3Database of Uncertainties about the Effects of Treatments, James Lind Alliance Secretariat, James Lind Initiative, Oxford OX2 7LG,
- 4UK Cochrane Centre, Oxford OX2 7LG,
- 5Centre for Reviews and Dissemination, University of York, York YO10 5DD,
- 6National Coordinating Centre for Health Technology Assessment, University of Southampton, Southampton SO16 7PX,
- 7Scottish Intercollegiate Guidelines Network, Edinburgh EH2 1EN,
- 8Update Software, Oxford OX2 7LG
- Correspondence to: PBrown
- Accepted 22 September 2006
Long awaited reports of new research, systematic reviews, and clinical guidelines are too often a disappointing anticlimax for those wishing to use them to direct future research. After many months or years of effort and intellectual energy put into these projects, authors miss the opportunity to identify unanswered questions and outstanding gaps in the evidence. Most reports contain only a less than helpful, general research recommendation. This means that the potential value of these recommendations is lost.
In 2005, representatives of organisations commissioning and summarising research, including the BMJ Publishing Group, the Centre for Reviews and Dissemination, the National Coordinating Centre for Health Technology Assessment, the National Institute for Health and Clinical Excellence, the Scottish Intercollegiate Guidelines Network, and the UK Cochrane Centre, met as members of the development group for the Database of Uncertainties about the Effects of Treatments (see bmj.com for details on all participating organisations). Our aim was to discuss the state of research recommendations within our organisations and to develop guidelines for improving the presentation of proposals for further research. All organisations had found weaknesses in the way researchers and authors of systematic reviews and clinical guidelines stated the need for further research. As part of the project, a member of the Centre for Reviews and Dissemination under-took a rapid literature search to identify information on research recommendation models, which found some individual methods but no group initiatives to attempt to standardise recommendations.
Suggested format for research recommendations on the effects of treatments
E Evidence (What is the current state of the evidence?)
P Population (What is the population of interest?)
I Intervention (What are the interventions of interest?)
C Comparison (What are the comparisons of interest?)
O Outcome (What are the outcomes of interest?)
T Time stamp (Date of recommendation)
d Disease burden or relevance
t Time aspect of core elements of EPICOT
s Appropriate study type according to local need
In January 2006, the National Coordinating Centre for Health Technology Assessment presented the findings of an initial comparative analysis of how different organisations currently structure their research recommendations. The National Institute for Health and Clinical Excellence and the National Coordinating Centre for Health Technology Assessment request authors to present recommendations in a four component format for formulating well built clinical questions around treatments: population, intervention, comparison, and outcomes (PICO).1 In addition, the research recommendation is dated and authors are asked to provide the current state of the evidence to support the proposal.
Clinical Evidence, although not directly standardising its sections for research recommendations, presents gaps in the evidence using a slightly extended version of the PICO format: evidence, population, intervention, comparison, outcomes, and time (EPICOT). Clinical Evidence has used this inherent structure to feed research recommendations on interventions categorised as “unknown effectiveness” back to the National Coordinating Centre for Health Technology Assessment and for inclusion in the Database of Uncertainties about the Effects of Treatments (http://www.duets.nhs.uk/).
We decided to propose the EPICOT format as the basis for its statement on formulating research recommendations and tested this proposal through discussion and example. We agreed that this set of components provided enough context for formulating research recommendations without limiting researchers. In order for the proposed framework to be flexible and more widely applicable, the group discussed using several optional components when they seemed relevant or were proposed by one or more of the group members. The final outcome of discussions resulted in the proposed EPICOT+ format (box).
A recent BMJ article highlighted how lack of research hinders the applicability of existing guidelines to patients in primary care who have had a stroke or transient ischaemic attack.2 Most research in the area had been conducted in younger patients with a recent episode and in a hospital setting. The authors concluded that “further evidence should be collected on the efficacy and adverse effects of intensive blood pressure lowering in representative populations before we implement this guidance [from national and international guidelines] in primary care.” Table 1 outlines how their recommendations could be formulated using the EPICOT+ format. The decision on whether additional research is indeed clinically and ethically warranted will still lie with the organisation considering commissioning the research.
Table 2 shows the use of EPICOT+ for an unanswered question on the effectiveness of compliance therapy in people with schizophrenia, identified by the Database of Uncertainties about the Effects of Treatments.
Discussions around optional elements
Although the group agreed that the PICO elements should be core requirements for a research recommendation, intense discussion centred on the inclusion of factors defining a more detailed context, such as current state of evidence (E), appropriate study type (s), disease burden and relevance (d), and timeliness (t).
Initially, group members interpreted E differently. Some viewed it as the supporting evidence for a research recommendation and others as the suggested study type for a research recommendation. After discussion, we agreed that E should be used to refer to the amount and quality of research supporting the recommendation. However, the issue remained contentious as some of us thought that if a systematic review was available, its reference would sufficiently identify the strength of the existing evidence. Others thought that adding evidence to the set of core elements was important as it provided a summary of the supporting evidence, particularly as the recommendation was likely to be abstracted and used separately from the review or research that led to its formulation. In contrast, the suggested study type (s) was left as an optional element.
A research recommendation will rarely have an absolute value in itself. Its relative priority will be influenced by the burden of ill health (d), which is itself dependent on factors such as local prevalence, disease severity, relevant risk factors, and the priorities of the organisation considering commissioning the research.
Similarly, the issue of time (t) could be seen to be relevant to each of the core elements in varying ways—for example, duration of treatment, length of follow-up. The group therefore agreed that time had a subsidiary role within each core item; however, T as the date of the recommendation served to define its shelf life and therefore retained individual importance.
Applicability and usability
The proposed statement on research recommendations applies to uncertainties of the effects of any form of health intervention or treatment and is intended for research in humans rather than basic scientific research. Further investigation is required to assess the applicability of the format for questions around diagnosis, signs and symptoms, prognosis, investigations, and patient preference.
When the proposed format is applied to a specific research recommendation, the emphasis placed on the relevant part(s) of the EPICOT+ format may vary by author, audience, and intended purpose. For example, a recommendation for research into treatments for transient ischaemic attack may or may not define valid outcome measures to assess quality of life or gather data on adverse effects. Among many other factors, its implementation will also depend on the strength of current findings—that is, strong evidence may support a tightly focused recommendation whereas a lack of evidence would result in a more general recommendation.
The controversy within the group, especially around the optional components, reflects the different perspectives of the participating organisations—whether they were involved in commissioning, undertaking, or summarising research. Further issues will arise during the implementation of the proposed format, and we welcome feedback and discussion.
No common guidelines exist for the formulation of recommendations for research on the effects of treatments
Major organisations involved in commissioning or summarising research compared their approaches and agreed on core questions
The essential items can be summarised as EPICOT+ (evidence, population, intervention, comparison, outcome, and time)
Further details, such as disease burden and appropriate study type, should be considered as required
Details of participating organisations are on bmj.com
We thank Patricia Atkinson and Jeremy Wyatt.
Contributors and sources All authors contributed to manuscript preparation and approved the final draft. NJH is the guarantor.
Competing interests None declared.