BMJ  2006;333:804-806 (14 October), doi:10.1136/bmj.38987.492014.94

Analysis and comment

Health research

How to formulate research recommendations

¹ BMJ Publishing Group, London WC1H 9JR, ² National Institute for Health and Clinical Excellence, London WC1V 6NA, ³ Database of Uncertainties about the Effects of Treatments, James Lind Alliance Secretariat, James Lind Initiative, Oxford OX2 7LG, ⁴ UK Cochrane Centre, Oxford OX2 7LG, ⁵ Centre for Reviews and Dissemination, University of York, York YO10 5DD, ⁶ National Coordinating Centre for Health Technology Assessment, University of Southampton, Southampton SO16 7PX, ⁷ Scottish Intercollegiate Guidelines Network, Edinburgh EH2 1EN, ⁸ Update Software, Oxford OX2 7LG

"More research is needed" is a conclusion that fits most systematic reviews. But authors need to be more specific about what exactly is required

Long awaited reports of new research, systematic reviews, and clinical guidelines are too often a disappointing anticlimax for those wishing to use them to direct future research. After many months or years of effort and intellectual energy put into these projects, authors miss the opportunity to identify unanswered questions and outstanding gaps in the evidence. Most reports contain only a less than helpful, general research recommendation. This means that the potential value of these recommendations is lost.

Current recommendations

In 2005, representatives of organisations commissioning and summarising research, including the BMJ Publishing Group, the Centre for Reviews and Dissemination, the National Coordinating Centre for Health Technology Assessment, the National Institute for Health and Clinical Excellence, the Scottish Intercollegiate Guidelines Network, and the UK Cochrane Centre, met as members of the development group for the Database of Uncertainties about the Effects of Treatments (see bmj.com for details on all participating organisations). Our aim was to discuss the state of research recommendations within our organisations and to develop guidelines for improving the presentation of proposals for further research. All organisations had found weaknesses in the way researchers and authors of systematic reviews and clinical guidelines stated the need for further research. As part of the project, a member of the Centre for Reviews and Dissemination under-took a rapid literature search to identify information on research recommendation models, which found some individual methods but no group initiatives to attempt to standardise recommendations.

Suggested format for research recommendations on the effects of treatments Core elements E Evidence (What is the current state of the evidence?) P Population (What is the population of interest?) I Intervention (What are the interventions of interest?) C Comparison (What are the comparisons of interest?) O Outcome (What are the outcomes of interest?) T Time stamp (Date of recommendation) Optional elements d Disease burden or relevance t Time aspect of core elements of EPICOT s Appropriate study type according to local need

In January 2006, the National Coordinating Centre for Health Technology Assessment presented the findings of an initial comparative analysis of how different organisations currently structure their research recommendations. The National Institute for Health and Clinical Excellence and the National Coordinating Centre for Health Technology Assessment request authors to present recommendations in a four component format for formulating well built clinical questions around treatments: population, intervention, comparison, and outcomes (PICO).¹ In addition, the research recommendation is dated and authors are asked to provide the current state of the evidence to support the proposal.

Clinical Evidence, although not directly standardising its sections for research recommendations, presents gaps in the evidence using a slightly extended version of the PICO format: evidence, population, intervention, comparison, outcomes, and time (EPICOT). Clinical Evidence has used this inherent structure to feed research recommendations on interventions categorised as "unknown effectiveness" back to the National Coordinating Centre for Health Technology Assessment and for inclusion in the Database of Uncertainties about the Effects of Treatments (www.duets.nhs.uk).

We decided to propose the EPICOT format as the basis for its statement on formulating research recommendations and tested this proposal through discussion and example. We agreed that this set of components provided enough context for formulating research recommendations without limiting researchers. In order for the proposed framework to be flexible and more widely applicable, the group discussed using several optional components when they seemed relevant or were proposed by one or more of the group members. The final outcome of discussions resulted in the proposed EPICOT+ format (box).

Examples

A recent BMJ article highlighted how lack of research hinders the applicability of existing guidelines to patients in primary care who have had a stroke or transient ischaemic attack.² Most research in the area had been conducted in younger patients with a recent episode and in a hospital setting. The authors concluded that "further evidence should be collected on the efficacy and adverse effects of intensive blood pressure lowering in representative populations before we implement this guidance [from national and international guidelines] in primary care." Table 1 outlines how their recommendations could be formulated using the EPICOT+ format. The decision on whether additional research is indeed clinically and ethically warranted will still lie with the organisation considering commissioning the research.

View this table: [in this window] [in a new window]   Table 1 Research recommendation based on gap in the evidence identified by a cross sectional study of clinical guidelines for management of patients who have had a stroke

 

Table 2 shows the use of EPICOT+ for an unanswered question on the effectiveness of compliance therapy in people with schizophrenia, identified by the Database of Uncertainties about the Effects of Treatments.

View this table: [in this window] [in a new window]   Table 2 Research recommendation based on a gap in the evidence on treatment of schizophrenia identified by the Database of Uncertainties about the Effects of Treatments

 

Discussions around optional elements

Although the group agreed that the PICO elements should be core requirements for a research recommendation, intense discussion centred on the inclusion of factors defining a more detailed context, such as current state of evidence (E), appropriate study type (s), disease burden and relevance (d), and timeliness (t).

Initially, group members interpreted E differently. Some viewed it as the supporting evidence for a research recommendation and others as the suggested study type for a research recommendation. After discussion, we agreed that E should be used to refer to the amount and quality of research supporting the recommendation. However, the issue remained contentious as some of us thought that if a systematic review was available, its reference would sufficiently identify the strength of the existing evidence. Others thought that adding evidence to the set of core elements was important as it provided a summary of the supporting evidence, particularly as the recommendation was likely to be abstracted and used separately from the review or research that led to its formulation. In contrast, the suggested study type (s) was left as an optional element.

A research recommendation will rarely have an absolute value in itself. Its relative priority will be influenced by the burden of ill health (d), which is itself dependent on factors such as local prevalence, disease severity, relevant risk factors, and the priorities of the organisation considering commissioning the research.

Similarly, the issue of time (t) could be seen to be relevant to each of the core elements in varying ways—for example, duration of treatment, length of follow-up. The group therefore agreed that time had a subsidiary role within each core item; however, T as the date of the recommendation served to define its shelf life and therefore retained individual importance.

Applicability and usability

The proposed statement on research recommendations applies to uncertainties of the effects of any form of health intervention or treatment and is intended for research in humans rather than basic scientific research. Further investigation is required to assess the applicability of the format for questions around diagnosis, signs and symptoms, prognosis, investigations, and patient preference.

When the proposed format is applied to a specific research recommendation, the emphasis placed on the relevant part(s) of the EPICOT+ format may vary by author, audience, and intended purpose. For example, a recommendation for research into treatments for transient ischaemic attack may or may not define valid outcome measures to assess quality of life or gather data on adverse effects. Among many other factors, its implementation will also depend on the strength of current findings—that is, strong evidence may support a tightly focused recommendation whereas a lack of evidence would result in a more general recommendation.

The controversy within the group, especially around the optional components, reflects the different perspectives of the participating organisations—whether they were involved in commissioning, undertaking, or summarising research. Further issues will arise during the implementation of the proposed format, and we welcome feedback and discussion.

Summary points No common guidelines exist for the formulation of recommendations for research on the effects of treatments Major organisations involved in commissioning or summarising research compared their approaches and agreed on core questions The essential items can be summarised as EPICOT+ (evidence, population, intervention, comparison, outcome, and time) Further details, such as disease burden and appropriate study type, should be considered as required

---

Details of participating organisations are on bmj.com

We thank Patricia Atkinson and Jeremy Wyatt.

Contributors and sources: All authors contributed to manuscript preparation and approved the final draft. NJH is the guarantor.

Competing interests: None declared.

References

1. Richardson WS, Wilson MC, Nishikawa J, Hayward RSA. The well-built clinical question: a key to evidence-based decisions [editorial]. ACP J Club 1995;123: A12-3.[Medline]
2. Mant J, McManus RJ, Hare R. Applicability to primary care of national clinical guidelines on blood pressure lowering for people with stroke: cross sectional study. BMJ 2006;332: 635-7.[Abstract/Free Full Text]
3. Rashid R, Leonardi-Bee J, Bath P. Blood pressure reduction and secondary prevention of stroke and other vascular events. A systematic review. Stroke 2003;34: 2741-9.[Abstract/Free Full Text]
4. PROGRESS Collaborative Group. Randomised trial of a perindopril based blood pressure lowering regimen among 6,105 individuals with a previous stroke or transient ischaemic attack. Lancet 2001;358: 1033-41.[CrossRef][ISI][Medline]
5. Warburton E. Stroke management. Clinical Evidence 2006;15: 49-51.
6. Lip GYH, Rothwell P, Sudlow C. Stroke prevention. Clinical Evidence 2006;15: 52-5.
7. McIntosh AM, Conlon L, Lawrie SM, Stanfield AC. Compliance therapy for schizophrenia. Cochrane Database Syst Rev 2006;(3): CD003442.
8. O'Donnell C, Donohoe G, Sharkey L, Owens N, Migone M, Harries R, et al. Compliance therapy: a randomised controlled trial in schizophrenia. BMJ 2003;327: 834-6.[Abstract/Free Full Text]
9. Cannon M, Jones P. Neuroepidemiology: schizophrenia. J Neurol Neurosurg Psychiatry 1996;61: 604-13.
10. Jablensky A, Sartorius N, Ernberg G, Anker M, Korten A, Cooper JE, et al. Schizophrenia: manifestations, incidence and course in different cultures. A World Health Organization ten-country study. Psychol Med Monogr Suppl 1992;20: 1-97.[Medline]

(Accepted 22 September 2006)

CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

Related Articles

Let’s identify research questions

Andrew Cook
BMJ 2008 337: a1259. [Extract] [Full Text]

How to formulate research recommendations: Format is not enough

Vasiliy Vlassov
BMJ 2006 333: 917. [Extract] [Full Text]

How to formulate research recommendations: The pie or the slice?

Trisha Greenhalgh
BMJ 2006 333: 917. [Extract] [Full Text]

Obviously

Fiona Godlee
BMJ 2006 333: 0. [Extract] [Full Text] [PDF]

Applicability to primary care of national clinical guidelines on blood pressure lowering for people with stroke: cross sectional study

Jonathan Mant, Richard J McManus, and Rachel Hare
BMJ 2006 332: 635-637. [Abstract] [Full Text] [PDF]

Compliance therapy: a randomised controlled trial in schizophrenia

Colin O'Donnell, Gary Donohoe, Louise Sharkey, Nicholas Owens, Maria Migone, Raewynn Harries, Anthony Kinsella, Conall Larkin, and Eadbhard O'Callaghan
BMJ 2003 327: 834. [Abstract] [Full Text] [PDF]

This article has been cited by other articles:

• Cook, A. (2008). Let's identify research questions. BMJ 337: a1259-a1259 [Full text]  
• Chalkidou, K., Walley, T., Culyer, A., Littlejohns, P., Hoy, A. (2008). Evidence-informed evidence-making. J Health Serv Res Policy 13: 167-173 [Abstract] [Full text]  
• Holden, G., Thyer, B. A., Baer, J., Delva, J., Dulmus, C. N., Shanks, T. W. (2008). Suggestions to Improve Social Work Journal Editorial and Peer-Review Processes: The San Antonio Response to the Miami Statement. Research on Social Work Practice 18: 66-71 [Abstract]  
• Nicholson, P J (2007). How to undertake a systematic review in an occupational setting. Occup. Environ. Med. 64: 353-358 [Full text]  
• Croft, A. M (2007). A lesson learnt: the rise and fall of Lariam and Halfan. JRSM 100: 170-174 [Full text]  
• Vlassov, V. (2006). How to formulate research recommendations: format is not enough.. BMJ 333: 917-917 [Full text]  
• Greenhalgh, T. (2006). How to formulate research recommendations: the pie or the slice?. BMJ 333: 917-917 [Full text]  

Rapid Responses:

Read all Rapid Responses

Format is not enough

Vasiliy Vlassov
bmj.com, 15 Oct 2006 [Full text]

The pie or the slice?

Trisha Greenhalgh
bmj.com, 16 Oct 2006 [Full text]

Making research gaps accessible

Philippa F Middleton, et al.
bmj.com, 26 Oct 2006 [Full text]

Authors Response

Polly Brown, et al.
bmj.com, 28 Nov 2006 [Full text]