Intended for healthcare professionals

Education And Debate

North of England evidence based guidelines development project: methods of guideline development

BMJ 1996; 312 doi: (Published 23 March 1996) Cite this as: BMJ 1996;312:760
  1. Martin Eccles, clinical senior lecturera,
  2. Zoe Clapp, junior research associatea,
  3. Jeremy Grimshaw, programme directorb,
  4. Philip C Adams, cardiologistc,
  5. Bernard Higgins, chest physiciand,
  6. Ian Purves, directore,
  7. Ian Russell, head of departmentf
  1. a Centre for Health Services Research, University of Newcastle upon Tyne, Newcastle upon Tyne NE2 4AA
  2. b Health Services Research Unit, University of Aberdeen, Aberdeen AB9 2ZD
  3. c Royal Victoria Infirmary, Newcastle upon Tyne NE1 4LP
  4. d Freeman Hospital, Newcastle upon Tyne NE7 7DN
  5. e Sowerby Unit for Primary Care Informatics, Department of Primary Health Care, University of Newcastle upon Tyne, Newcastle upon Tyne NE2 4AA
  6. f Department of Health Sciences and Clinical Evaluation, University of York, York YO1 5DD
  1. Correspondence to: Dr Eccles.
  • Accepted 9 December 1995

There is increasing interest in clinical guidelines in Britain. With this interest has come increasing awareness of the methodological issues in the development of valid guidelines.1 2 3 Practice guidelines are considered valid if “when followed, they lead to the health gains and costs predicted for them.”1 When appropriately disseminated and implemented, valid guidelines can lead to changes in clinical practice and improvements in patient outcome.4 5 6 7 Conversely, the dissemination and implementation of invalid guidelines may lead to wasteful use of resources on ineffective interventions or, worse, deterioration in patients' health.

Validity has been related to three principal factors in guideline development—namely, the composition of the guideline development panel and its processes; the identification and synthesis of evidence; and the method of guideline construction.5 Though these factors have been discussed at theoretical5 and more practical levels,8 there have been few attempts to put them into practice in Britain. In this series of three papers we describe the methods used to develop evidence based guidelines for the primary care management of two common chronic conditions—namely, asthma in adults and stable angina—and summary versions of the two guidelines that resulted.9 10

Guideline development groups

The guideline development groups were composed of relevant health care professionals and patients; a specialist resource (a consultant chest physician for asthma and a consultant cardiologist for stable angina) and an experienced small group leader; and members of the research team. All group members were offered reimbursement of travelling expenses, and general practitioners and practice nurses were offered reimbursement of any locum expenses.

Evidence review and synthesis SEARCH STRATEGY

The search was carried out with Medline and covered the 10 years 1985-94. This was a pragmatic decision influenced by the volume of papers and the time and resources available. All searches were confined to studies of human adults written in English. For both topics we conducted medical subject heading and free text searches using the terms meta-analysis, randomised controlled trial, review, cohort study, and case-control study. For asthma we also sought the terms asthma, peak expiratory flow rate, obstructive lung disease, forced expiratory volume, and paroxysmal dyspnoea; for stable angina we sought the terms coronary disease and angina pectoris. Additional specific Medline searches were carried out by using the following terms: decision making, theophylline, terbutaline, antihistamine, isosorbide, myocardial infarction plus secondary prevention, and buccal.

The BIDS (Bath Information and Data Services, Institute for Scientific Information, University of Bath) electronic database was also searched (by using the terms “asthma + management” and “angina + management”). In addition, references were identified from two other sources. Firstly, if there was no recent evidence in a clinically important topic the specialist resource was asked to identify from personal knowledge key articles published before 1985. Secondly, the reference lists of non-systematic reviews were checked. We did not attempt to access the grey literature, nor did we identify letters in response to original articles.


The sets of references generated by the searches were sifted for relevance to the clinical topic of the guidelines. The initial sifting was done by a clinically qualified health services researcher (ME) on the basis of article titles. Irrelevant articles were removed. If there was doubt about the relevance of an article it was retained for the next round of sifting, which identified references needing detailed assessment. This was performed by the specialist resource on the basis of title and abstract (when these were available). The identified papers were reviewed against explicit methodological criteria informed by several sources.11 12 13 14 15 16 17 18 Owing to the volume of literature, this task was extended beyond the single specialist resource in each group by inviting group members to review discrete areas on behalf of the group. Reviewers were provided with explicit written instructions to ensure consistency. Some studies which passed methodological sifting still had flaws—for example, negative studies without a power calculation. This was made explicit within the description of the study or within accompanying comments.


Once individual papers had been checked for methodological rigour and clinical relevance they were categorised according to study design. The three categories shown in box 1 were adapted from those of the Canadian Task Force.19 In the early meetings tables summarising the evidence were circulated to group members beforehand. Initially the evidence was discussed and synthesised into recommendations by the group within meetings. In later meetings this process was conducted by the reviewer and the research team outside group meetings and provisional recommendations were made. These were circulated for comment as a postal exercise to ensure time for finalising recommendations within the meetings.

Box 1—Categories of evidence

  • (I) Based on well designed randomised controlled trials, meta-analyses, or systematic reviews

  • (II) Based on well designed cohort or case-control studies

  • (III) Based on uncontrolled studies or consensus

The evidence was synthesised by qualitative methods. These entailed summarising the content of identified papers into brief statements that the group thought accurately reflected relevant evidence. Quantitative (meta-analysis) techniques were not used, as we were dealing with studies other than randomised controlled trials. Recommendations were derived by informal consensus methods. Though interpreting evidence inevitably involves value judgments, by making this process explicit we made the scientific basis of these judgments as clear as possible. Box 2 shows the relation between strength of recommendation and category of evidence. The strength of recommendation was shown as A, B, or C after each recommendation.

Box 2—Strength of recommendation

  • (A) Directly based on category I evidence

  • (B) Directly based on category II evidence or extrapolated recommendation from category I evidence

  • (C) Directly based on category III evidence or extrapolated recommendation from category I or II evidence


Consensus methods were used to develop recommendations for topics without evidence. The asthma group agreed to refer to the consensus recommendations of British Thoracic Society asthma guidelines.20 In the absence of an equivalent document the angina group drew up its own consensus recommendations.

External review of guidelines

External reviewers were chosen to reflect three groups: potential users of the guidelines; content topic experts; and guideline methodologists. Though their comments influenced the style and content of the guidelines, these remained the responsibility of the development group.

Scheduled review of guidelines

The guidelines should be reviewed for their content and evidence base no later than three years after completion.


  • Implications

  • The guidelines were developed by using methods to maximise their validity: identification of evidence by systematic review, development by a multidisciplinary group, and the use of explicit links between evidence and recommendations

  • The explicit nature of the guideline development process allows potential users to critically appraise the validity of the guidelines and make an informed judgment about whether to adopt them in their clinical practice

  • The guidelines propose principles of good practice; these could be modified locally, based on the evidence and strength of recommendation and taking into account local preferences and resources

We thank Liz Wood for administrative support. The views expressed are ours and not necessarily those of the funding bodies.


  • Funding The north of England evidence based guidelines development project was funded by the research and development directorate of the former Northern Regional Health Authority, now the Northern and Yorkshire Regional Health Authority. The health services research unit is funded by the chief scientist's office of the Scottish Office Home and Health Department.

  • Conflict of interest PCA frequently receives fees from pharmaceutical companies for lecturing on coronary artery disease.


  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
  8. 8.
  9. 9.
  10. 10.
  11. 11.
  12. 12.
  13. 13.
  14. 14.
  15. 15.
  16. 16.
  17. 17.
  18. 18.
  19. 19.
  20. 20.