ArticlesAn experimental study of determinants of group judgments in clinical guideline development
Introduction
In many countries, there are clinical guidelines for disseminating good practice in medicine.1, 2 Ideally, guidelines should be based on evidence from large, well conducted studies, but often such research does not exist3 and, where it does, how the results might be applied to particular patients can be unclear.1 Also, guidelines may depend implicitly on interpretation of the literature, on judgments about value and risk,4 on the funding and organisation of health services,5 and, if public funding is involved, on policies about priorities and equity. The synthesis of the research evidence may be rigorous and transparent, but the judgments tend to be opaque.
Formal consensus development methods, often based on the nominal group technique, are widely used because, unlike informal methods such as committees, they offer structured, transparent, and replicable ways of synthesising individual judgments.6 In the UK, NICE and other professional bodies have used modified nominal group techniques, as have at least seven other countries.2, 7, 8
In the modified nominal group technique, participants first express their views independently via a postal questionnaire. They then meet for review and discussion, after which they complete the questionnaire again privately, revising their views if they wish. The practical application of this process has been far from uniform. A systematic review revealed a dearth of research into its workings9 and despite some subsequent studies,10, 11, 12, 13, 14, 15 the key questions posed by the review have not yet been adequately answered. Our aim was to investigate the effect on the judgments produced and on the extent to which there was agreement with research evidence for: (1) three types of factor used to generate clinical scenarios provided in questionnaires—the clinical condition, the treatment, and clinical or social cues; (2) three ways in which nominal groups can differ—provision of a literature review or not, group composition, and background assumptions about the level of health-care resources available.
We also aimed to explore qualitatively the reasons behind the group judgments.
The other research priorities identified by the systematic review were to assess the reliability and representativeness of formal consensus techniques. Results of these investigations will be reported elsewhere.
Section snippets
Methods
Three conditions (chronic back pain, irritable bowel syndrome, and chronic fatigue syndrome) were selected because they fulfilled the following criteria:
(1) there was a mismatch between current clinical practice and research evidence; (2) care was provided by at least two groups of clinicians (general practitioners (GPs) and mental-health professionals); (3) these conditions are important problems; and (4) national guidelines for the conditions had not been published in the UK at the time the
Results
There were 177 participants in the 16 groups, of whom 76% were GPs and 24% mental-health professionals. Mean age was 47 years, most were men (62%) and white (84%).
The relation between ratings for physical and for psychological outcomes for each scenario across the 16 groups was assessed by plotting their medians (figure 2). In view of the close agreement, subsequent analyses used the physical outcome ratings only.
Initial ratings produced by the GP-only nominal groups showed moderate agreement (κ
Discussion
A formal consensus development method produced judgments that were consistent with our assessments of the research evidence in about half the scenarios considered. The extent of concordance varied between the conditions and treatments studied. Concordance was more likely if a literature review was provided and if this evidence supported clinicians' experiences and beliefs. If clinical experience and beliefs were not consistent with research evidence, then the experience and beliefs seemed to
References (29)
- et al.
A consensus process to adapt the World Health Organisation selected practice recommendations for UK use
Contraception
(2003) - et al.
Understanding variability in physician ratings of the appropriateness of coronary angiography after acute myocardial infarction
J Clin Epidemiol
(1999) - et al.
Effect of panel composition on physician ratings of appropriateness of abdominal aortic aneurysm surgery: elucidating differences between multi-specialty panel results and specialty society recommendations
Health Policy
(1997) - et al.
Potential benefits, limitations and harms of clinical guidelines
BMJ
(1999) - et al.
Towards evidence-based clinical practice: an international survey of 18 clinical guideline programmes
Int J Qual Health Care
(2003) How do we decide whether an investigation or procedure is appropriate?
- et al.
Innovation in healthcare: how does credible evidence influence professionals?
Health Soc Care Community
(2003) Clinical guidelines: using clinical guidelines to improve patient care in the NHS
(1996)- et al.
A group process model for problem identification and programme planning
J Appl Behav Sci
(1971) Preoperative tests: the use of routine preoperative tests for elective surgery
(2003)
Consensus development methods, and their use in clinical guideline development
Health Technol Assessment
Development of appropriateness criteria for colonoscopy: comparison between a standardized expert panel and an evidence-based medicine approach
Int J Qual Health Care
Consensus among experts and research synthesis: a comparison of methods
Int J Technol Assess Health Care
Effect of specialty and nationality on panel judgement of the appropriateness of coronary revascularisation: a pilot study
Med Care
Cited by (127)
Guideline panel social dynamics influence the development of clinical practice recommendations: a mixed-methods systematic review
2024, Journal of Clinical EpidemiologyGerman-language translation of the PANELVIEW instrument to evaluate the guideline development process from the perspective of the guideline group
2022, Zeitschrift fur Evidenz, Fortbildung und Qualitat im GesundheitswesenThe injustice of unfit clinical practice guidelines in low-resource realities
2021, The Lancet Global HealthCitation Excerpt :When comparing multiple international, high-quality maternal health CPGs, important inconsistencies and disagreements are seen, even between concurrently published so-called high-quality CPGs.16 These inconsistencies and disagreements highlight limitations in experimental studies and the strong influence of values, culture, and professional tradition, even in what are considered high-quality CPGs.17 Translating evidence into recommendations inevitably requires judgements about the balance between benefits and risks, and involves combining selective and restricted study findings.
Developing standards of care for obese women during pregnancy
2020, Obesity and ObstetricsA Systematic Critical Appraisal of Evidence-Based Clinical Practice Guidelines for the Rehabilitation of Children With Moderate or Severe Acquired Brain Injury
2019, Archives of Physical Medicine and Rehabilitation