External validity of clinical research – a widespread problem!
The editorial by MET McMurdo et al. calls attention to the chasm
between patients in the real world and those participating in clinical
trials, and how this greatly disadvantages older people.1 Transferral of
research results between sexes raises similar problems, only 3% of
research protocols are designed to show sex differences.2
But the problem extends far beyond age and sex. Evidence based
knowledge often relies on research in subsets of patients who fulfil
certain well defined but often less known and applicable criteria.
Clinical research in patients with functional gastrointestinal disorders
(FGID) is, with few exceptions, based on internationally accepted
diagnostic criteria (the Rome II criteria). But most patients with FGID
are treated by general practitioners who are unaware of diagnostic
criteria and use more intuitive and pragmatic diagnostic decision
procedures. Consequently, the agreement between diagnoses given by the GPs
and those based on the Rome II criteria is nearly like tossing a coin
(kappa statistics values are low and close to zero).3;4 In addition to an
unreliable diagnosis, comorbidity, which is a confounder with significant
differences between groups of patients with FGID, is seldom accounted for
in clinical research. The results from clinical research in patients with
FGID are therefore unreliable in general practice. This is likely to be a
widespread problem. Greater consideration to external validity is to be
recommended in the design, reporting and reading of clinical research.5
1. McMurdo ME, Witham MD, Gillespie ND. Including older people in
clinical research. BMJ 2005;331:1036-7.
2. Roehr B. NIH research funding does not recognise importance of sex
differences. BMJ 2005;330:1170.
3. Vandvik PO, Aabakken L, Farup PG. Diagnosing irritable bowel
syndrome: poor agreement between general practitioners and the Rome II
criteria. Scand J Gastroenterol 2004;39:448-53.
4. Farup PG, Vandvik PO, Aabakken L. How useful are the Rome II
criteria for identification of upper gastrointestinal disorders in general
practice? Scand J Gastroenterol 2005;40:1284-9.
5. Rothwell PM. External validity of randomised controlled trials:
"to whom do the results of this trial apply?" Lancet 2005;365:82-93.
Competing interests: No competing interests