Large randomised clinical trials are considered to represent the strongest form of evidence in assessing whether a particular healthcare intervention works. However, little attention has been paid to the fact that people treated in large multicentre randomised trials may not accurately reflect the population receiving the drug in real world settings.1

Recently, van Staa and colleagues assessed the external validity of published cost effectiveness studies of selective cyclo-oxygenase-2 (COX 2) inhibitors by comparing the data used in these studies (typically from randomised trials) with observed clinical data.2 The trial data suggested that the cost of avoiding one adverse gastrointestinal event by switching patients from conventional non-steroidal anti-inflammatory drugs to COX 2 inhibitors would be about $20 000 (£12 500; €14 000). However, when the same analysis was performed using the UK’s General Practice Research Database, comprising anonymised medical records of general practitioners, the cost of preventing one bleed was fivefold greater ($104 000).2 The authors concluded that the published cost effectiveness analyses of COX 2 inhibitors neither had external validity nor represented the patients treated in clinical practice. They emphasised that external validity should be an explicit requirement for cost effectiveness analyses that are used to guide treatment policies and practices.