Rating the quality of medical websites may be impossible

“There's a lot of harmful medical information on the internet. Something needs to be done.” The obvious solution is to provide users with some sort of quality rating, guaranteed by a trusted third party. 1 2 Bodies as diverse as the European Union and NHS Direct Online are attracted to this solution to their problems of protecting the public health. But implementing such a solution is likely to be difficult—if not impossible.

Experience to date has not been encouraging. Three years ago Jadad and Gagliardi counted 47 instruments for rating websites, none of them apparently tested for reliability or validity. They wondered “whether they should exist in the first place, whether they measure what they claim to measure, or whether they lead to more good than harm.”3 After the appearance of a further 15 unvalidated instruments they stopped counting (A Jadad, personal communication).

Why is it so difficult to get beyond the good ideas phase? Take the quality criteria most frequently mentioned: accuracy, …