BMJ 2002;324:569-573 ( 9 March )

Papers

Examination of instruments used to rate quality of health information on the internet: chronicle of a voyage with an unclear destination

Anna Gagliardi, senior research associate, ^a Alejandro R Jadad, professor. ^b

^a See Education and debate p 606 Graduate Department of Health Policy, Management and Evaluation, Faculty of Medicine, University of Toronto, Toronto, ON, Canada, ^b Departments of Health Policy, Management and Evaluation, and Anaesthesia, University Health Network, University of Toronto, Toronto, ON, Canada

Correspondence to: A R Jadad, Director, Centre for Global eHealth Innovation, University Health Network, Toronto General Hospital, Fraser Elliott Building, 4th Floor, 190 Elizabeth Street, Toronto, ON M5G 2C4, Canada ajadad{at}uhnres.utoronto.ca

Objective: This study updates work published in 1998, which found that of 47 rating instruments appearing on websites offering health information, 14 described how they were developed, five provided instructions for use, and none reported the interobserver reliability and construct validity of the measurements.
Design: All rating instrument sites noted in the original study were visited to ascertain whether they were still operating. New rating instruments were identified by duplicating and enhancing the comprehensive search of the internet and the medical and information science literature used in the previous study. Eligible instruments were evaluated as in the original study.
Results: 98 instruments used to assess the quality of websites in the past five years were identified. Many of the rating instruments identified in the original study were no longer available. Of 51 newly identified rating instruments, only five provided some information by which they could be evaluated. As with the six sites identified in the original study that remained available, none of these five instruments seemed to have been validated.
Conclusions: Many incompletely developed rating instruments continue to appear on websites providing health information, even when the organisations that gave rise to those instruments no longer exist. Many researchers, organisations, and website developers are exploring alternative ways of helping people to find and use high quality information available on the internet. Whether they are needed or sustainable and whether they make a difference remain to be shown.

What is already known on this topic The rapid growth of healthcare websites in the 1990s was accompanied by initiatives to rate their quality, including award-like symbols on websites A systematic review of the reliability and validity of such rating instruments, published in 1998, showed that they were incompletely developed What this study adds Few of the rating instruments identified in 1998 remain functional; 51 new instruments were identified Of the 51 newly identified instruments, 11 were not functional, 35 were available but provided no information, and five provided information but were not validated Many researchers, organisations, and website developers are exploring alternative ways of helping people to find high quality information on the internet

---

© BMJ 2002
CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    StumbleUpon    Technorati    What's this?

Relevant Articles

Developing a quality criteria framework for patient decision aids: online international Delphi consensus process

Glyn Elwyn, Annette O'Connor, Dawn Stacey, Robert Volk, Adrian Edwards, Angela Coulter, Richard Thomson, Alexandra Barratt, Michael Barry, Steven Bernstein, Phyllis Butow, Aileen Clarke, Vikki Entwistle, Deb Feldman-Stewart, Margaret Holmes-Rovner, Hilary Llewellyn-Thomas, Nora Moumjid, Al Mulley, Cornelia Ruland, Karen Sepucha, Alan Sykes, Tim Whelan The International Patient Decision Aids Standards (IPDAS) Collaboration
BMJ 2006 333: 417. [Abstract] [Full Text] [PDF]

Are virtual communities good for our health?

Alejandro R Jadad, Murray W Enkin, Sholom Glouberman, Philip Groff, and Anita Stern
BMJ 2006 332: 925-926. [Extract] [Full Text] [PDF]

Is a consultation needed?

Frank Sullivan and Jeremy C Wyatt
BMJ 2005 331: 625-627. [Extract] [Full Text] [PDF]

Unvalidated tools to rate web health information are proliferating
BMJ 2002 324: 0. [Full Text] [PDF]

The invention of talk
BMJ 2002 324: 0. [Full Text] [PDF]

The quality of health information on the internet

Gretchen P Purcell, Petra Wilson, and Tony Delamothe
BMJ 2002 324: 557-558. [Extract] [Full Text] [PDF]

This article has been cited by other articles:

• DESHPANDE, A., JADAD, A. R. (2009). Trying to Measure the Quality of Health Information on the Internet: Is It Time to Move On?. The Journal of Rheumatology 36: 1-3 [Full text]  
• Adams, S., Bal, R. (2009). Practicing Reliability: Reconstructing Traditional Boundaries in the Gray Areas of Health Information Review on the Web. Science Technology Human Values 34: 34-54 [Abstract]  
• Clauson, K. A, Polen, H. H, Boulos, M. N K., Dzenowagis, J. H (2008). Scope, Completeness, and Accuracy of Drug Information in Wikipedia. The Annals of Pharmacotherapy 42: 1814-1821 [Abstract] [Full text]  
• Bath, P. A. (2008). Health informatics: current issues and challenges. Journal of Information Science 34: 501-518 [Abstract]  
• Akram, G, Thomson, A H, Boyter, A C, Morton, M J S (2008). Characterisation and evaluation of UK websites on attention deficit hyperactivity disorder. Arch. Dis. Child. 93: 695-700 [Abstract] [Full text]  
• Keselman, A., Logan, R., Smith, C. A., Leroy, G., Zeng-Treitler, Q. (2008). Developing Informatics Tools and Strategies for Consumer-centered Health Communication. J. Am. Med. Inform. Assoc. 15: 473-483 [Abstract] [Full text]  
• Ferreira-Lay, P., Miller, S. (2008). The quality of internet information on depression for lay people. Psychiatr. Bull. 32: 170-173 [Abstract] [Full text]  
• Khechine, H., Pascot, D., Premont, P. (2008). Use of health-related information from the Internet by English-speaking patients. Health Informatics Journal 14: 17-28 [Abstract]  
• Datta, A. K, Selman, T. J, Kwok, T., Tang, T., Khan, K. S (2008). Quality of information accompanying on-line marketing of home diagnostic tests. JRSM 101: 34-38 [Abstract] [Full text]  
• Porter, A., Edirippulige, S. (2007). Parents of Deaf Children Seeking Hearing Loss-Related Information on the Internet: the Australian Experience. J Deaf Stud Deaf Educ 12: 518-529 [Abstract] [Full text]  
• Elwyn, G., O'Connor, A., Stacey, D., Volk, R., Edwards, A., Coulter, A., Thomson, R., Barratt, A., Barry, M., Bernstein, S., Butow, P., Clarke, A., Entwistle, V., Feldman-Stewart, D., Holmes-Rovner, M., Llewellyn-Thomas, H., Moumjid, N., Mulley, A., Ruland, C., Sepucha, K., Sykes, A., Whelan, T., The International Patient Decision Aids Standards, (2006). Developing a quality criteria framework for patient decision aids: online international Delphi consensus process. BMJ 333: 417- [Abstract] [Full text]  
• Jadad, A. R, Enkin, M. W, Glouberman, S., Groff, P., Stern, A. (2006). Are virtual communities good for our health?. BMJ 332: 925-926 [Full text]  
• Hargrave, D. R., Hargrave, U. A., Bouffet, E. (2006). Quality of health information on the Internet in pediatric neuro-oncology. Neuro Oncol Duke 8: 175-182 [Abstract] [Full text]  
• Schwartz, K. L., Roe, T., Northrup, J., Meza, J., Seifeldin, R., Neale, A. V. (2006). Family Medicine Patients' Use of the Internet for Health Information: A MetroNet Study. J Am Board Fam Med 19: 39-45 [Abstract] [Full text]  
• Ritterband, L. M., Borowitz, S., Cox, D. J., Kovatchev, B., Walker, L. S., Lucas, V., Sutphen, J. (2005). Using the Internet to Provide Information Prescriptions. Pediatrics 116: e643-e647 [Abstract] [Full text]  
• Sullivan, F., Wyatt, J. C (2005). Is a consultation needed?. BMJ 331: 625-627 [Full text]  
• Broom, A. (2005). Medical specialists' accounts of the impact of the Internet on the doctor/patient relationship. Health (London) 9: 319-338 [Abstract]  
• Ansani, N. T., Vogt, M., Henderson, B. A. F., Mckaveney, T. P., Weber, R. J., Smith, R. B., Burda, M., Kwoh, C. K., Osial, T. A., Starz, T. (2005). Quality of arthritis information on the Internet. Am J Health Syst Pharm 62: 1184-1189 [Abstract] [Full text]  
• Purvis, J. M., Alexander, A. H., Einhorn, T. A., Griffin, L. Y. (2005). Symposium Evaluating the Flood of Orthopaedic Media and Marketing Information. JBJS 87: 1392-1398 [Full text]  
• McCray, A. T. (2005). Promoting Health Literacy. J. Am. Med. Inform. Assoc. 12: 152-163 [Abstract] [Full text]  
• SPALLEK, H. (2004). A resource guide for practice development through technology. Journal of the American Dental Association 135: 38S-44S [Full text]  
• Charnock, D., Shepperd, S. (2004). Learning to DISCERN online: applying an appraisal tool to health websites in a workshop setting. Health Educ Res 19: 440-446 [Abstract] [Full text]  
• Anselmo, M. A., Lash, K. M., Stieb, E. S., Haver, K. E. (2004). Cystic Fibrosis on the Internet: A Survey of Site Adherence to AMA Guidelines. Pediatrics 114: 100-103 [Abstract] [Full text]  
• Kim, S., Mouradian, W. E., Leggott, P. J., Schaad, D. C., Shaul, C. (2004). Implications for Designing Online Oral Health Resources: A Review of Fifty-Six Websites. J Dent Educ 68: 633-643 [Abstract] [Full text]  
• Theodosiou, L., Green, J. (2003). Emerging challenges in using health information from the internet. Adv. Psychiatr. Treat. 9: 387-396 [Abstract] [Full text]  
• Bath, P. A., Bouchier, H. (2003). Development and Application of a Tool Designed to Evaluate Web Sites Providing Information on Alzheimer's Disease. Journal of Information Science 29: 279-297 [Abstract]  
• Nettleton, S., Burrows, R. (2003). E-Scaped Medicine? Information, Reflexivity and Health. Critical Social Policy 23: 165-185 [Abstract]  
• Evans, R, Edwards, A, Elwyn, G (2003). The future for primary care: increased choice for patients. Qual Saf Health Care 12: 83-84 [Full text]  
• Helft, P. R., Hlubocky, F., Daugherty, C. K. (2003). American Oncologists' Views of Internet Use by Cancer Patients: A Mail Survey of American Society of Clinical Oncology Members. JCO 21: 942-947 [Abstract] [Full text]  
• Al-Shahi, R, Sadler, M, Rees, G, Bateman, D (2002). The internet. J. Neurol. Neurosurg. Psychiatry 73: 619-628 [Abstract] [Full text]  
• Purcell, G. P, Wilson, P., Delamothe, T. (2002). The quality of health information on the internet. BMJ 324: 557-558 [Full text]  
• Wilson, P., Risk, A. (2002). How to find the good and avoid the bad or ugly: a short guide to tools for rating quality of health information on the internet * Commentary: On the way to quality. BMJ 324: 598-602 [Full text]  
• Bland, J M., Altman, D. G (2002). Statistics Notes: Validating scales and indexes. BMJ 324: 606-607 [Full text]  

Rapid Responses:

Read all Rapid Responses

Evidence-based Medical Practice: A Canadian Website Directory and Instrument to rate quality.

France Légaré , Md, MSc, (F)CCMF, et al.
bmj.com, 9 Mar 2002 [Full text]

Overrated Perception of the Internet by Medical Students of a Developing Country

Pedro Horna, et al.
bmj.com, 12 Mar 2002 [Full text]

tools for scoring miscarraige websites

j christopher r. hardwick, et al.
bmj.com, 18 Mar 2002 [Full text]

Online poll

Find out more >>