Accuracy of information on apparently credible websites: survey of five common health topics
BMJ 2002; 324 doi: https://doi.org/10.1136/bmj.324.7337.581 (Published 09 March 2002) Cite this as: BMJ 2002;324:581- Heinke Kunst, specialist registrara,
- Diederik Groot, medical studentb,
- Pallavi M Latthe, specialist registrarc,
- Manish Latthe, general practitionerd,
- Khalid S Khan (k.s.khan{at}bham.ac.uk), consultant, education resource centrec
- a Department of Respiratory Medicine, St Mary's Hospital, London W2 1NY
- b Maastricht University, 6200 MD Maastricht, Netherlands
- c Birmingham Women's Health Care NHS Trust, Birmingham B15 2TG
- d Tower Hill Medical Centre, Birmingham B42 1LJ
- Correspondence to: K S Khan
- Accepted 4 February 2002
The internet provides an easily accessible forum to disseminate both accurate and inaccurate health information—so it has the potential to facilitate but also to jeopardise healthcare provision. 1 2 Many criteria have been alleged to capture the quality of health websites, 3 4 but the validity of these criteria needs to be examined.5 The source, currency, and hierarchy of the evidence posted on a website may be used to judge its credibility—that is, the power of inspiring belief. If these criteria were fulfilled, the contents of the website would be expected to be accurate. We determined whether websites that seem to be credible provide accurate health information.
Methods and results
We determined the relation between credibility features and accuracy of contents of 121 websites that provided information on five common health topics: chronic obstructive pulmonary disease (23 sites), ankle sprain (36), emergency contraception (32), menorrhagia (9), and female sterilisation (21). These sites were identified either by searching each of the most commonly used engines (such as Altavista, Excite, Hotbot, Infoseek, Lycos, Northern Light, Webcrawler) or by simultaneously consulting them using a meta-search engine, Copernic 4.1 (www.copernic.com/). We selected English language websites whose content provided information about the topics. Website selection and data extractions were performed in duplicate, and agreement between the two assessors was high.
The entire contents of the selected websites were assessed for three credibility features (source, currency, and evidence hierarchy) and accuracy of contents. Source and currency are widely used to assess scientific credibility of a website. 3 4 The source of medical information is usually regarded as the main criterion for its credibility; sites should display the source of the information clearly. Currency is shown by websites that display the date of the original document or content posting on the internet, and that of any updates. We looked at the hierarchy of evidence posted on each website, examining whether the levels assigned to various pieces of information were related to their validity or methodological quality. This allows users to assess the strength of the recommendations being made. Our assessment showed that 113/121 (93%) websites described source, 59 (49%) currency, and 22 (18%) evidence hierarchy.
Accuracy of website contents was judged against rigorously developed, peer reviewed, and published guidelines for each of the five health topics (see table on bmj.com). The data on accuracy were extracted as a proportion of guideline statements included in the website's contents and they were converted into three accuracy levels. In level I, more than two thirds of guideline statements were covered (28/121 (24%) websites); in level II, one third to two thirds of guideline statements were covered (43 (35%) websites); in level III, less than one third of guideline statements were covered (50 (41%) websites).
We examined the relation between features of website credibility and level of accuracy of contents by cross tabulation and assessed the strength of association with Kendall's rank correlation, which adjusts for tied ranks in the data. The value of the coefficient (tau b) ranges from −1 to 1. Interpretation is subjective, but values near zero may be taken to indicate no correlation whereas values near 1 indicate a strong correlation. As shown in the table, websites with description of credibility features tended to have higher levels of accuracy of contents, but this relationship was not strong.
Comment
Our study shows that features of website credibility—source, currency, and evidence hierarchy—have only slight or at best moderate correlation with accuracy of information in five common health topics. Thus, apparently credible websites may not necessarily provide higher levels of accurate health information.
Acknowledgments
We thank Julie Morris for statistical input.
KSK conceived and designed the study. HK, DG, PML, ML, and KSK searched, selected, and assessed websites. KSK and DG analysed and interpreted the data and wrote the manuscript. HK, PML, and ML revised the manuscript. KSK is guarantor for the paper.
Footnotes
-
Funding None.
-
Competing interests None declared.
-
A table of sources of websites appears on bmj.com