Follow up of quality of public oriented health information on the world wide web: systematic re-evaluationBMJ 2002; 324 doi: http://dx.doi.org/10.1136/bmj.324.7337.582 (Published 09 March 2002) Cite this as: BMJ 2002;324:582
- Chiara Pandolfini (), senior research fellow,
- Maurizio Bonati, head
- Laboratory for Mother and Child Health, Istituto di Ricerche Farmacologiche “Mario Negri”, 20157 Milan, Italy
- Correspondence to: C Pandolfini
- Accepted 24 January 2002
In 1997 one of the first studies to evaluate the quality of health information on the internet was published.1 This article assessed the reliability of information for managing fever in children at home and found that the quality of information was poor. Four years after publication these findings were mentioned in 78 journals (from Journal Citation Report), and the message should therefore have reached a wide audience. We investigated the effects of the earlier findings by re-evaluating the quality of the original web pages four years later, as well as that of a more recent sample of pages, using the same methods.
Methods and results
On 28 June 2001 we searched articles through the Institute for Scientific Information's citation index for references to the earlier study.1 We searched for the 41 web pages evaluated in the original study to see if they still existed and if they did whether they had been substituted with new pages or their content had been modified. We compared the content with copies of the original pages, and we noted changes and assigned scores by using the guidelines and scoring system applied in the original study.1 We then repeated the 1997 search for new pages, found 40, and scored them as well.
Overall, 19 of the 41 (46%) original pages still existed. Of these, two had additional information leading, in one case, to a one point increase in completeness score. Five pages had been substituted with new content, leading to the score being increased in two of them (3 to 4 and 2 to 4 points). Three pages scored the same as the original pages (2, 3, and 5 points). Only one of the four pages that had adhered completely to the guidelines (5 points) remained. Thus, the overall improvement in the pages' scores was only 4 points and was not significant with the Wilcoxon signed rank test.
To compare completeness scores, we made the old and new pages more homogeneous by considering pages in English only (table). The average score of the 38 original pages was 2.6 (SD 1.19), whereas that of the 40 new web pages was 3.4 (SD 0.81). The difference was significant (P<0.001) when measured with the Mann-Whitney U test.
Only two (5%) of the new pages adhered to the guidelines for quality of content compared with three (8%) of the original pages. However, 18 (45%) of the new pages compared with only three (8%) of the original pages adhered to most of the guidelines, except for one indication. All of the new pages concurred with paracetamol, dressing the child lightly, giving extra fluids, and when to call the doctor but 13 failed to specify that an antipyretic should be given before a sponge bath to lower the hypothalamic set point. Furthermore, none of the new pages suggested aspirin for children, whereas two of the original pages had (despite possible severe consequences).
The quality of health information on the internet has improved over the past few years despite concerns over poor quality and its possible consequences.2 The importance of low quality or incomplete information has led researchers to evaluate web pages in different disciplines and to define lists of requirements for users to evaluate the quality of web pages.3–5 Despite these apparently unproductive attempts to amend the situation, it has improved. Monitoring health information on the internet for accuracy, completeness, and consistency is still fundamental.
Contributors: CP was primarily responsible for data entry, management, and analysis; she will act as guarantor for the paper. MB guided the analyses. CP and MB contributed equally to writing the manuscript.
Competing interests None declared.