Breast cancer on the world wide web: cross sectional survey of quality of information and popularity of websites
BMJ 2002; 324 doi: https://doi.org/10.1136/bmj.324.7337.577 (Published 09 March 2002) Cite this as: BMJ 2002;324:577
All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
Dr Masters suggests that students were unable to find information about a
"child with rheumatic heart disease in South Africa"
from the Google Search engine.
Click
here to perform a search of google for "child with rheumatic heart disease
in South Africa".
I fail to see any "significant favoring of a few, mainly US sites",
unless PubMed is regarded as a "US site"?
The Internet is indeed presenting major problems for traditional medical
publishing. Where one might have, in the past,
been able to cite an obscure reference with little expectation that anybody
would actually go and try to locate it,
the Internet researcher expects such key citations to be supplied as links,
and to be verifiable and accurate.
Competing interests:
None declared
Competing interests: No competing interests
The value of popularity in the results in the paper by Merc, Bernsram and Mirrza et al ([1]), is indicative of deeper mechanisms
and broader problems of search engine results on medical and other sites.
Although Google indeed measures the number of links to a page, it is not simply the number of direct links that determine the value. Google's PageRank system applies a weighting factor to each page, and that weighting factor
is carried over to any further pages
[2,3]. This means that, in order to measure "popularity" based on links, not only should the number of linking pages be counted; for each of those linking pages, their popularity is also
determined by the number of linking pages, and should also be counted.
Further, recent studies into the structure of the Internet reveal it as a scale-free network, defined loosely as a network in which a large number of nodes have a minimum number of connections, and a small number of nodes
have a large number of connections (or links)
([4,5,6]). The importance of this is that not only does Google read this scale-free structure,
but actually contributes to it by increasing access to the very "popular" sites [7]. (The fact that other search engines are also used, does not detract from this, as, in many cases, these are powered
by Google).
In a recent study we asked South African undergraduate medical students to search for sites related to a case of a child with rheumatic heart disease in South Africa. The overwhelming use of Google and Google-powered search
engines led to a significant favouring of a few, mainly US sites, with very little reference to local sites [7].
This has several implications, for new sites and for sites outside the US. An added problem, morevoer, is verifiability. When a researcher finds a "strange" fact in a journal, a common method of verification is to look
for other supporting literature. Because the Internet has limited enforceable controls regarding copying, material has often been copied from one site to another by web page authors who have not identified errors. The very fact
of copying, however, serves to duplicate the error. The unsuspecting researcher, finding these "supporting" references, perceives the error to be validated.
This is of grave concern to medical practitoners and educators whose students use the Internet for research.
- Meric F, Bernstam EV, Mirza NQ et al. Breast cancer on the world wide web: cross sectional survey of quality of information and popularity of websites. BMJ 2002; 324: 577-81.
- Craven, P. Google's PageRank Explained. WebWorkshop. 2003. Online:
http://webworkshop.net/pagerank.html,
29 May 2003. - Ridings, C and Shishigin, M. PageRank Uncovered. 2002. Online:
http://www.supportforums.org/PageRank.pdf,
29 May 2003. - Barabási, A-L, Albert, R and Jeong, H. Scale-free characteristics of random networks: the topology of the world-wide web. Physica A. 2000;281:69-77.
- Barabási, A-L, Dezsö, Z, Ravasz, E, Yook, S-H and Oltvai, Z. Scale-free and hierarchical structures in complex networks. Unpublished, Appearing: Sitges Proceedings on Complex Networks. 2004.
- Barabási,A-L and Bonabeau, E. Scale-Free Networks. Scientific American. 2003; 288, 5: 50-59.
- Masters, KA, Vivian, LM, Davies, M-A. What Students Find: An analysis and implications of web sites found by medical students researching a topic. Proceedings of the International Conference on Education and Information Systems: Technologies and Applications. Orlando, Florida. 31 July - 2 August 2003.(Eds. F.Malpica, A. Tremante & N. Sala): 386-391.
Competing interests:
None declared
Competing interests: No competing interests
EDITOR - Only two articles in your Internet theme issue (9th March)
mentioned the prevalence of Internet access and website use to search for
medical information amongst “patients” (or “medical endusers”), and then
only in passing.1,2 Although the quality of medical information available
on the web is crucial, as emphasized in the theme issue, its direct
relevance to the practice of medicine depends on how many people use it
and whether they access the correct information. Data from the USA
suggest 40-54% of patients,1 or 100 million adults regularly go on line
for information about health care.2 Are these figures reflective of the
situation in the UK?
I have informally surveyed consecutive new outpatients in general
neurology clinics at two district general hospitals in north-west England
(January to March 2001;3 and January to March 2002); and patients and
their families and/or carers attending a cognitive function clinic at a
regional neurosciences centre (October 2001 to March 2002). Questions
were asked about Internet access and use (very rarely was this information
spontaneously volunteered).3 The results (Table) suggest fewer UK
patients and/or families access website medical information than in the
USA. Inappropriate self-diagnosis, a legitimate concern in any form of
self-directed information-seeking behaviour, occurred as a consequence of
21% to 33% of Internet searches in the general neurology clinic
populations. When asked the hypothetical question whether a website with
relevant information would be accessed if this suggestion came directly
from the clinic doctor, 82% of patients and families/carers attending the
cognitive function clinic answered in the affirmative, a figure comparable
with that from an American genetic disorders clinic (92%).4
Asking patients whether they have accessed the Internet, what they
have discovered, and being ready to suggest appropriate websites, may be
an increasingly important component of the clinical consultation.
TABLE
Clinic General General Cognitive
(Jan-March 2001)(Jan-March 2002) (Oct 2001-March 2002)
n 198 211 104
Internet access:
33% 38% 52%
Internet use:
those with access
22% 26% 52%
all patients
7.6% 10% 27%
Incorrect self-diagnosis:
33% 21% -
AJ Larner
onsultant neurologist
Walton Centre for Neurology and Neurosurgery, Liverpool, L9 7LJ
1. Meric F, Bernstam EV, Mirza NQ et al. Breast cancer on the world
wide web: cross sectional survey of quality of information and popularity
of websites. BMJ 2002; 324: 577-81 (9th March).
2. Wilson P. How to find the good and avoid the bad or ugly: a short
guide to tools for rating quality of health information on the internet.
BMJ 2002; 324: 598-602 (9th March).
3. Larner AJ. Use of Internet medical websites and of NHS Direct by
neurology outpatients prior to consultation. Int J Clin Pract 2002; 56:
(in press).
4. Taylor MRG, Alman A, Manchester DK. Use of the Internet by patients
and their families to obtain genetics-related information. Mayo Clin Proc
2001; 76: 772-6.
Competing interests: TABLEClinic General General Cognitive (Jan-March 2001)(Jan-March 2002) (Oct 2001-March 2002) n 198 211 104Internet access: 33% 38% 52%
EDITOR- Health on the Net is a non-profit foundation with an eight
point code of conduct for sites providing health Information (Honcode)(1).
We agree with Meric F. and colleges(2) and with, Shepperd S. and Charnock
D.(3). on their affirmations about kitemarks or any other quality seals
for medical webs. In fact, we would like to draw your attention about
another interesting subject related to the Honcode seal.
In a pilot study (unpublished) on the quality of profesional medical
web, we evaluated the compliance with Honcode of the webs carrying the
Honcode seal, verifying if every web had their specific codenumber given
by Honcode (obtained after clicking the seal), and registering the date of
the last revision. We looked for webs containing the words “professional
medical web” as well as “Honcode” and then we recorded the presence of the
seal, site´s locations, the state of the agreement with Honcode and the
date of the last review.
We did this using the FAST ( www.alltheweb.com) searchengine
(accessed on January 22, 2002). Initially, it offered 48299 sites but did
not facilitate more than 4000. As long as 3744 were repeated, there were
only 256 different sites of which 185 showed the seal and 69 did not. From
these 185 showing the seal, there were 94 (50.81%) in compliance with
Honcode with an average of update (respect 15/02/02) of 693 days. On the
other hand, there were 91 webs (49.18%) showing the seal unproperly,
without permission or during the evaluation period.
Honcode has for some time been promoting a code of conduct and there
have been several overlapping initiatives in the United States(4), but
then again, their main drawback is that there is no external verification
so the system becomes open to abuse. Indeed, the Honcode seal presence is
not enough to guarantee credibleness at medical web sites.
Daniel Moro Quesada , MD, Family Physician, Gerencia de Atención Primaria
, Area IV, Oviedo, 33011 Asturias, Spain
danielmoro@inicia.es
José Luis Díez Jarilla, MD, Professor of Internal Medicine, University
Hospital, 33006 Oviedo, Asturias, Email
jldiezjarilla@interbook.net
Competing interests: None declared
1.- Health On the Net Foundation. HON code of conduct (HONcode) for
medical and health web sites: principles.
www.hon.ch/HONcode/Conduct.html.
2.- Meric F, Bernstam E V, Mirza N Q, Hunt K K, Ames F C, Ross M I,
et al. Breast cancer on the world wide web: cross sectional survey of
quality of information and popularity of websites. BMJ 2002;324:577-
581[Abstract/Full Text]
3.- Shepperd S. , Charnok D. Against internet exceptionalism. BMJ
2002; 324 : 556-557.[ Full Text]
4.- Rigby M, Forsström J, Roberts R, Wyatt J. Verifying quality and
safety in health informatics services. BMJ 2001;323:552-556.[Abstract/Full
Text]
Competing interests: No competing interests
Teaching the Pilot the currents
One of the issues raised in our previous correspondence concerned the number
of US sites found by South African medical students when researching a topic.
This was raised in the context of discussing the mechanism of PageRank
and the structure of the Internet.
Dr Marshall is quite correct when he points out that the Internet and Google
should not be blamed for "pilot error" - we don't wish them to be
blamed; we wish them to be understood so that they can be used
appropriately by patients and students. For that reason, we should have written
"The overwhelming or undiscerning use of Google…" rather than "The
overwhelming use of Google…"
In the link supplied, Dr Marshall demonstrates very effectively that adding
the country to the search phrase immediately reduces the number of non-South
African sites.
The problem is that when students (and possibly patients) wish to investigate
a medical condition, there is a tendency to enter only the name of the condition
as the search phrase, as our students did in the study we mentioned. Under
these circumstances, the nature of the Internet and Google is to deliver
mostly US sites. Patients and students need to be aware of why these are
not necessarily the most appropriate sites (they're not even necessarily
the most appropriate sites in the US); rather, these are the sites delivered
based on the search phrase, Google's PageRank, and the nature of the Internet.
PageRank is not necessarily either bad or good; it is merely a particular
system. Once it is understood, patients and students have a greater chance
of understanding the nature and implications of the search result, and are
in a better position to utilise the power of the Internet. They are also
less likely to be duped by those sites that exploit the nature of PageRank
and the Internet in order to increase ranking (such as the use of "link farms"
and paid reciprocal linking).
The first, and most sensible approach, as given by Dr Marshall, is to include
the region or country in the search phrase. But even that should be used
in conjunction with rather than in place of searches that exclude
the country or region. In our example, to search for only South
African sites would run the risk of ignoring a wealth of foreign experience
and research. In addition, the listing resulting from such a search in Google
reduces the scope, but still lists them according to PageRank's criteria
- again students and patients need to realise that these might not necessarily
be the most appropriate SA sites.
A second step should be to attempt to gauge the date of the site, for PageRank
sometimes allows old sites (because of the number of links to them) to rate
unexpectedly highly. This occurs not only in the medical field. A current
search on "most commonly used search engines" using Google yielded a document
in #2 position at
http://www.ariadne.ac.uk/issue6/survey/that did not mention Google. The reason is probably because the document
was last updated in 1996, but this does not affect its high ranking. The
implications for medical sites is that older sites will receive a ranking,
based, not on their accuracy, but on their age. While they may be valuable
for historical reasons, there is little to warn to the visitor that the medical
information may now be questionable because of subsequent research.
Other steps include using a variety of search engines, using local search
engines, and searching subject-specific databases. All this, however, takes
effort, and can only be justified if the reason is clear. The reason, of
course, is the one first raised in the original article by Meric et al (that
the popularity of sites is not related to their quality), expanded upon in
our responses (that an understanding of the nature of the Internet and Google
is required) and dealt with in Dr Marshall's letter (that "pilot error" is
a major contributing factor to inappropriate search results).
Competing interests:
None declared
Competing interests: No competing interests