Rapid Responses to:
|
|
Rapid Responses: Rapid Responses published:
-
![[Read Rapid Response]](/icons/misc/sectionDOWN.gif)
Internet Breast Cancer List
- Donald E M Price (7 March 2006)
-
Some background on the list
- Brian M Lynch (21 April 2006)
-
Questionable Methodology and Unsurprising Findings
- Peter Davison (22 April 2006)
-
"List mining" raises novel issues in research ethics
- James E. Till (24 April 2006)
-
Limited use of this finding
- Ketan K Dhatariya (24 April 2006)
-
Re: Questionable Methodology and Unsurprising Findings
- Elmer V. Bernstam, Funda Meric-Bernstam, Adol Esquivel (24 April 2006)
-
Re: Questionable Methodology and Unsurprising Findings
- Brian J Anthony (24 April 2006)
-
Re: Questionable Methodology and Unsurprising Findings
- Gunther Eysenbach (26 April 2006)
|
|
|||
|
Donald E M Price, Retired FE lecturer None
Send response to journal:
|
As reported in the BMJ the internet breast-cancer mailing list is good at self-correction on the rare occasions it's needed. This is far from the whole story. The list (join at www.bclist.org, search the archives at bclist.petebevin.com) offers health professionals insights into breast- cancer treatment from the patient and carer points of view, and much more. The newly-diagnosed can get answered those questions she, or he, thinks of only after leaving the doctor's presence - am I offered the best treatment? are there alternatives? how am I likely to feel during chemo- or radiotherapy (answer - it varies enormously)? should I opt for reconstruction (this guarantees lively debate)? and the one all new patients ask: Why won't my medical team tell me the symptoms of metastatic breast- cancer? Formed in 1994, the list rapidly became a strong community offering factual information, research findings, personal experiences and - most importantly perhaps - emotional support from a worldwide membership covering all stages of the disease. Regular "off topic" references to hobbies, family matters and holidays help the newly-diagnosed realise there is life after (and during) breast-cancer treatment. Postings from the terminally-ill have helped others facing metastasis come to terms with advanced disease. Widowers, myself included, have been helped through final stage caring and bereavement; one such, at his own expense, maintains the archives mentioned above. Competing interests: None declared |
|||
|
|
|||
|
Brian M Lynch, Senior Research Professor, Chemistry St. Francis Xavier University, Antigonish, NS B2G2W5, Canada
Send response to journal:
|
The Breast Cancer list referred to in the paper is hosted at Memorial University of Newfoundland, St. John's, NF and was begun by Jon Church. The sole control on the list is a limit on the number of daily messages permitted. An archive for the content is maintained by Pete Bevin of Toronto, whose partner Menya Wolfe died from the purportedly rare inflammatory breast cancer. Neither the list nor the archive are supported by Canadian and provincial governments. The majority of subscribers are from the U.S.A., but there are members from Australia, Germany, Great Britain, Israel,the Netherlands, New Zealand, South Africa and Sweden. Members of the list have an annual get-together, held so far in North America. Competing interests: None declared |
|||
|
|
|||
|
Peter Davison, MD PhD Toronto, N6G3M7
Send response to journal:
|
The numbers in this study [1], which suggests that the majority (2/3) of the misleading statements made in a virtual community were subsequently corrected by somebody, have to be questioned because of a major methodological problem which almost certainly introduced a substantial bias: The author who evaluated the accuracy of the postings was not blinded to the community responses. Under these circumstances it is no surprise that he will preferentially classify statements as "misleading" to which others in the community responded with a correction/critique, at the expense of those which did not evoke a response. The "clean" and rigorous methodology to determine an unbiased proportion of misleading and subsequently corrected statements would have been to blind the assessor(s) to the community responses, and only after a statement has been classified as misleading have an independent evaluator assess whether a response has rectified an earlier response. I am mystified how the peer-reviewer and the BMJ hanging committee could let this slip through, as in my mind this completely invalidates the study. On the other hand, most experts on virtual communities (in fact, anybody with common sense) would agree with the obvious truism that if people say something wrong in public other people will correct them - and few would be surprised by the findings of this study - which raises the question if there is anything new in this paper, and if research grants shouldn't be better spent to address the real problems of this world. I also have a problem with the suggestion in the discussion that the self- corrective power of communities is something "new" or something that emerged just recently. The ancient paper from Culver which they cite as historical comparison did not find similar results simply because Culver did not systematically look at this issue of subsequent corrections, not because "the Internet was new" or because people have changed. As an aside, the statement that "the Internet was new" [in 1994] is simply wrong - the World Wide Web may have been new, but not the Internet, and certainly not virtual communities. Virtual communities long predate the World Wide Web (Bulletin Board Systems, FIDOnet, Usenet etc). I have observed these discussions for the past 30 years as moderator and participant - I do not think anything substantial has changed in terms of people getting better or worse in posting and/or correcting misleading information. I think the authors, peer-reviewers, and editors of this piece were getting a bit carried away with their excitement about technology when publishing this paper or even citing complexity theory, chaos theory or self-organization [2] to describe the simple (= not complex!) phenomenon that when people speak or interact with each other (be it offline or online) people will correct each other. The same self-corrective measures are luckily also true for the scientific community. If somebody says something stupid, others will react. Am I the only one who sees that this emperor has no clothes? Can we now go back to the real problems of the world? 1. Adol Esquivel, Funda Meric-Bernstam, and Elmer V Bernstam. Accuracy and self correction of information received from an internet breast cancer list: content analysis. BMJ 2006 332: 939-942 2. Jadad et al. Are virtual communities good for our health? BMJ.2006; 332: 925-926 Competing interests: None declared |
|||
|
|
|||
|
James E. Till, Professor Emeritus University of Toronto, Toronto, Canada, M5S 1A1
Send response to journal:
|
"List mining" can be defined as the use, for research purposes, of messages sent to Internet-based mailing lists.
The ethical questions raised by list mining are especially important ones for health-related lists. For example, should those people who were the original sources of messages sent to such lists properly be regarded as "research subjects"? (This perspective raises questions pertaining to privacy, informed consent, whether the research is intrusive and has potential for harm, and whether the list should be perceived as "private" or "public" space).
Should they be regarded as "published authors"? (Although most participants may not even regard themselves as "amateur authors", there are issues of copyright and proper attribution to be considered if messages sent to such lists are cited verbatim).
Should they be regarded as "members of a community"? (If so, how best to deal with individuals within such communities, whose messages may need to be examined critically, because they might endanger public health? Fortunately, the article by Esquivel and colleagues provides evidence that, on the Breast-Cancer mailing list, there's self policing that works).
A few relevant references about Internet research ethics are available (see, for example, Internet research ethics).
Competing interests: None declared |
|||
|
|
|||
|
Ketan K Dhatariya, Consultant Endocrinologist Norfolk and Norwich University Hospital NHS Trust
Send response to journal:
|
Editor, I believe that the article by Esquivel et al is a little misleading in some respects. I believe that their findings were as a result of internet correspondence between individuals regarding a common condition which is given a lot of ‘webspace’, and media attention. My own (admittedly anecdotal) experience shows that there remains a vast amount of ignorance when considering rarer conditions. Between 2001 and 2003 I attempted to recruit hypoadrenal women for a study looking into adrenal hormone replacement. As the incidence of these conditions is very rare, I resorted to advertising for subjects on two websites – ‘self help groups’ for hypoadrenalism (after this had been approved by the ethics committee). I managed to recruit sufficient numbers for the study, but kept myself enrolled to check on progress amongst my volunteers until mid 2005. It was very surprising to see the depth of inaccuracy and distinct lack of knowledge amongst the people who posted on the sites. I refused to be drawn into discussions, as it became very obvious that I would have ended up being the ‘group doctor’, which I had no intention of becoming. Indeed, the one person on those sites who seemed to be giving the most information (not always well informed) was a senior haematology laboratory technician whose wife was hypoadrenal. Thus he was the authority that people deferred to as he was in some small way ‘medical’. Furthermore, from personal experience and that of my colleagues, the ever increasing band of ‘informed’ patients who come to clinic armed with items they ‘found on the web’ that vary from wildly inaccurate to frankly amusing suggests that many people refer to the web, but that an awful lot of nonsense is to be found there. I have no problem with tackling the fears and questions of concerned patients, but when one is seen to be contradicting the ‘wisdom’ of the all powerful net, it appears to provide yet another reason to consider early retirement. Competing interests: None declared |
|||
|
|
|||
|
Elmer V. Bernstam, Assistant Professor The University of Texas Health Science Center at Houston, Houston, TX 77030, Funda Meric-Bernstam, Adol Esquivel
Send response to journal:
|
We are happy to see that our paper has generated so much interest and passion. We also thank Dr. Davison and the BMJ for the opportunity to clarify several points. As stated in the paper, “[o]ur study was limited by the fact that a single reviewer determined the statements that might be false or misleading. We may therefore have missed some false or misleading statements. … Furthermore, reviewers were not blinded to the study hypotheses.” We agree with Dr. Davison that this may have introduced bias, as acknowledged in the discussion. Indeed, if there were additional false/misleading statements that were missed, this could have affected the results. In ongoing studies, we are continuing to test the self-correction hypothesis in other domains and using multiple independent reviewers. We respectfully disagree with Dr. Davison regarding both the importance of this topic and the novelty of our findings. As clinicians, we often encounter patients who have been mis-informed by family and friends. As patients increasingly turn to online communities for both social support and health information, it is important to ensure that these communities do not propagate mis-information. Hypotheses that are seemingly obvious have sometimes been shown to be false. For example, the “simple fact” that the world is flat seemed obvious to brilliant minds for centuries. Similarly, the benefit of post- menopausal hormone supplementation seemed obvious for decades, but is currently being reassessed. Finally, as Dr. Jadad suggests in the accompanying editorial, the self-correction hypothesis has less to do with technology and more to do with social interaction. Technology merely provides a convenient medium and allows researchers access to the data. Adol E. Esquivel, MD, MS
Competing interests: None declared |
|||
|
|
|||
|
Brian J Anthony, Senior Lecturer Middlesex University, London N19 5LW
Send response to journal:
|
It may be obvious to Dr Davidson, from his 30 years' experience, that self-correction takes place in online discussions. Unfortunately many others do not share his insight - writers in the professional health care literature as well as the popular media often express concern about the dangers of taking too much notice of material found on the Internet. Some writers and researchers have concluded that these dangers are so great that online discussion groups should be professionally moderated. In turn, some professionals will not recommend these groups to their patients because they fear that they may be misled or endangered by what they find. For my part, I think this study does deal with "real problems of the world", and was worth doing, because it helps to demonstrate some of the internal processes that make an online support group work. It shows that among the members of the Breast Cancer Mailing List there are responsible and thoughtful individuals who take an interest in the quality and accuracy of the discussions in their list. However, as the authors note, their findings may not be generalisable to other online forums. This is certainly true because the quality of discussions in online forums are entirely dependent on the individuals who happen to be using them at any particular time - their motives, beliefs, attitudes and experiences. An online group changes over time: it may go through periods of conflict and variations in activity level; it may be temporarily or permanently colonised by individuals with axes to grind or obsessions to pursue. I think that these processes are worth studying so that health professionals can get a deeper understanding of how online support groups work. Competing interests: None declared |
|||
|
|
|||
|
Gunther Eysenbach, Senior Scientist & Editor-in-chief, Journal of Medical Internet Research http://www.jmir.org Toronto, M5G2C4
Send response to journal:
|
I agree with most of what has been said above in other rapid responses - as the authors of this study themselves acknowledge, the study design is not very rigorous, and the study has considerable limitations and flaws affecting internal and external validity (acknowledging these in the discussion does not make them go away, and I wish the BMJ would put the bar for publishing such studies a bit higher). The comment of Jim Till, with whom I authored a piece on the ethics of doing research in virtual communities [1], is also appreciated - there is no indication in the Esquivel paper that the authors considered these issues (although they might have - but it is not explictly mentioned). The assertion of Dr Antony in his rapid reply that "some writers and researchers have concluded that these dangers are so great that online discussion groups should be professionally moderated." is not referenced and I doubt that anybody actually said something like this. I am not aware of any evidence suggesting that moderated communities are "better" than unmoderated communities, or the other way around [2]. The Esquivel paper unfortunately doesn't contribute to answering this question (they should have done a comparative study instead of a purely descriptive one). Yes, there is clearly inaccurate information on the Internet, and yes, there are people out there (professionals and laypersons) using several methods to correct this information, be it through responding to false claims, be it through narrative reviews, collections of "trusted" links, rating systems or filtering systems. So the question is not who is right: those who warn of the pitfalls of relying on Internet information - or, heck, any information for that matter, or those who say there is also some sort of self-regulation - BOTH are right. Duh! Speaking of rating systems, I do not agree with Jadad who claims in his editorial - citing his two (outdated) reviews on the lack of published validity and reliability estimates for rating systems - that such systems are an universal failure. Collaborative filtering methods such as http://www.stumbleupon.com are becoming increasingly popular, and gateways run by governments or professionally run organizations are widely used and appreciated by consumers. These systems are very much part of a larger self-regulatory system, which is not confined to virtual communities, and not confined to *consumers* (as opposed to professionals) counteracting the "dark forces" of snake oil therapies and misinformation - professionals obviously also have a role here, and many consumers - while acknowledging self-corrective forces by other consumers - continue to look to health professionals for guidance. And there is nothing wrong with this, even in the era of consumerism and "consumer health informatics". Thus, to conclude from the two Jadad reviews that rating systems are a universal failure, due to "reliability and validity concerns" means throwing out the baby with the bathwater. As I wrote in more detail elsewhere [3], certain aspects of evaluating a health website, especially when conducted by third-party evaluators, are a matter of taste and inherently lead to a high inter-observer variability. However, low reliability does not automatically mean that these ratings or reviews are useless. The analogy here is movie review, or a peer-review report of an academic paper. To conclude that due to this low inter-rater reliability it is impossible to assess the quality of a movie or a academic paper (or not worth reporting results) would mean throwing out the baby with the bathwater. Although certain evaluations may be subjectivist, taken together, these movie reviews help consumers to make up their minds which movie to go to, peer-review reports help editors to decide whether an academic paper is valid and helps authors to improve it. On the same level, website evaluations - whether conducted by the developer himself, or by third parties - help developers to improve their sites. Even if reviews of a website are not reliable in a sense that different people spot different flaws (i.e. low inter-observer reliability), evaluation may be useful to educate developers and users what to look at, and help them to improve sites - in analogy to the function of an art critic, for which many say the task is not to judge art, but to educate the public what to look for. We were previously quite critical of the myriads of "me-too" studies employing methods to estimate the "proportion" of inaccurate information on the Internet [4], and the same methodological concerns mentioned in that systematic review are applicable to the Esquivel paper. x% of information is inaccruate, while y% where subsequently corrected - the question is really "SO WHAT?" - in particular as in other virtual communities the picture may look very different. While I appreciate the work the authors have done, I still think we should move beyond the point of doing purely descriptive infodemiology studies [5, 6]. I do not dismiss the fact that research into virtual communities is important, as "virtual communities are probably the one Internet application area with the greatest effect on persons with cancer", and "although much discussion of online cancer information has focused on the negative aspects of the Internet, such as inaccurate information, the glass is filled almost to the top with unprecedented opportunities. (...) It would be a mistake to leave them untapped by focusing our attention on the small area that is empty." [7] References 1. Eysenbach G, Till JE. Ethical issues in qualitative research on internet communities. : BMJ. 2001 Nov 10;323(7321):1103-5. http://bmj.bmjjournals.com/cgi/content/full/323/7321/1103 2. Eysenbach G, Powell J, Englesakis M, Rizo C, Stern A. Health related virtual communities and electronic support groups: systematic review of the effects of online peer to peer interactions. BMJ. 2004 May 15;328(7449):1166. http://bmj.bmjjournals.com/cgi/content/full/328/7449/1166 3. Eysenbach G. Design and evaluation of consumer health information websites. In: Lewis D, Eysenbach G, Kukafka R, Jimison H, Stavri Z (eds.): Consumer Health Informatics. Springer New York (2005) 4. Eysenbach G, Powell J, Kuss O, Sa ER. Empirical studies assessing the quality of health information for consumers on the world wide web: a systematic review. JAMA. 2002 May 22-29;287(20):2691-700 5. Eysenbach G. Infodemiology: the epidemiology of (mis)information. Am J Med 2002;113: 763-5 6. Eysenbach G. Websites on screening for breast cancer: "infodemiology" studies have surely had their day. BMJ. 2004 Mar 27;328(7442):769; author reply 769-70. http://bmj.bmjjournals.com/cgi/content/full/328/7442/769-b 7. Eysenbach G. The impact of the Internet on cancer outcomes. CA Cancer J Clin. 2003 Nov-Dec;53(6):356-71. http://caonline.amcancersoc.org/cgi/content/full/53/6/356 Competing interests: None declared |
|||