BMJ 2000;321:1511-1515 ( 16 December )

Information in practice

Quality of web based information on treatment of depression: cross sectional survey

Kathleen M Griffiths, visiting fellow, Helen Christensen, senior fellow. 

Centre for Mental Health Research, The Australian National University, Canberra 0200, Australia

Correspondence to: K M Griffiths kathy.griffiths{at}anu.edu.au


    Abstract
Top
Abstract
Introduction
Methods
Results
Discussion
References

Objectives: To evaluate quality of web based information on treatment of depression, to identify potential indicators of content quality, and to establish if accountability criteria are indicators of quality.
Design: Cross sectional survey.
Data sources: 21 frequently accessed websites about depression.
Main outcome measures: (i) Site characteristics; (ii) quality of content---concordance with evidence based depression guidelines (guideline score), appropriateness of other relevant site information (issues score), and subjective rating of site quality (global score); and (iii) accountability---conformity with core accountability standards (Silberg score) and quality of evidence cited in support of conclusions (level of evidence score).
Results: Although the sites contained useful information, their overall quality was poor: the mean guideline, issues, and global scores were only 4.7 (range 0-13) out of 43, 9.8 (6-14) out of 17, and 3 (0.5-7.5) out of 10 respectively. Sites typically did not cite scientific evidence in support of their conclusions. The guideline score correlated with the two other quality of content measures, but none of the content measures correlated with the Silberg accountability score. Content quality was superior for sites owned by organisations and sites with an editorial board.
Conclusions: There is a need for better evidence based information about depression on the web, and a need to reconsider the role of accountability criteria as indicators of site quality and to develop simple valid indicators of quality. Ownership by an organisation and the involvement of a professional editorial board may be useful indicators. The study methodology may be useful for exploring these issues in other health related subjects.


    Introduction
Top
Abstract
Introduction
Methods
Results
Discussion
References

The web represents an unprecedented opportunity to provide high quality, accessible healthcare information to consumers and health providers. In the absence of editorial controls, however, the information may be of low quality and potentially harmful.1

In an influential paper Silberg et al proposed that accountability standards (disclosure of authorship, ownership, and currency of information) may be useful indicators of the quality of web based health information.2 These accountability criteria have been widely assumed to reflect website quality,3-5 but their validity as indicators of quality of content have not been investigated. Moreover, there have been few systematic studies of the actual quality of the content of health information on the web,6-9 and these studies have typically used textbook summaries5 or author opinion8 as the gold standard for assessing content quality rather than meta-analyses of the available evidence.10 Finally, no published studies have systematically evaluated the quality of mental health websites even though mental disorders are a common cause of disability and the World Health Organisation has predicted that depression will be the second largest cause of disability within 20 years.11 Since only a minority of people with depressive disorders receive treatment,12 websites are potentially useful for encouraging depressed people to seek help.

In this study we aimed to survey websites that a "typical" user might access when searching for information on depression. We evaluated the quality of the information on the treatment of depression (including comparison with evidence based guidelines and meta-analyses) and the relation between content quality and accountability indicators and other site characteristics.


    Methods
Top
Abstract
Introduction
Methods
Results
Discussion
References

Selection of sites
To identify potential sites for our survey, we used two search engines, DirectHit (www.directhit.com) and MetaCrawler (www.go2net.com/search.html), to conduct searches in March 1999 using the key word "depression." DirectHit returns 10 "popular" sites based on analyses of previous user activity for a query (primarily frequency of "clickthroughs" from a result list). MetaCrawler integrates the results for a query from several well known search engines including Alta Vista, Excite, Infoseek, Lycos, WebCrawler, Yahoo, LookSmart, Thunderstone, and Mining Co. The usefulness of Direct Hit and MetaCrawler in identifying popular sites has not been the subject of formal independent evaluation. However, in the absence of any other suitable search engine tools, the list of sites yielded by our search methodology provided the best available approximation to a list of depression sites that would be most commonly encountered by a "typical" user.

We excluded sites not relevant to depression or no longer active and one site concerned solely with seasonal affective disorder. All other sites identified by Direct Hit (n=9) and the highest ranked sites from the MetaCrawler search (n=11) were included in our analysis. We rated separately a "stand alone" book imported from a third party source by one site. We identified and printed out site material by systematically following all internal links. We excluded external links, news sections (typically internally contradictory), sections relating to bipolar disorder and schizophrenia, and book reviews.

Site assessment
We each independently evaluated the sites' characteristics, content, and accountability. We resolved any disagreements soon after rating by discussion and reference to site material.

Characteristics of the site
We rated each site as to its purpose, scope, ownership, country of origin and for involvement of a drug company, professional editorial board, or health professional. We also rated sites according to whether they promoted products or services and whether they contained a disclaimer or qualifier regarding information provided.

Quality of content
Guideline score---We assessed concordance between site information and best practice by using a 43 item rating scale developed from the evidence based guidelines on clinical practice for treating depression published by the Agency for Health Care Policy and Research (AHCPR).13 These guidelines are one of a set of US federal guidelines developed according to the general principles outlined in the US Institute of Medicine's guidelines for developing evidence based guidelines on clinical practice.14 The guidelines were developed by a multidisciplinary panel from systematic reviews of the scientific evidence (meta-analyses of randomised controlled trials using modified "intention to treat" analyses) and underwent extensive review by all panel members, a methodologist, 28 scientific reviewers, and 73 organisations. Each item in our rating scale corresponded to one of the statements in the guidelines. The scale covered the use of drugs, psychotherapy, combined drugs and psychotherapy, and electroconvulsive therapy. Topics included effectiveness, indications, selection within a treatment type, failure to respond, and frequency of visits. We calculated a guideline score for each site by counting the number of items on the scale for which site information agreed with the guidelines. We also calculated a core guideline score (out of 5) from a subset of key items relating to indications for and effectiveness of the four major treatment types (see box 1).


Box 1 : Core guidelines adapted from AHCPR clinical practice guidelines13

  • Antidepressant drugs are an effective treatment for major depressive disorder
  • Antidepressants are the first line treatment for moderate to severe depression or psychotic or melancholic atypical symptoms (overeating, oversleeping, weight gain), at patient's request, if psychotherapy unavailable, or if previous response to drug. (Site must have identified at least one of the above qualifiers to be rated as in agreement with guideline)
  • Psychotherapy can be an effective first line treatment for mild to moderate depression
  • Initial treatment with a combination of drugs and psychotherapy is reasonable in only some circumstances (such as chronic prior course of illness or poor recovery between depressive episodes, psychotherapy alone or drugs alone only partly effective, history of psychosocial problems both during and outside depressive episodes, history of poor adherence to treatment)
  • Electroconvulsive therapy may be effective in certain cases of severe depression

Issues score---We evaluated other treatment issues with a 17 item scale designed to assess the appropriateness of site information about important treatment and management issues not adequately or not directly evaluated by the guideline scale (such as the importance of seeking help, discussion of side effects, depression in young people, and the relation between depression and suicide risk).

Global score---We each provided a subjective judgment of the overall quality of a site (score out of 10) and then calculated an average score for each site. There was a moderately high correlation between our scores (r=0.69, P=0.001), and the mean scores for each of us did not differ significantly (mean difference 0.38 (SD 1.5), t 20=1.16), suggesting acceptable inter-judge agreement despite the unstructured and subjective nature of the task.

Interventions recommended---We rated the sites according to whether each of a range of interventions were mentioned; were said or implied to be effective or useful or were recommended as a first line, second line, or adjunct treatment for all or some groups; and were said to be ineffective or were not recommended. Interventions denoted effective but explicitly not recommended were coded as "not recommended."

Sources of help recommended---We rated potential sources of help for depression as recommended, not recommended, or not mentioned.

Accountability
Silberg score---Sites were rated on a 9 point scale according to Silberg et al's criteria of authorship (whether authors and their affiliations and credentials were clearly identified), attribution (whether sources and references were mentioned), disclosure (whether ownership of the site and sponsorship was disclosed), and currency (whether the site has been modified in the past month and year and whether the date the site was created or modified was specified).2

Level of evidence score---We recorded the stated level of evidence associated with each intervention using a 5 point scale adapted from a previously published scale of hierarchy of evidence (box 2).15 We based the evidence scores only on information explicitly provided by the site and not on our knowledge of the cited study or relevant literature.


Box 2 : Quality of evidence rating system (adapted from National Health and Medical Research Council15)

Level 1---Evidence obtained from a review of all relevant randomised controlled trials
Level 2---Evidence obtained from at least one randomised controlled trial
Level 3---Evidence obtained from controlled trials without randomisation
Level 4---Evidence obtained from multiple time series with or without intervention
Level 5---Other evidence (such as opinions or policies of respected authorities based on clinical experience, descriptive studies, or reports of expert committees; summary by writers using a variety of written material; expert testimony; reference to the philosophy of a particular practitioner; reference to personal experience)

Statistical analyses
We assessed site quality and accountability as a function of site characteristic using Mann-Whitney tests, Kruskall-Wallis analyses followed by Mann-Whitney tests, or Fisher's exact probability tests. We calculated non-parametric confidence intervals for the main findings using the procedure outlined by Campbell and Gardner.16 We calculated intercorrelations between variables using Pearson's correlation tests and Phi tests.


    Results
Top
Abstract
Introduction
Methods
Results
Discussion
References

Site characteristics
Of the 21 sites included in our analysis, 19 were US based, one was European, and the remaining site was of unknown origin. The principal purpose of the sites was to provide information or educational material (10 sites), links (4), a consumer forum (1), or information in combination with either links or consumer forum or both (6). Table 1 lists other characteristics of the sites.


                              
View this table:
[in this window]
[in a new window]
 

Table 1. Quality of content and accountability of 21 websites with information on treating depression as a function of site characteristics. Values are mean scores unless stated otherwise

Quality of content
The mean guideline, issues, and global scores were 4.7 out of 43, 9.8 out of 17, and 3 out of 10 respectively (table 1), indicating little concordance with guideline recommendations, inadequate consideration of management or treatment issues, and generally low overall ratings.

In part, the low guideline score reflected poor coverage: on average, the sites lacked material relevant to over two thirds of the guideline items. However, the information that the sites did provide was often inaccurate: in the case of the five core guidelines (box 1) most of the sites (average 58%) contradicted or provided material inconsistent with the guidelines.

Sites usually recognised that antidepressants and psychotherapy are effective but were often inaccurate in the specified indications for these treatments. For example, many sites emphasised one form of these treatments over the other regardless of the severity of the depression and other important factors; almost half of the sites recommended combined use of antidepressants and psychotherapy as a first line treatment when this is not recommended by the AHCPR guidelines. Sites were often internally inconsistent, especially when material was derived from more than one author or source.

Between eight and 13 of the sites failed to discuss contraindications for drugs, failed to recognise individual differences in the effects of antidepressants, or did not identify the importance of switching drugs as required. Few sites acknowledged that chronic and subsequent episodes of depression may require a different approach, that the management and treatment of depression in young people may differ from that for adults, or that the availability of treatment may be a factor in selecting treatments. However, most of the sites discussed side effects and the long term nature of antidepressant treatments, and most of those that mentioned herbal or dietary supplements included some discussion of their side effects.

All sites indicated that depression can be treated, most indicated that the depression should be treated, and only one failed to mention the risk of suicide in depression. Although most sites mentioned effective treatments such as selective serotonin reuptake inhibitors, tricyclic antidepressants, psychotherapy, and cognitive therapy, less than half mentioned several important evidence based conventional treatments (such as newer antidepressants, interpersonal therapy, behaviour therapy, and cognitive behaviour therapy) and only six recommended St John's wort despite level 1 evidence suggesting it is effective for mild depression.

All sites promoted consultation with a health professional for diagnosis or treatment, and most provided a list of contact organisations for further information or assistance. All sites recommended a doctor as a source of help (see table 2). Psychiatrists, psychopharmacologists, psychologists, and psychotherapists are professionals with expertise in delivering known effective treatments for depression, but six sites did not mention any of these professionals as potential sources of help. Sites were as likely to recommend websites, family members, the clergy, or friends as they were to recommend psychiatrists (table 2).


                              
View this table:
[in this window]
[in a new window]
 

Table 2. Sources of help recommended by 21 websites with information on treating depression

Accountability
The mean Silberg score was 5.4 out of 9 (table 1). Most of the sites clearly specified the authors of the web content (13 sites) and their credentials (11 sites) and affiliations (11 sites). Nine of the sites mentioned at least some sources and references on the site (although such information was typically not comprehensive). All but one site disclosed an owner of the site, and three mentioned sponsors. Most sites indicated when the site had been created or modified: most had been modified in the past year, and nine had been modified in the past month.

The 21 sites mentioned a total of 53 different interventions, but sites typically did not provide supporting scientific information or refer in general terms to the level of evidence available to support their recommendations. Since most sites mentioned antidepressants and psychotherapy and these therapies are supported by level 1 evidence, we analysed the data for these interventions further (taking the highest level of evidence across generic and individual forms of each treatment type). Only five sites mentioned any scientific evidence in support of the use of antidepressants, and only one of these referred to level 1 evidence. Similarly, only three sites that recommended or did not recommend psychotherapy cited scientific evidence in support of their conclusions, and only one site cited level 1 evidence.

Association between quality of content, accountability, and site characteristics
The guideline score was significantly correlated with the other two measures of quality of content (with global score, r=0.53, P<0.05; with issues score, r=0.74, P<0.01). However, none of the measures of quality of content correlated significantly with the Silberg accountability score (r=-0.5 to 0.21). Of the sites offering recommendations about psychotherapy, those citing scientific evidence were more likely to achieve an above median guideline score (Phi=0.50, P=0.034) and showed a tendency to achieve above median issues scores (Phi=0.45, P=0.058). There was no comparable significant relationship for antidepressants.

As table 1 shows, sites owned by organisations had significantly higher guideline and issues scores than those owned by individuals (difference in mean guideline scores 3.8 (95% confidence interval 1 to 6), U=21, P=0.016; difference in mean issues scores 2.1 (0 to 4), U=26, P=0.043), as did sites with an editorial board compared with others (difference in mean guideline scores 3.8 (0 to 7), U=16, P=0.05; difference in mean issues scores 2.1 (0 to 4), U=16, P=0.05). Only sites owned by organisations reported scientific evidence to support their endorsement of antidepressants and psychotherapy. Sites owned by organisations were significantly more likely than individually owned sites to cite scientific evidence in support of antidepressants (50% v 0%, Fisher's exact test P=0.03), as were sites involving drug companies compared with others (75% v 13%, P=0.03).

However, there was no significant association between the total Silberg score and site characteristics. In fact, analyses of individual Silberg items showed that sites owned by organisations and those involving drug companies were less likely than their counterparts to indicate the author's identity, affiliation, and credentials. Thus, for sites owned by organisations, author's identity was given by 36% (v 90% of others, Fisher's exact test P=0.024), affiliation by 27% (v 80%, P=0.03), and credentials by 27% (v 80%, P=0.03). For sites involving drug companies, author's identity was given by none (v 77%, P=0.012), affiliation by none (v 65%, P=0.04), and credentials by none (v 65%, P=0.04).

There was no association between the characteristics of a site and the total number of sources of help endorsed or whether psychiatrists, psychopharmacologists, psychologists, or psychotherapists were specifically nominated as sources of help. However, sites involving health professionals were more likely than other sites to endorse the general category "mental health specialists/professionals" as a source of help: 71% of sites involving a psychiatrist or medical practitioner and 100% of sites involving a psychologist v 10% of other sites (P<0.05 for each case).


    Discussion
Top
Abstract
Introduction
Methods
Results
Discussion
References

In our review of 21 popular websites containing information about treating depression we found that the quality of this information was poor. This finding reinforces concerns raised by other studies which have found inadequate quality6-9 or poor coverage17 of important health issues on the web. There is a need to improve the accuracy and coverage of information about depression on the web with regard to the relative effectiveness of different treatments, the main indications for particular treatments, important management issues such as duration of treatment, reviewing and changing treatments, and the relevance of professional expertise and patient preferences. Sites should also warn readers that tricyclic antidepressants are ineffective for adolescents and that drugs may not be the first line of treatment for this age group.18

Our findings raise questions about the usefulness of Silberg et al's accountability criteria as indicators of website quality2 and suggest that further investigation of indicators of quality is warranted. Particular site characteristics (such as ownership by an organisation or existence of a professional editorial board) may prove more useful indicators of content quality than disclosure of information per se. Our results also suggest that the number of different types of interventions mentioned may be a predictor of site quality, as may the citation of scientific evidence in support of recommended treatments.

The critical question is whether the attributes which were associated with the better quality sites about depression are valid indicators of the quality of other types of health related sites. Our methodology could be used to address this question and to identify those attributes that are common predictors of quality for different medical subjects. The methodology lends itself to replication in different subjects since any systematically produced set of guidelines can serve as a rating scale with which to evaluate websites.

It is possible that the inadequacies we documented are not restricted to websites but reflect the beliefs and level of knowledge of many health professionals. McClung et al have reported that even medical teaching centres disseminated inadequate reviews on the web.7 It is unlikely that the AHCPR guidelines are outdated or inadequate since a review of more recent evidence concluded that the major AHCPR conclusions are still applicable and that, when rigorously implemented, the guidelines result in improved outcomes compared with usual care.19 The guidelines have been criticised for their failure to recommend psychotherapy as a first line treatment for severe depression,20 but, although there is some evidence to support this criticism,21 few studies have directly compared the efficacy of different treatments for severe depression and the findings have been inconsistent. By contrast, a large number of randomised controlled trials have demonstrated the efficacy of antidepressants in treating severe depression.

Despite their generally low scores for content quality, many sites did contain important and potentially useful information. It is even possible that a formal evaluation might show that such sites improve the mental health outcomes of those who visit them. Silberg et al have referred to the importance of distinguishing the flowers from the weeds on the internet superhighway. However, a single site, whether owned by a consumer or a health professional, may grow both flowers and weeds. The real challenge is to devise strategies that selectively eliminate the weeds but leave the flowers to bloom.


What is already known on this topic

Depression is a major source of disability in the community

Websites offer an opportunity to disseminate information to the public about effective treatments

However, little is known about the quality of existing sites about depression or about indicators of a good health website

What this study adds

An audit of 21 popular websites revealed that the general quality of information on the treatment of depression is poor

Currently popular criteria for evaluating the quality of websites were not indicators of content quality, but sites with an editorial board and sites owned by organisations produced higher quality sites than others



    Acknowledgments

   Contributors: KG conceived and designed the study, rated websites, analysed and interpreted the data, and wrote the paper. HC conceived and designed the study, rated websites, analysed data, and edited the paper. Jo Medway collected and organised data and commented on the manuscript. Ailsa Korten calculated confidence intervals for non-parametric data and commented on statistical procedures. Andrew MacKinnon contributed to discussions during the design phase. KG and HC are guarantors for the paper.

    Footnotes

Funding: This work was supported by grant 973302 from the National Health and Medical Research Council.

Competing interests: None declared

This article is part of the BMJ's randomised controlled trial of open peer review. Documentation relating to the editorial decision making process is available on the BMJ's website


    References
Top
Abstract
Introduction
Methods
Results
Discussion
References

1. Bower H. Internet sees growth of unverified health claims. BMJ 1996; 313: 381[Free Full Text].
2. Silberg WM, Lundberg GD, Musacchio RA. Assessing, controlling and assuring the quality of medical information on the internet. JAMA 1997; 277: 1244-1245[Abstract/Free Full Text].
3. O'Mahony B. Irish health care web sites: a review. Ir Med J 1999; 92: 334-337[Medline].
4. Jadad AR, Gagliardi A. Rating health information on the internet: navigating to knowledge or to Babel? JAMA 1998; 279: 611-614[Abstract/Free Full Text].
5. Shon J, Musen MA. The low availability of metadata elements for evaluating the quality of medical information on the world wide web. Proc AMIA Symp 1999; (1-2): 945-949.
6. Impicciatore P, Pandolfini C, Casella N, Bonati M. Reliability of health information for the public on the world wide web: systematic survey of advice on managing fever in children at home. BMJ 1997; 314: 1875-1879[Abstract/Free Full Text].
7. McClung HJ, Murray RD, Heitlinger LA. The internet as a source for current patient information. Pediatrics 1998; 101: 1-4[Abstract/Free Full Text].
8. Soot LC, Moneta GL, Edwards JM. Vascular surgery and the internet: a poor source of patient-oriented information. J Vasc Surg 1999; 30: 84-91[CrossRef][Medline].
9. Davison K. The quality of dietary information on the world wide web. Clin Perform Qual Health Care 1997; 5: 64-66[Medline].
10. Wyatt JC. Measuring quality and impact of the world wide web. BMJ 1997; 314: 1879-1881[Free Full Text].
11. Murray CJL, Lopez AD, eds. The global burden of disease: a comprehensive assessment of mortality and disability, injuries and risk factors in 1990 and projected to 2020. Cambridge, MA: Harvard University Press, 1996.
12. Kessler RC. Some considerations in making resource allocation decisions for the treatment of psychiatric disorders. In: Andrews G, Henderson S, eds. . Unmet need in psychiatry: problems, resources, responses. Cambridge: Cambridge University Press, 2000:59-84.
13. Depression Guideline Panel. Depression in primary care. , Vol 2. Treatment of major depression Maryland: US Department of Health and Human Services, 1993. (clinical practice guidelines No 5. AHCPR Publication No 93-0551.)
14. Institute of Medicine. Guidelines for clinical practice: from development to use. Washington, DC: National Academy Press, 1992.
15. National Health and Medical Research Council. Guidelines for the development and implementation of clinical practice guidelines. Canberra: NHMRC, 1995.
16. Campbell MJ, Gardner MJ. Calculating confidence intervals for some non-parametric analyses. In: Gardner MJ, Altman D, eds. . Statistics with confidence: confidence intervals and statistical guidelines. London: BMJ Publishing Group, 1989.
17. Sandvik H. Health information and interaction on the internet: a survey of female urinary incontinence. BMJ 1999; 319: 29-32[Abstract/Free Full Text].
18. National Health and Medical Research Council. Clinical practice guidelines: depression in young people. Canberra: NHMRC, 1997.
19. Schulberg HC, Katon W, Simon GE, Rush AJ. Treating major depression in primary care practice: an update of the Agency for Health Care Policy and Research Practice guidelines. Arch Gen Psychiatry 1998; 55: 1121-1127[Abstract/Free Full Text].
20. Persons JB, Thase ME, Crits-Cristoph P. The role of psychotherapy in the treatment of depression: review of two practice guidelines. Arch Gen Psychiatry 1996; 53: 283-290[Abstract/Free Full Text].
21. Schulberg HC, Pilkonis PA, Houck P. The severity of major depression and choice of treatment in primary care practice. J Consult Clin Psychol 1998; 66: 932-938[CrossRef][Medline].

(Accepted 17 July 2000)

© BMJ 2000
Add to CiteULike CiteULike   Add to Complore Complore   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us   Add to Digg Digg   Add to Reddit Reddit   Add to StumbleUpon StumbleUpon   Add to Technorati Technorati    What's this?

Relevant Articles

Quality of health information about depression on internet
André Tomlin, Karin Lee Dearness, John Geddes, S J Darmoni, M C Haugh, B Lukacs, J P Boissel, Kathleen M Griffiths, and Helen Christensen
BMJ 2001 322: 1366. [Extract] [Full Text]

Getting closer to patients and their families
BMJ 2000 321: 0. [Full Text] [PDF]

Getting closer to patients and their families
BMJ 2000 321: 0. [Full Text] [PDF]

Quality of web based information on depression is poor
BMJ 2000 321: 0. [Full Text]

This article has been cited by other articles:

  • Akram, G, Thomson, A H, Boyter, A C, Morton, M J S (2008). Characterisation and evaluation of UK websites on attention deficit hyperactivity disorder. Arch. Dis. Child. 93: 695-700 [Abstract] [Full text]  
  • Caron, S., Berton, J., Beydon, L. (2007). Quality of anaesthesia-related information accessed via Internet searches. Br J Anaesth 99: 195-201 [Abstract] [Full text]  
  • Powell, J., Clarke, A. (2006). Internet information-seeking in mental health: Population survey. Br. J. Psychiatry 189: 273-277 [Abstract] [Full text]  
  • Chang, T. (2005). Online Counseling: Prioritizing Psychoeducation, Self-Help, and Mutual Help for Counseling Psychology Research and Practice. The Counseling Psychologist 33: 881-890 [Abstract]  
  • Baker, L., Rideout, J., Gertler, P., Raube, K. (2005). Effect of an Internet-Based System for Doctor-Patient Communication on Health Care Spending. J. Am. Med. Inform. Assoc. 12: 530-536 [Abstract] [Full text]  
  • Martins, E N, Morse, L S (2005). Evaluation of internet websites about retinopathy of prematurity patient education. Br J Ophthalmol 89: 565-568 [Abstract] [Full text]  
  • Anselmo, M. A., Lash, K. M., Stieb, E. S., Haver, K. E. (2004). Cystic Fibrosis on the Internet: A Survey of Site Adherence to AMA Guidelines. Pediatrics 114: 100-103 [Abstract] [Full text]  
  • Nicholas, D., Huntington, P., Homewood, J. (2003). Assessing used content across five digital health information services using transaction log files. Journal of Information Science 29: 499-515 [Abstract]  
  • Sheehan, N. L, Kelly, D. V, Tseng, A. L, van Heeswijk, R. P., Beique, L. C, Hughes, C. A (2003). Evaluation of HIV Drug Interaction Web Sites. The Annals of Pharmacotherapy 37: 1577-1586 [Abstract] [Full text]  
  • Solomons, R. E. B. (2003). Assisted Internet Access for Patients in a General Practitioner Surgery: Helping the Information Poor. Health Informatics Journal 9: 199-206 [Abstract]  
  • Slater, M. D., Zimmerman, D. E. (2003). Descriptions of Web Sites in Search Listings: A Potential Obstacle to Informed Choice of Health Information. AJPH 93: 1281-1282 [Full text]  
  • Bath, P. A., Bouchier, H. (2003). Development and Application of a Tool Designed to Evaluate Web Sites Providing Information on Alzheimer's Disease. Journal of Information Science 29: 279-297 [Abstract]  
  • Christensen, H., Griffiths, K. (2003). The internet and mental health practice. Evid. Based Ment. Health 6: 66-69 [Full text]  
  • Baker, L., Wagner, T. H., Singer, S., Bundorf, M. K. (2003). Use of the Internet and E-mail for Health Care Information: Results From a National Survey. JAMA 289: 2400-2406 [Abstract] [Full text]  
  • Bouchier, H., Bath, P. A. (2003). Evaluation of Websites that Provide Information on Alzheimer's Disease. Health Informatics Journal 9: 17-31 [Abstract]  
  • Fallis, D., Fricke, M. (2002). In reply:--. J. Am. Med. Inform. Assoc. 9: 403-404 [Full text]  
  • Eysenbach, G., Powell, J., Kuss, O., Sa, E.-R. (2002). Empirical Studies Assessing the Quality of Health Information for Consumers on the World Wide Web: A Systematic Review. JAMA 287: 2691-2700 [Abstract] [Full text]  
  • Gagliardi, A., Jadad, A. R (2002). Examination of instruments used to rate quality of health information on the internet: chronicle of a voyage with an unclear destination. BMJ 324: 569-573 [Abstract] [Full text]  
  • Kunst, H., Groot, D., Latthe, P. M, Latthe, M., Khan, K. S (2002). Accuracy of information on apparently credible websites: survey of five common health topics. BMJ 324: 581-582 [Full text]  
  • Pandolfini, C., Bonati, M. (2002). Follow up of quality of public oriented health information on the world wide web: systematic re-evaluation. BMJ 324: 582-583 [Full text]  
  • D'Alessandro, D., Kingsley, P. (2002). Creating a Pediatric Digital Library for Pediatric Health Care Providers and Families: Using Literature and Data to Define Common Pediatric Problems. J. Am. Med. Inform. Assoc. 9: 161-170 [Abstract] [Full text]  
  • Tomlin, A., Dearness, K. L., Geddes, J., Darmoni, S J, Haugh, M C, Lukacs, B, Boissel, J P, Griffiths, K. M, Christensen, H. (2001). Quality of health information about depression on internet. BMJ 322: 1366-1366 [Full text]  
  • Berland, G. K., Elliott, M. N., Morales, L. S., Algazy, J. I., Kravitz, R. L., Broder, M. S., Kanouse, D. E., Munoz, J. A., Puyol, J.-A., Lara, M., Watkins, K. E., Yang, H., McGlynn, E. A. (2001). Health Information on the Internet: Accessibility, Quality, and Readability in English and Spanish. JAMA 285: 2612-2621 [Abstract] [Full text]  
  • (2001). Poor Quality of Web-Based Depression Information. JWatch Psychiatry 2001: 1-1 [Full text]  

Rapid Responses:

Read all Rapid Responses

Assessment of the quality of health information
Peter Mills
bmj.com, 18 Dec 2000 [Full text]
Proves very little
John McManamy
bmj.com, 19 Dec 2000 [Full text]
Assessing the quality of medically related web sites
Vincent Abbott
bmj.com, 19 Dec 2000 [Full text]
Is there a need for better methodologies?
Christopher Gale
bmj.com, 9 Jan 2001 [Full text]
Quality of web based information on treatment of depression
André Tomlin
bmj.com, 17 Jan 2001 [Full text]
Quality of web based information on treatment of depression
Margaret Haugh
bmj.com, 19 Jan 2001 [Full text]
Subject gateways
Fiona McLean
bmj.com, 10 Apr 2001 [Full text]
Access jobs at BMJ Careers
Whats new online at Student BMJ