BMJ  2008;336:655-657 (22 March), doi:10.1136/bmj.39482.526713.BE (published 21 February 2008)

Research

Prediction of citation counts for clinical articles at two years using data available within three weeks of publication: retrospective cohort study

Cynthia Lokker, research associate1, K Ann McKibbon, associate professor1, R James McKinlay, data analyst1, Nancy L Wilczynski, research manager1, R Brian Haynes, professor1

1 Health Information Research Unit, Department of Clinical Epidemiology and Biostatistics, McMaster University Faculty of Health Sciences, Hamilton, ON, Canada L8N 3Z5

Correspondence to: C Lokker lokkerc{at}mcmaster.ca

Objective To determine if citation counts at two years could be predicted for clinical articles that pass basic criteria for critical appraisal using data within three weeks of publication from external sources and an online article rating service.

Design Retrospective cohort study.

Setting Online rating service, Canada.

Participants 1274 articles from 105 journals published from January to June 2005, randomly divided into a 60:40 split to provide derivation and validation datasets.

Main outcome measures 20 article and journal features, including ratings of clinical relevance and newsworthiness, routinely collected by the McMaster online rating of evidence system, compared with citation counts at two years.

Results The derivation analysis showed that the regression equation accounted for 60% of the variation (R2=0.60, 95% confidence interval 0.538 to 0.629). This model applied to the validation dataset gave a similar prediction (R2=0.56, 0.476 to 0.596, shrinkage 0.04; shrinkage measures how well the derived equation matches data from the validation dataset). Cited articles in the top half and top third were predicted with 83% and 61% sensitivity and 72% and 82% specificity. Higher citations were predicted by indexing in numerous databases; number of authors; abstraction in synoptic journals; clinical relevance scores; number of cited references; and original, multicentred, and therapy articles from journals with a greater proportion of articles abstracted.

Conclusion Citation counts can be reliably predicted at two years using data within three weeks of publication.


Add to CiteULike CiteULike   Add to Complore Complore   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us   Add to Digg Digg   Add to Reddit Reddit   Add to StumbleUpon StumbleUpon   Add to Technorati Technorati    What's this?

Relevant Articles

Validating prediction models
Ewout W Steyerberg and Hester F Lingsma
BMJ 2008 336: 789. [Extract] [Full Text] [PDF]

Relation between online "hit counts" and subsequent citations: prospective study of research papers in the BMJ
Thomas V Perneger
BMJ 2004 329: 546-547. [Extract] [Full Text] [PDF]

This article has been cited by other articles:

  • Holden, G., Rosenberg, G., Barker, K., Lioi, J. (2010). Research on Social Work Practice: A Bibliometric Evaluation of the First Decade. Research on Social Work Practice 20: 11-20 [Abstract]  
  • Steyerberg, E. W, Lingsma, H. F (2008). Validating prediction models. BMJ 336: 789-789 [Full text]  

Rapid Responses:

Read all Rapid Responses

Validation of a prediction model: what’s in a name?
Ewout W Steyerberg, et al.
bmj.com, 29 Mar 2008 [Full text]
Validating Metrics, Early and Late
Stevan Harnad
bmj.com, 21 Apr 2008 [Full text]
Prediction of citation counts: a comparison of results from alternative statistical models.
Cynthia Lokker, et al.
bmj.com, 9 Sep 2008 [Full text]
Access jobs at BMJ Careers
Whats new online at Student BMJ