The Cochrane Collaboration at 20BMJ 2013; 347 doi: https://doi.org/10.1136/bmj.f7383 (Published 18 December 2013) Cite this as: BMJ 2013;347:f7383
All rapid responses
Is Richard spouting mince pie once again?
The evidence for his statement "... of 358 questions asked in dermatology only three could be answered by a single systematic review" amounts to a sentence in Jon Bassey's blog (Brassey J. A critique of the Cochrane Collaboration. Trip2013. http://blog.tripdatabase.com/2013/04/a-critique-of-cochrane-collaboratio...) stating the same thing.
I would love to be able to examine Jon's study methods and results a bit deeper.
A priori Jon's conclusions conflict with conclusions from our 2004 study when my lab found that "although few (3%) entries in review databases currently address dermatology topics, entries address many (8/10) of the top 10 dermatology diagnoses. Contrary to popular belief, a substantial number (40%) of such reviews report sufficient evidence to inform clinical decisions making." (http://www.jaad.org/article/S0190-9622(03)00883-1)
Competing interests: Editor for the Cochrane Skin Group
While it's lovely to get mentioned in the above article I was slightly disappointed that my third mention mischaracterises my thinking. Richard states "Brassey suggests that it might be possible to depend simply on the best trial or trials reported in core journals."
If that interpretation came from my article then it's down to my clumsy editing of a longer article I wrote. The section of the article Richard comments on above started with an introductory sentence stating:
“In recent years there have been a number of articles that have suggested, to differing degrees, that doing things more quickly can give you the same or similar results to the Cochrane methodology.”
The point being, can we take short-cuts to arrive at robust answers? This is a vitally important question as there is an absolute need for the costs associated with systematic review production to be dramatically reduced. Using the largest trial, or articles from a sample of publications, can give us real hints at the likely outcome of a fuller systematic review. But this area is poorly researched. Frustratingly much research in this area seems to increase the cost of production, weeding out small amounts of bias for a disproportionately large amount of effort. I sometimes wonder (at my most cynical) that this presents an advantage for Cochrane; longer more methodologically intensive systematic reviews increase the barriers to entry for competitors. After all, if you could do a systematic review with the press of a button where would that leave Cochrane’s business model?
But that is, given the time of year, highly uncharitable and should not detract from the amazing efforts of the founders and volunteers at Cochrane.
Competing interests: I run, and am a shareholder in, the Trip Database (www.tripdatabase.com). I am also working on systems and methods to reduce the cost of systematic review production (e.g. http://blog.tripdatabase.com/2013/10/trip-rapid-reviews-systematic-reviews.html)