Rapid Responses to:
|
|
Rapid Responses: Rapid Responses published:
-
![[Read Rapid Response]](/icons/misc/sectionDOWN.gif)
Open peer review
- D F P Larkin (8 January 1999)
-
Opening up BMJ peer review
- Raj Bhopal (12 January 1999)
-
Open peer review system already exists, albeit, post publication
- Babatunde A Gbolade (18 January 1999)
-
Re: Open peer review system already exists, albeit, post publication
- Vincent V Richman (25 February 1999)
-
Evaluating a change in peer review
- Annette Leclerc (27 February 1999)
-
Peer Review & Clinical Negligence
- Andrew Reid (14 November 2000)
-
Editor's reply
- Richard Smith (14 November 2000)
-
Re:Institutional and geographical bias in peer review system
- Niranjan Bhattacharya (31 October 2001)
-
Re: Opening up BMJ peer review
- M Justin S Zaman (17 May 2007)
|
|
|||
|
D F P Larkin
Send response to journal:
|
Dear Sir On reading your editorial announcing the BMJ policy of identifying reviewers to authors of papers [1], I was struck by the absence of evidence supporting this change - notably in the accompanying paper by van Rooyen and co-authors (of which you were one)[2]. Although as you suggest, abusive or destructive reviews in a cloak of anonymity are unacceptable, is an experienced editor not likely to detect these? Nor is another abuse of anonymity which you identify, theft of ideas, enough to justify opening up peer review on a large scale. This abuse could be detected or prevented by a number of controls. The trial by van Rooyen and co-authors found that open peer review had no effect on the quality of the review or recommendation regarding publication. However it was found that it increased the likelihood of chosen reviewers declining to review. As you speculate, this consequence of open peer review may mean that junior researchers, the reviewers that consistently are found to give the 'best' opinions [3,4], will be reluctant to referee papers by senior researchers or, in my view worse still, review but not criticise them. Your change to open peer review is not an evidence -based change in practice. Editors, authors and readers benefit from rigorous and fair reviews, but efforts to obtain them may be frustrated and not served by the transparency that you champion. Yours sincerely Frank Larkin Consultant Ophthalmic Surgeon Moorfields Eye Hospital City Road London EC1V 2PD References 1 Smith R. Opening up BMJ peer review. BMJ 1999;318:4-5 2 van Rooyen S, Godlee F, Evans S, Black N, Smith R. Effect of opening up peer review on quality of reviews and on reviewers' recommendations: a randomised trial. BMJ 1999;318:23-7 3 Black N, van Rooyen S, Godlee F, Smith R, Evans S. What makes a good reviewer and a good review for a general medical journal? JAMA 1998;280:231-3 4 Goldbeck-Wood S. Evidence on peer review - scientific quality control or smokescreen? BMJ 1999;318:44-5 |
|||
|
|
|||
|
Raj Bhopal
Send response to journal:
|
Writers and reviewers will be watching your experiment carefully. You write, "the main argument against open peer review - a sad one - is that junior reviewers will be reluctant to criticize the work of senior researchers for fear of reprisals." The short story by Montague James "Casting the Runes" is required reading for your prospective reviewers, but let us trust that it will not implant ideas in the minds of those whose papers are rejected. In Montague's short story, based on the authority of peer review the Council of a Learned Association rejects Mr Karswell's paper "The Truth of Alchemy". On appeal the writer was assured by the secretary of the Association that the fullest consideration was given to the submitted draft, and that it was declined on the judgement of a most competent authority, whose name it was impossible for the Association to reveal. On second appeal, which requested the name of the reviewer, the secretary refused the request and closed the correspondence. Sadly, as you know writers can guess or discover the identity of reviewers. Mr Karswell's earlier book "A history of witchcraft" had been rejected by Mr John Harrington FRS, a scholar found dead three months after his review, having fallen off a tree in mysterious circumstances. Mr Karswell quickly identified Mr Dunning as the reviewer of "The Truth of Alchemy". Strange events soon overtook Mr Dunning. Black magic was at work. Within days he was a nervous wreck, who dared not go home or his place of study, the museum, for fear that Karswell might turn up there. His appearance became forlorn, his conversation empty. Clearly, Harrington's fate awaited Dunning. Clearly, he had misjudged the truth of the content of the paper! This story of inadvertent `open' scientific peer review had a murderous end. I shall not spoil the story by telling you and BMJ readers who murdered who. I hope for a less macabre outcome of your bold experiment. Yours sincerely Prof Raj Bhopal Department of Epidemiology and Public Health, School of Health Sciences, The Medical School, University of Newcastle upon Tyne, Framlington Place, Newcastle NE2 4HH References 1.Smith R, Opening up BMJ peer review, BMJ, 1999: (318); 4-5 2.James MR, Casting the runes (from, Ghost stories of an antiquary, 1911). Reprinted in, The Oxford Library of Classic English Short Stories Volume 1. Guild Publishing, London, 1989. |
|||
|
|
|||
|
Babatunde A Gbolade, Consultant Gynaecologist & Director of Fertility Control Unit St. James's University Hospital, Beckett Street, Leeds LS9 7TF
Send response to journal:
|
Editor - Richard Smith’s editorial[1] failed to recognise that an open peer
review system, with reviewers known to the authors, has existed since the
advent of paper-based publications. Although this system occurs post
publication, it is in no way inferior to the pre-publication closed system
and results from wider exposure to the scientific world.
I am referring to correspondences to editors, which in the form of critical comments on specific articles form important and integral parts of most journals. Those that get published (at least in the BMJ) are usually original, and contain assertions supported by data or citation.[2] Collective appraisal of articles in the process of acquiring critical appraisal skills can be developed and enhanced in the forum of a journal club.[3] The importance of the journal club in the unofficial peer review process is highlighted by Sandifer et al.[4] In their evaluation of their journal club's activities in the first six months of its existence, the impact on commissioning policy and the publication of letters to the editor of the journal from which the articles were selected were used as proxy outcomes. Six out of ten letters generated after collective appraisal were published. One of the cardinal rules for successful critical appraisal of articles is to avoid prejudging the articles on account of reputation, source, authors or preconceptions or bias. In the ambience of a journal club, this condition is easily met because of the variation in the composition of its membership. Criticism of any article is therefore more likely to be based purely on scientific principles. A letter to the editor in response to an article is therefore less prone to the criticisms of the closed peer review system outlined in Smith’s editorial. Such letters at times identify gross defects in studies that have been missed by a journal’s peer reviewers and may help in identifying instances of scientific fraud. Such letters tend to be courteous because the writers know that the authors of the original articles have the last word. The writers (junior or senior) of these letters put their names and addresses at the end of the correspondences. If these “unofficial reviewers” are not afraid of identification and therefore “reprisals”, why should the “official reviewers”? My premise is that an open peer review system already exists, albeit, as a post publication exercise. All that remains is for it to be used more widely pre-publication. Babatunde A Gbolade Consultant Gynaecologist & Director of Fertility Control Unit St James’s University Hospital Beckett Street, Leeds LS9 7TF E-mail B.A.Gbolade@leed.ac.uk References 1 Smith R. Opening up BMJ peer review. A beginning that should lead to complete transparency. BMJ 1999; 318:4-5. 2 BMJ. Advice to authors. BMJ 1997; 314:370. 3 Sackett DL, Haynes RB, Guyatt GH, Tugwell P. Clinical epidemiology: a basic science for clinical medicine. 2nd edition. Boston: Little, Brown and Company, 1991:398-418 4 Sandifer QD, Lo S, Crompton G. Evaluation of a journal club as a forum to practice critical appraisal skills. J R Coll Physic Lond 1996; 30: 520-22. |
|||
|
|
|||
|
Vincent V Richman, Research Associate AlgoPlus Consulting Limited
Send response to journal:
|
Rejected Letters to the Editor and Independent Audit Committees: Dear Sir: Mr. Gbolade [1] states that an open peer review system exists after publication. There is a potential problem in the case of letters to the editor that identify failures of the editorial process and raise issues about managerial competence. Such letters present conflicts of interest to the journal management. Currently, there is no recourse for the writer whose rejected letter raised fundamental concerns about the validity of published articles. A standard practice of both public and private organizations is to appoint independent audit committees to review the disputed decisions of management. I propose that medical journals have a similar mechanism for writers of rejected letters that have questioned the editorial process by identifying gross defects within articles. Even if, as I expect, the majority of the letters have been appropriately rejected, the independent review is needed. Yours sincerely, Vincent V. Richman MBA PhD Research Associate, AlgoPlus Consulting Limited, Suite 502, 5675 Spring Garden Road, Halifax, Canada B3J 1H1 Email: vrichman@compuserve.com References: [1] Gbolade BA, Open peer review system already exists, albeit, post publication, eBMJ, 17 Jan 1999. |
|||
|
|
|||
|
Annette Leclerc
Send response to journal:
|
EDITOR - The BMJ plans to let the authors know the identity of the reviewers (1). The decision is supported by a randomized trial on 125 manuscripts, and a few other scientific studies in which the main outcome measure was the quality of the review (2,3). There are clear arguments against close peer review ; I am not sure that all the arguments against open peer review have been considered, especially concerning the relationships within the scientific community, at a national and international level. Why not tackle this question with an evaluative epidemiology approach, as if it was, let us say, evaluation of an ergonomic or psychosocial change in a working environment ? It seems that it is too late for performing a before-after comparison based on the relevant dimensions of the possible effects of the change. For some of the dimensions ( for example, recommandations concerning publication ) a retrospective evaluation would be possible, but we know that retrospective evaluation is a source of bias. It seems also difficult (and it woud be ineffective) to randomise the scientific community to two groups, open and close peer review. Randomisation through randomisation of scientific journals is also unrealistic. At least it woud be possible to have comparisons based on a natural experiment ; this remains feasible as far as some scientific journals keep closed peer review, that these journals are comparable to those which have changed to open peer review, and that the choice of a journal for submission by the authors is independant of whether the peer review will be open or not. All the relevant dimensions of the possible effects of the change have to be defined ; the range is large, since it includes effects on the manuscript, on the authors and reviewers, and on the scientific community as a whole. Quality is probably the most important point for the journals ; for authors and reviewers, open peer review can be seen as a threat to satisfactory relationships with their peers. Whether the threat is real or not remains to be evaluated. Evaluating an intervention is most often a complicated task, which may need the help of specialists from several disciplines ; the situation here is not much more difficult than evaluating other kinds of intervention. One may consider that the treatment of manuscripts is unimportant : after all, if it is suboptimal, nobody will die ; but if it deserves some interest, then widening the debate in the scientific community, beyond specialists of peer-review, might be beneficial. It woud also alleviate the task of peer review researchers. Annette Leclerc, epidemiologist researcher at INSERM Unit 88 HNSM 14 rue du val d'Osne 94410 St-Maurice France (1) Smith R. Opening up BMJ peer review. BMJ 1999 ; 318 :4-5. (2) Van Rooyen S, Goodlee F, Evans S, Black N, Smith R. Effect of open peer review on quality of reviews and on reviewers'recommandations : a randomised trial. BMJ 1999, 318 :23-27. (3) Goldbeck-Wood S. Evidence on peer review - scientific quality control or smokescreen? BMJ 1999, 318 :44-45. |
|||
|
|
|||
|
Andrew Reid, NHS regional Medicolegal adviser & solicitor Peterborough District Hospitals NHS Trust
Send response to journal:
|
On what conditions of confidentiality are authors sent the comments of reviewers? Are there any conditions or guidance about the submitting of a case report which is also the subject of claim for clinical negligence? Should a journal object to lawyers seeing the comments? There are legal arguements that a patient claiming clinical negligence can obtain disclosure of all documents created for the purposes of publication of their case as their dominant purpose is not for litigation. Reviewers comments cannot be a substitute for a medicolegal opinion but may be evidence of a responsible body of opinion relevant to the issues in both the case report & the litigation. What is the BMJ's position? |
|||
|
|
|||
|
Richard Smith, Editor, BMJ
Send response to journal:
|
The BMJ does not attach any conditions to the reviewers' comments that we send to authors. Perhaps we should, but we have not so far experienced any sort of problem. We do not publish case reports without the consent of the patient. Any patient who is making a claim of clinical negligence would thus know that a case report might be published. If he or she consented, then there would be no problem from our point of view. If he or she did not consent, then we wouldn't publish the paper. It's hard to imagine that a patient would consent to the publication of a report when a case of negligence is pending. Probably the patient's lawyers would advise against publication. Richard Smith
|
|||
|
|
|||
|
Niranjan Bhattacharya, Surgeon and Superintendent Vijaygarh State Hospital ,Calcutta,India Vijaygarh State Hospital,Calcutta-700029,India
Send response to journal:
|
Dear Sir, Thank you very much for a very interesting article on open peer review system in BMJ by Richard Smith 1999;318:4-5.With progressive globalisation of the search of truth in the medical arena,we should rise progressively above the institutional ,regional and other forms of human biases that may often gag the expression of medical science,in the present days of e-media .For example,the wisdom of treatment of non eltor mutagenic strain of vibrio cholera in Calcutta is possibly more than the clinical counterpart of Bayler college of Washington ,similarly teloisomerase and cellular senescence and its implications would possibly be better understood by the scientists of the Bayler College.Hence I have a humble suggestion ;Is it possible to strike out the name and the place of work of the authors before we send the papers for peer review in BMJ? Sincerely Yours Dr Niranjan Bhattacharya MBBS,MD,MS,FACS(USA),Dr Sanjukta Bhattacharya PhD,Dr Mahua BhattacharyaMBBS,DA,DGO,Dr Kanailal Mukherjee MBBS,Phd,PhD. |
|||
|
|
|||
|
M Justin S Zaman, Research Fellow in Epidemiology/Specialist Registrar in Cardiology University College London
Send response to journal:
|
Raj Though this is a late response to your note, I feel it is worth getting it 'on air'. It is well-known that junior doctors often do not complain over working conditions for fear of a bad reference. Sure, some things are worth complaining about - MTAS in 2007 - but there is no discernible disadvantage to complaining about this, as senior doctors hate it too! At many ivory tower teaching hospitals across the land, I am sure a generation of junior doctors worked in exploitative conditions just to get the 'golden circuit' reference. Analagous to this, junior researchers will NOT give open-minded reviews on the peer-review process if identified. I would argue for a DOUBLE-BLIND process - with the authors of the report being anonymous too. This would prevent over-zealous critiscm, but equally would prevent reviewers from assuming that the work must be good because the author/institution is of a high status. Your story was thus highly pertinent! Competing interests: None declared |
|||