Intended for healthcare professionals

Analysis

Report of the independent panel considering the retraction of two articles in The BMJ

BMJ 2014; 349 doi: https://doi.org/10.1136/bmj.g5176 (Published 15 August 2014) Cite this as: BMJ 2014;349:g5176

This article has corrections. Please see:

  1. Iona Heath, panel chair and past president, Royal College of General Practitioners,
  2. Stephen Evans, professor of pharmacoepidemiology, London School of Hygiene and Tropical Medicine,
  3. Curt Furberg, professor emeritus of public health sciences, Wake Forest University School of Medicine,
  4. Julia Hippisley-Cox, professor of epidemiology and general practice, University of Nottingham,
  5. Harlan Krumholz, Harold H Hines Jr professor of medicine (cardiology) and professor of investigative medicine and of public health, Yale School of Medicine,
  6. Cynthia Mulrow, senior deputy editor, Annals of Internal Medicine,
  7. Paul Wicks, vice president of innovation, Patients Like Me

In May The BMJ corrected an error relating to adverse effects of statins in two articles but was asked to retract the articles. The editor referred the decision to an expert panel. Here, we publish its findings

In October 2013 The BMJ published an Analysis article by Abramson et al1 arguing that cholesterol lowering guidelines should not be widened to include statin therapy for low risk individuals (five year risk <10%) and an Observations article by Malhotra2 suggesting that saturated fat is not the main cause of cardiovascular disease. The Abramson et al article1 questioned the balance of risk and benefit presented in the recently updated Cochrane review3 and the 2012 Cholesterol Treatment Trialists’ (CTT) Collaboration meta-analysis4 (on which the updates to the 2013 Cochrane review3 are largely based). Both articles quoted an article by Zhang et al5 to claim that the rate of side effects with statins was around 20%. This was an error. In fact, Zhang et al5 referred to “statin-related clinical events that may be interpreted as adverse reactions by patients or their clinicians.” As Zhang et al themselves pointed out in a rapid response to Abramson et al’s article, “implicit in this definition is the recognition that the causative association between each identified event and statin use was unknown.”6

This error of interpretation was first suggested in a rapid response from Takhar7 immediately after publication of the Abramson et al article1 and subsequently clarified by Zhang et al themselves in a letter published in June 2014.8 Numerous rapid responses were posted, reflecting a vigorous debate on the merits and limitations of statins for those at low risk of cardiovascular disease.

On 30 October 2013, a few days after publication, Rory Collins, professor of medicine and epidemiology at the Clinical Trial Service Unit at Oxford University and an author on the meta-analysis by the Cholesterol Treatment Trialists’ (CTT) Collaboration published in the Lancet in 20124, sent an email to the editor of The BMJ, Fiona Godlee, stating that the journal seemed to have taken a stand against statins and that there was a danger that misrepresentation of the evidence in The BMJ could cause harm. He discussed this in person with Fiona Godlee in December 2013 and talked her through a set of slides (later submitted to the panel with additional annotations, SP16a on bmj.com). At that meeting Godlee invited Collins to write an article presenting evidence on the benefits and harms of statins: “Although your article would be a response to the two articles, and to Abramson et al in particular, it would be helpful if you could use the opportunity to set your piece in the wider context of the evidence on the benefits and harms of statins.” (See SP13, email 2 December). Following this discussion Collins submitted a number of written but not for publication criticisms to Godlee, focused mainly on the Abramson et al article, and was again invited to write an article in response. At the time of this report he had not yet done so in the form of a submitted article.

In another letter to Godlee, marked “not for publication” and dated 28 April 2014, Collins called for retraction of both articles, writing: “What the BMJ needs to do is withdraw these seriously damaging claims explicitly and unreservedly with a clear explanation of why they are so wrong and what is likely be correct, and to demonstrate that it is serious about rectifying the damage that it has caused by retracting both of these papers.” He emphasised the seriousness of his concerns, describing “the need to rectify the harm that has been caused—perhaps resulting in large numbers of unnecessary deaths, heart attacks and strokes among patients at elevated risk—by misleading doctors and the public with gross over-estimates of the rates of side-effects with statins.” (SP20)

On 15 May 2014, corrections were posted for both articles, withdrawing the statement that side effects of statins occur in about 18-20% of patients.9 10 “The authors withdraw this statement. Although it was based on statements in the referenced observational study by Zhang and colleagues, that ‘the rate of reported statin-related events to statins was nearly 18%,’ the article did not reflect necessary caveats and did not take sufficient account of the uncontrolled nature of the study.” Godlee highlighted the corrections in an editorial on 15 May 2014.11

In response to the request for retraction, Godlee set up an independent panel to consider the question of retraction and to review the processing of the articles (www.bmj.com/content/independent-statins-review-terms-of-reference).

She provided the panel with the following terms of reference, outlining the specific tasks to be undertaken:

  • To consider whether either or both articles should be retracted

  • To review and comment on the process by which the articles were published

  • To review and comment on how criticisms and complaints against the articles were raised and how the journal responded

  • To summarise its findings and make recommendations to The BMJ’s editor in chief in a report that will be published on bmj.com.

Independence

The panel members were invited to contribute by Godlee and all have some connection with The BMJ. All are committed to maintaining the journal’s reputation for scientific integrity through the correction of errors and, if necessary, through retraction. The members of the panel have had no contact with any of The BMJ staff about its deliberations. The journal has not sought to influence the panel’s discussions nor had prior notice of the findings before submission of this the final report on 31 July 2014. The potential conflicts of interest of all the members of the panel, including roles in The BMJ, have been declared and are appended to this report (SP28). The members were not offered and have not received any payment for their service on the panel.

Grounds for retraction

The panel decided to use the 2009 Retraction Guidelines issued by the Committee on Publication Ethics (COPE)12:

Journal editors should consider retracting a publication if:

  • They have clear evidence that the findings are unreliable, either as a result of misconduct (eg data fabrication) or honest error (eg miscalculation or experimental error)

  • The findings have previously been published elsewhere without proper cross referencing, permission or justification (that is, cases of redundant publication)

  • It constitutes plagiarism

  • It reports unethical research.

COPE also states that “Retraction is a mechanism for correcting the literature and alerting readers to publications that contain such seriously flawed or erroneous data that their findings and conclusions cannot be relied upon. Unreliable data may result from honest error or from research misconduct. Retractions are also used to alert readers to cases of redundant publication (i.e. when authors present the same data in several publications), plagiarism, and failure to disclose a major competing interest likely to influence interpretations or recommendations. The main purpose of retractions is to correct the literature and ensure its integrity rather than to punish authors who misbehave.”12

The panel has not found any guidelines for the retraction of opinion pieces and could not identify any precedent for retracting an opinion piece unless it was based on fraudulent data. The panel noted that retraction has almost always been reserved for papers where original data are presented.

General approach

Collins first requested retraction of the two articles in his letter to Godlee of 31 March 2014 (SP17), six months after their publication. In his formal submission to the panel, Collins stated the following: “Such serious misrepresentations of the evidence seem likely to lead to people who are at elevated risk of heart attacks and strokes stopping their statin therapy or not starting it in the first place. As a consequence, this may well result in unnecessary heart attacks, strokes and vascular deaths. With regard to the issue of offering statins to people who are at the lower end of the risk spectrum, such misinformation about side-effect rates would prevent them from making a properly informed choice, although the impact is likely to be less catastrophic.” (SP15)

The panel recognised that Analysis and Observations pieces should provide latitude for scientists to interpret the available data but acknowledged that wrong information should not be presented. The question the panel set out to answer was whether the arguments provided in the two articles were misleading in a way that was likely to cause harm or whether they represented alternative views that are reasonable and based on a plausible interpretation of what is known.

The panel also noted that the individual patient level data for the relevant trials are held in confidence by the investigators and have not yet been made available for public scrutiny by those who question the interpretations of the trialists.

The panel asserted the importance of not discouraging scientific debate because of disagreements of interpretation or opinion. Nevertheless, distortions of the evidence that amount to frank misrepresentations should be corrected and if extreme in nature, retracted. Journals need to protect the rights of people to express opinions and debate the evidence while being steadfast in not allowing facts to be misrepresented.

It is important to note that the panel has not been asked to pass judgment on the risks and benefits of statins per se, nor on the appropriate use of statin medication in low risk individuals. Instead the panel has been asked to decide whether there are sufficient grounds to require retraction of one or both of the articles from the scientific literature. The panel has been at pains not to take sides and not to support one view at the expense of another.

The panel was not asked to address the question of whether a paper on the use of statins in people at low risk might influence patients at high risk to stop their statins unnecessarily. The question of how patients react to “scares” is complex and controversy in the media can feed such scares.

Nonetheless the panel also recognised that attempts to exclude patients and citizens from scientific debate because of the possibility that they might be harmed are not usually welcomed by the patients and citizens affected.

Evidence received

The panel requested and received submissions from Collins as the complainant (SP15-21) and from the authors of the two articles (SP22-24). The panel also considered all the rapid responses (up until 15 July 2014) relating to the original articles1 2 and to Godlee’s editorial11 and wish to express their appreciation of all those who contributed to the open debate.

Thirteen researchers sent private submissions to the panel (SP25). The chair of the panel invited the authors of each submission to post it as a rapid response to the original article or the editorial about the panel’s establishment in The BMJ but most declined, preferring to have their submissions published at the time the panel reported. These submissions have been subject to normal legal prepublication checking, and some statements have been redacted for legal reasons, but no substantive changes have been made to the arguments presented.

The panel also reviewed the original CTT paper,4 the 2011 Cochrane review,13 the updated Cochrane review from 2013,3 all the references cited in the two articles under investigation, and other relevant papers.

Accompanying documents

Items with bold numbers indicate accompanying documents that are available at www.bmj.com/content/independent-statins-review-panel-report-0. Other numbers simply link via the indicated URL.

1. Submissions to the Panel
From the BMJ
From Rory Collins and the authors
  • SP15 Note from Rory Collins for the panel

  • SP16 Supplement to note from Rory Collins for the panel

  • SP16a Annotated slides

  • SP16b Annotated version of Abramson et al (annotations by Rory Collins)

  • SP16c Annotated version of Malhotra (annotations by Rory Collins)

  • SP17 Letter to Fiona Godlee 31 March 2014, not for publication

  • SP18 Letter to Fiona Godlee 14 April 2014, not for publication

  • SP19 Letter to Fiona Godlee 25 April 2014, not for publication

  • SP20 Letter to Fiona Godlee 28 April 2014, not for publication

  • SP21 Grants to CTSU

  • SP22 Initial Response to Rory Collins’ submission from Abramson et al

  • SP23 Response to Rory Collins’ submission from Abramson et al

  • SP24 Response to Rory Collins’ submission from Malhotra

From others
  • SP25 Submissions from 13 individuals or groups

  • SP26 Declarations of interests by the authors of submissions (see SP25)

2. Panel documents
  • SP27 List of documents accompanying the report

  • SP28 Declarations of interests of panel members

  • SP29 Article by Abramson et al annotated by panel to identify statements that need statistical review

  • SP30 Proforma for statistical review of Abramson et al

  • SP31 Statistical report on Abramson et al by Stephen Evans

  • SP32 Statistical report on Abramson et al by Carol Coupland

To consider whether either or both articles should be retracted

Abramson, Rosenberg, Jewell and Wright article

The panel recognised that this was published as an Analysis article,1 which implies some degree of interpretation and opinion.14 It is not an original research article or a systematic review and is designed to provide a commentary on the existing literature.

The panel found no suggestion of data fabrication in the complaint or in the rapid responses. The panel also noted the correction that had been published before the panel was convened. The panel therefore decided to concentrate on the issue of honest error and the possibility of miscalculation in the article. It aimed to answer the following questions:

  • Whether any inaccuracies, taken as a whole, rose to the level of outweighing what was accurate in the article

  • Whether the authors had distorted the evidence available in such a way that the article (even with the current corrections) was so misleading it should be retracted.

Statistical review

The panel agreed to undertake a statistical check on each numerical statement in the Abramson et al article. This was to be assessed by two people independently: the statistician on the panel (SE) and an external statistician. One of the panel members (JH-C) proposed Carol Coupland (CC), associate professor and reader in medical statistics at the University of Nottingham, as an additional statistician with relevant skills and expertise in the methods and subject matter under consideration. The article was annotated to highlight each numerical statement (SP29). A proforma relating to each specific numerical statement was jointly developed by SE and CC and endorsed by the panel (SP30). Each statement was to be categorised as follows:

  • A to mean definitely justified on the basis of the quoted evidence.

  • B to indicate that the point does not necessarily agree with the quoted sources but where it is a matter of judgement and could be justified.

  • C to mean clearly wrong and definitely misrepresenting the quoted sources.

The calculations in the Abramson et al article were then independently reviewed by the two statisticians with neither seeing the submission of the other until both reports had been completed (SP31 and 32). Their analyses were in substantial agreement. They found that the numbers in the article by Abramson et al1 were not clearly wrong, except for the already acknowledged misrepresentation of Zhang et al.5 11 The authors’ use of observational data in regard to muscle and other problems is contentious but remains a matter of opinion. Abramson et al also state “no difference” when they mean no statistically significant difference,1 and in the view of the panel, this is potentially misleading. However, this sort of thing occurs so frequently in the medical literature that it cannot be considered grounds for retraction.

Findings

On the basis of the statistical reviews and their own reading of the background papers, the panel concluded that the calculations in the article had been largely sound. There were very minor differences between the two statistical reports, and these were, at least in part, due to some lack of clarity in the original report of the CTT meta-analysis.4

There are two major areas of contention that are crucial to decisions to be made by both patients and their clinicians: the effect of statins on all cause mortality when used in primary prevention in low risk populations, and the incidence of adverse events.

All cause mortality—A recent editorial by Vinay Prasad in Annals of Internal Medicine15 illustrates a fundamental problem that has consistently concerned the panel. Prasad compared two meta-analyses of statins in primary prevention3 16 that differed in their statistical conclusions by less than half a percentage point and yet reached opposite conclusions—namely that that “statins reduce . . . total mortality” or conversely that “data . . . showed no reduction in mortality associated with treatment with statins.” Unfortunately, patients and clinicians have to make decisions in the grey area between these two diametrically opposed conclusions. The panel supports Prasad’s contention that “The Cholesterol Treatment Trialists’ study has a robust set of de-identified individual-patient data, which can improve our understanding, and those data should be made widely available.”15

Adverse events—Adverse events may be coincidental or associated with the disease being treated and therefore not true adverse reactions (or side effects or adverse effects) that are directly caused by the treatment. Numerous rapid responses underlined the discrepancy between the incidence of debilitating adverse events experienced by patients seen in clinical practice, who believe those events to be caused by their statin, and those reported as statistically significant differences in randomised clinical trials. In their detailed response to the panel, Abramson et al noted a number of possible explanations for this discrepancy. These include the exclusion of people with substantial multiple comorbidity from randomised clinical trials and the further exclusion of those who report early adverse effects during the run-in periods such as occurred in the Heart Protection Study.17 The report of this trial noted that this strategy could raise “questions about the generalisability of the safety analyses because patients may have withdrawn during the run-in due to adverse effects caused by simvastatin.”

The panel is also aware of the history of serious adverse effects that were not originally reported in randomised clinical trials being revealed by subsequent meta- analyses18 or postmarketing surveillance.19 The panel has no doubt that observational studies have a role in the identification of both adverse events (associated with the taking of a medication) and true adverse reactions or side effects (known to be caused by a medication).

There have been suggestions from all parties to the current dispute that “myopathy” may have been conflated with “muscle pain” (myalgia) and this might lead to confusion. The panel agreed that this is difficult area since there is no widely accepted standard definition for “statin associated musculoskeletal adverse events,” whereas there are consensus statements for the definition of harms relating to the assessment of some other clinical outcomes.20 The CTT 2012 paper4 cited a review by Armitage in 2007,21 both for its figures on myopathy and, by inference, the definition of myopathy. Armitage defined myopathy (and myositis) as “any muscle symptom—pain, tenderness or weakness—accompanied by a creatine kinase concentration greater than ten times the upper limit of normal for the laboratory.”21 Armitage also stated that myalgia “referred to muscle pain with no rise in creatine kinase concentration to greater than ten times the upper limit of normal.”

One source of confusion is that Abramson et al reported three different outcomes in their article, all under a heading of “myopathy.”1 These were “myopathy” and the information presented was directly quoted from the CTT 20124; “muscle pains” as reported in the National Health and Nutrition Examination Survey22; and a retrospective cohort study reporting “incidence of musculoskeletal disorders overall and injuries.”23 The panel thought that including three different definitions of muscle problems, widely ranging in severity, all under a heading of the more serious myopathy, might lead to the reader to conflate these. However, as Abramson et al point out in their submission to the panel (SP23), myopathy and myalgia can be conflated in the opposite direction by referring to severe problems as if they included milder ones and this can also lead to misinterpretation.

Conclusion

The panel concluded that the only unequivocal error in the article by Abramson et al1 is the misrepresentation of the Zhang et al paper5 and considered this to be insufficient to justify retraction of the whole article. Nothing in the article suggested that the authors had acted malevolently or fraudulently.

Panel’s recommendation

The panel finds that the Abramson et al1 article does not meet the COPE criteria for retraction.

Malhotra article

This is an Observations article with the strapline “From the heart,” communicating clearly that this was an opinion piece open to disagreement and debate.2 The main focus of the article is on the role of saturated fat in the aetiology of cardiovascular disease. Most of the criticism of the article from Collins concentrates on an error related to a secondary point. This error concerned the interpretation and misrepresentation of the Zhang et al paper,5 which is peripheral to the overall thrust and conclusion of the Malhotra article2 and which has now had a correction posted.10

Findings

Nothing in the article suggested misconduct or that the author had acted malevolently or fraudulently. Strong and iconoclastic opinions are expected in opinion pieces and can enhance open scientific debate.

Panel’s recommendation

The panel finds that the Malhotra article2 does not meet the COPE criteria for retraction.

To review and comment on the process by which the articles were published

The editor of The BMJ had already set in motion a review of the process and had asked for a timeline to be prepared. This was made freely available to the panel without comment (SP14).

Initial submission

The initial review process seemed appropriate and adequate. It was done in a timely and reasonably thoughtful and careful manner. Editors and reviewers noted several areas where the article was potentially confusing, slanted, or incorrect, and suggested several ways to improve its presentation.

Suggestions

Editors could consider developing guidance for internal use in relation to when Analysis articles that contain calculations and numerical extrapolations warrant statistical review.

Revision and publication

The review of the revision took about a month. Decision making about its acceptability was made by the Analysis editor without obtaining additional peer review, statistical review, or opinions from the associate editors who had discussed the original submission. The authors did not provide the requested detailed letter explaining their responses to editors’ and reviewers’ comments, but their revision did seem to address most of the comments and suggestions that had been made.

The Analysis editor thought that the revision’s “message is a little too strong for the data presented.” She placed queries in the manuscript asking for better specification and justification of some sentences and requested more information (a figure or number) for the harm side of the benefit-harm equation. A manuscript that was copy edited by a technical editor and included the editor’s queries was sent to authors. The authors, who responded within 1 to 2 days, added numbers about harm that they gleaned from the recently published article by Zhang et al,5 and the reference to that article, and changed the fact box. The technical editor judged the authors’ revisions acceptable.

The published manuscript included the authors’ various contributions to the article and their potential conflicts of interest and clearly identified in the introduction that the authors were presenting a particular point of view, “We argue that the evidence does not show that the benefits of statins in low risk patients outweigh the harms and that the advice for treatment of this group should not be changed.” Arguments that were presented were generally clear, factual, and referenced (including criticism of the limitations of trial data) except that the presentation and the interpretation of data about adverse events from the article by Zhang et al5 added to the manuscript at the copy editing stage was incorrect. Methods used for the extrapolation of total mortality data from the CTT meta-analysis4 were only explained in a footnote to the table, and limitations of those methods were not mentioned.

Suggestions

Senior or deputy editors should sign off revisions that include substantive additions to an article or changes in a “fact” box. Extra attention should be given to manuscripts that have been noted by reviewers and editors to be controversial and potentially slanted or one sided. Editors might consider whether such pieces are best placed (and clearly labelled) as opinion pieces or whether they are best placed and clearly labelled as articles/analyses that are meant to have a more comprehensive and, according to author instructions, “an even-handed approach in evaluating evidence, a lucid line of argument, and a worthwhile conclusion.”14 Although the article by Abramson et al was not press released, that by Malhotra, which was also clearly presenting a controversial view, was (http://group.bmj.com/group/media/latest- news/Time%20to%20bust%20the%20myth%20of%20saturated%20fat2019s%20role% 20in%20heart%20disease-%20says%20cardiologist.pdf). Press releases should be used with great care in such contexts.

Editors might give extra attention to the following issues when considering potentially controversial articles that postulate strong arguments: possible selective citing of material, failure to critically appraise evidence that is used to support authors’ arguments, and over criticism of evidence that does not support authors’ arguments.

Editors should carefully consider whether articles that include extrapolations and recalculations of numerical data need statistical review. They should also consider whether a article that has been revised requires additional peer review.

Rapid responses

The journal received several rapid responses that raised substantive criticisms and discussion of the Analysis article. The editors’ selections of rapid response letters that were directed to John Abramson for further response and discussion were appropriate. Editors’ decision making about those selections and their follow-up was executed in a timely manner. Abramson and colleagues responded to the selected letters promptly.

Corrections

The BMJ and all authors formally corrected statements in both articles and the summary box that said that side effects of statins occur in 18-20% of people, and stated that “The BMJ articles did not reflect necessary caveats and did not take sufficient account of the uncontrolled nature of Zhang and colleagues’ data.” Godlee wrote in her editorial announcing the corrections that “This editorial aims to alert readers, the media, and the public to the withdrawal of these statements so that patients who could benefit from statins are not wrongly deterred from starting or continuing treatment because of exaggerated concerns over side effects.”11 She also wrote that the initial submission of the article by Abramson et al included a reference to the Zhang article5 and a “misreading of Zhang and colleagues’ data that was not picked up by the peer reviewers or editors.” In fact, the Zhang reference5 was not included in the initial submission (it was added at the proof stage) and this statement was subsequently corrected.24 An alert drawing attention to the corrections was inserted. Subsequently, Zhang et al sent a rapid response that further clarifies the continuing misinterpretation of their article.6

Assessment

Editors used due diligence in assessing the necessity for corrections, and made important corrections that are clearly labelled. The statement “not picked up by peer reviewers” was an error which was also corrected.

Panel’s conclusion

The panel has made several suggestions aimed at improving the editorial process and was concerned about the late inclusion of an unscrutinised reference on a short timescale. However, the panel concedes that the peer review and editorial processes must rely on goodwill to a very considerable extent and can never be completely foolproof —especially in view of the time pressures under which authors, peer reviewers, and editors are working.

To review and comment on how criticisms and complaints against the articles were raised, and how the journal responded

The panel had access to all the correspondence between Collins and Godlee (SP13), in addition to the other submissions already noted.

Findings

The panel noted with concern that despite the editor’s repeated requests that Collins should put his criticisms in writing as a rapid response, a letter to the editor, or as a standalone article, all his submissions were clearly marked “not for publication.” The panel considered this unlikely to promote open scientific dialogue in the tradition of The BMJ.

The delay in responding to criticism from Collins was not from October 2013 to May 2014, as he stated publicly after the correction appeared on 15 May 2014, because in an email dated 30 January 2014 he indicated that he was preparing an article responding to Abramson et al (SP13); this had yet to be submitted at the time of the panel’s report. In a BBC Radio 4 Today interview on 21 March 2014 (SP14), he again referred to the fact that he was writing an article clarifying the evidence. He did not formally submit his concerns in writing to The BMJ until the end of March 2014 (SP17), by which time he had already complained about The BMJ’s delay in responding and had shared his views with the mainstream media. On this basis, the delay cannot be considered to be longer than from March 2014 to May 2014.

However, there was a delay from October 2013 to May 2014 in correcting the misinterpretation in relation to the Zhang et al5 citation, which was indicated in the rapid responses on the day the Abramson article1 appeared in print. Takhar7 wrote: “the final point in key messages box . . . is not conclusively backed up by the detailed evidence and references presented.” Abramson et al responded to Takhar in their own rapid response, and this may have reassured the editorial staff.

Panel’s conclusion

The BMJ editorial staff should implement a significant event audit in relation to the need for the correction. The aim of the audit would be to try to identify what would need to have been in place to ensure that the correction was made in a more timely fashion.

Summary of conclusions and recommendations

The parties to this latest controversy over the role of statin medication in the primary prevention of cardiovascular disease have different professional backgrounds and experience, which results in different perspectives, interpretations, and judgments.

Unbiased groups of scientific investigators analysing the same data can reach very different conclusions.

The panel was unanimous in its decision that the two articles do not meet any of the criteria for retraction. The error did not compromise the principal arguments being made in either of the articles. These arguments involve interpretations of available evidence and were deemed to be within the range of reasonable opinion among those who are debating the appropriate use of statins. In making this assessment, the panel is not expressing an opinion about the merits of these arguments, as that work was beyond the scope of the panel.

The panel did have one final comment. It became clear to the panel that the fact that the trial data upon which this controversy is based are held by the investigators and not available for independent assessment by others may contribute to some of the uncertainty about risks and benefits. Different investigators may come to different conclusions with the same data. In fact, a particularly germane example occurred recently in which two experienced Cochrane groups were charged with evaluating a particular intervention and, despite being given the same instructions, data, and resources, did not arrive at identical results or conclusions.25 26 The panel strongly believes that the current debates on the appropriate use of statins would be elevated and usefully informed by making available the individual patient level data that underpin the relevant studies

Notes

Cite this as: BMJ 2014;349:g5176

Footnotes

  • Panel contributors: The panel met five times between 20 May and 28 July and conducted business by email between the meetings. All seven members of the panel attended every meeting, either in person or by telephone. All panel members reviewed and agreed this final version of the report. Jane Smith provided secretariat support.

  • Competing interests: See data supplement on bmj.com

References

View Abstract