Intended for healthcare professionals

CCBYNC Open access
Research

Association between conflicts of interest and favourable recommendations in clinical guidelines, advisory committee reports, opinion pieces, and narrative reviews: systematic review

BMJ 2020; 371 doi: https://doi.org/10.1136/bmj.m4234 (Published 09 December 2020) Cite this as: BMJ 2020;371:m4234
  1. Camilla H Nejstgaard, doctoral student1 2 3 4,
  2. Lisa Bero, professor5,
  3. Asbjørn Hróbjartsson, professor1 2 3,
  4. Anders W Jørgensen, ENT specialist6,
  5. Karsten J Jørgensen, acting director4,
  6. Mary Le, general practitioner7,
  7. Andreas Lundh, senior researcher1 2 3 8
  1. 1Centre for Evidence-Based Medicine Odense (CEBMO), Odense University Hospital, Odense, Denmark
  2. 2Department of Clinical Research, University of Southern Denmark, Odense, Denmark
  3. 3Open Patient data Exploratory Network (OPEN), Odense University Hospital, Odense, Denmark
  4. 4Nordic Cochrane Centre, Rigshospitalet, Copenhagen, Denmark
  5. 5Center for Bioethics and Humanities, University of Colorado, CO, USA
  6. 6ENT Clinic Hobro, Hobro, Denmark
  7. 7Stasjonsgata Legekontor, Hokksund, Norway
  8. 8Department of Infectious Diseases, Hvidovre Hospital, Hvidovre, Denmark
  1. Correspondence to: C H Nejstgaard Camilla.hansen3{at}rsyd.dk
  • Accepted 10 October 2020

Abstract

Objective To investigate the association between conflicts of interest and favourable recommendations in clinical guidelines, advisory committee reports, opinion pieces, and narrative reviews.

Design Systematic review.

Eligibility criteria Studies that compared the association between conflicts of interest and favourable recommendations of drugs or devices (eg, recommending a drug) in clinical guidelines, advisory committee reports, opinion pieces (eg, editorials), or narrative reviews.

Data sources PubMed, Embase, Cochrane Methodology Register (from inception to February 2020), reference lists, Web of Science, and grey literature.

Data extraction and analysis Two authors independently extracted data and assessed the methodological quality of the studies. Pooled relative risks and 95% confidence intervals were estimated using random effects models (relative risk >1 indicates that documents with conflicts of interest more often had favourable recommendations than documents with no conflicts of interest). Financial and non-financial conflicts of interest were analysed separately, and the four types of documents were analysed separately (preplanned) and combined (post hoc).

Results 21 studies that analysed 106 clinical guidelines, 1809 advisory committee reports, 340 opinion pieces, and 497 narrative reviews were included. Unpublished data were received for 11 studies (eight full datasets and three summary datasets). 15 studies showed risk of confounding because the compared documents could differ in factors other than conflicts of interest (eg, different drugs used for different populations). The relative risk for associations between financial conflicts of interest and favourable recommendations for clinical guidelines was 1.26 (95% confidence interval 0.93 to 1.69; four studies of 86 clinical guidelines), for advisory committee reports was 1.20 (0.99 to 1.45; four studies of 629 advisory committee reports), for opinion pieces was 2.62 (0.91 to 7.55; four studies of 284 opinion pieces), and for narrative reviews was 1.20 (0.97 to 1.49; four studies of 457 narrative reviews). An analysis of all four types of documents combined supported these findings (1.26, 1.09 to 1.44). In one study that investigated specialty interests, the association between including radiologists as authors of guidelines and recommending routine breast cancer was: relative risk 2.10, 95% confidence interval 0.92 to 4.77; 12 clinical guidelines).

Conclusions We interpret our findings to indicate that financial conflicts of interest are associated with favourable recommendations of drugs and devices in clinical guidelines, advisory committee reports, opinion pieces, and narrative reviews. Limitations of this review were risk of confounding in the included studies and the statistical imprecision of individual analyses of each document type. It is not certain whether non-financial conflicts of interest influence recommendations.

Systematic review registration Cochrane Methodology Review Protocol MR000040.

Introduction

Diagnostic and treatment recommendations in clinical guidelines or advisory committee reports have an important impact on patient care. Similarly, recommendations in opinion pieces, such as editorials, and narrative reviews written by key opinion leaders could influence clinical practice. But making recommendations requires judgment, and a concern is whether conflicts of interest might influence such recommendations.

Recommendations are often written by authors with financial conflicts of interest related to the drug or device industry.12 For example, in a study of 45 clinical guidelines, 53% of authors had financial conflicts of interest.3 Researchers have also studied non-financial conflicts of interest such as specialty and academic interests, although which interests and relationships constitute a non-financial conflict of interest and whether the term is appropriate is debatable.4

Numerous studies have investigated the impact of financial conflicts of interest on the interpretation of study results. One Cochrane methodology review reported an association between industry funding and favourable conclusions in primary research studies, mainly clinical trials,5 and similar results were reported in another Cochrane methodology review on financial conflicts of interest in systematic reviews.6

In the current systematic review we investigated to what degree financial and non-financial conflicts of interest are associated with favourable recommendations (eg, recommending a drug) in clinical guidelines, advisory committee reports, opinion pieces, and narrative reviews.

Methods

The details of the methods have been published in a Cochrane methodology review protocol.7 Here we describe the core methods.

Eligibility criteria

Studies considered eligible for review were published and unpublished studies in any language and of any design that assessed the association between conflicts of interest and favourable recommendations in clinical guidelines, advisory committee reports, opinion pieces, or narrative reviews of drug or device interventions. We defined advisory committee reports as transcripts or reports from meetings held in committees to advise an organisation on a drug or device intervention, such as records from the Food and Drug Administration advisory committee on oncological drugs. Opinion pieces were defined as commentaries, editorials, and letters. Narrative reviews (non-systematic reviews) were defined as literature reviews without a systematic search of the literature and without clear eligibility criteria (see supplementary appendix 1).

For financial conflicts of interest, we included studies regardless of the type of financial conflict—that is, financial conflicts of interest related to both industry funding of documents and authors’ company ties. For non-financial conflicts of interest, we included studies on intellectual, academic, professional, or specialty interests, and on personal or professional relationships.8

Studies were excluded if they concerned: financial conflicts of interest not related to the drug or device industry (eg, tobacco or nutrition industry) as the impact from conflicts of interest might differ between industries; beliefs (eg, religious), personal experiences (eg, experiencing the medical condition), or membership of certain groups (sex or ethnicity), even if the original authors defined this as non-financial conflicts of interest; both financial and non-financial conflicts of interest at the level of an institution (eg, employment at a university that collaborates with industry); and conflicts of interest related to reports from scientific grant committees.

Search strategy and study inclusion

PubMed, Embase, and the Cochrane Methodology Register (from inception to February 2020) were searched for studies and protocols. The search strategy we developed for PubMed was adapted for the other databases (see supplementary appendix 2). To identify additional studies and protocols, we searched reference lists of the included studies, Web of Science (from inception to March 2020) for studies that cited any of the included studies, and PubMed (from inception to March 2020) for publications by the first and last author of the included studies.

We also searched proceedings from peer review congresses,9 Cochrane colloquiums,10 and Evidence Live11 for conference abstracts published up to February 2020. PROSPERO (from inception to February 2020) was searched for registered systematic reviews, and the ProQuest database (from inception to February 2020) for dissertations and theses. Finally, we searched Google Scholar (from inception to March 2020).

One review author (CHN) screened titles and abstracts for obvious exclusions. Two review authors (CHN and AWJ or AL) independently assessed potentially eligible studies based on the full text. Disagreements were resolved by discussion, with arbitration by a third review author (AL or AH) when needed.

Outcomes and data extraction

Our primary outcome was favourable recommendations, defined as such by the authors of the included studies.

Two review authors (CHN and either AWJ, ML, or AL) independently extracted data from included studies. Disagreements were resolved by discussion, with arbitration by a third review author (AH or AL) when needed.

We extracted data on basic study characteristics and on the association between conflicts of interest and favourable recommendations. Extracted data on conflicts of interest were based on the definitions used by the authors of the included studies. Information was also extracted on funding and authors’ conflicts of interest for the included studies. Supplementary appendix 3 provides details of our data extraction.

Unpublished data

We contacted the authors of the included studies to obtain unpublished data, clarify problems in our assessment of methodological quality, or receive copies of unpublished protocols (supplementary appendix 4).

Assessment of methodological quality in included studies

As tools for assessing methodological quality in these types of studies have not been published, we developed our own criteria based on those used in previous Cochrane methodology reviews on financial conflicts of interest in primary research studies and systematic reviews.56

Two review authors (CHN and either AWJ, ML, or AL) independently assessed methodological quality in included studies. Disagreements were resolved by discussion, with arbitration by a third review author (AL or AH) when needed. We used the following criteria:

• Whether the methods for including documents were adequate (adequate methodological quality might, for example, include reporting of clear inclusion criteria, with two or more assessors independently selecting documents).

• Whether the methods for coding conflicts of interest were adequate (adequate methodological quality might, for example, include coding by two or more assessors based on multiple information sources).

• Whether the methods for coding recommendations were adequate (adequate methodological quality might, for example, include coding by two or more assessors blinded to conflicts of interest information).

• Whether the methods for dealing with confounding were adequate. The documents included in a study might differ on key aspects—for example, in a sample of clinical guidelines, the guidelines might differ in types of patients and conditions, interventions, the quality of the underlying evidence, and the quality of the guidelines, which could potentially confound the association between conflicts of interest and favourable recommendations. Therefore, adequate methodological quality could, for example, include documents with and without conflicts of interest discussing the same treatment used in similar groups of patients.

We coded a study as having overall adequate methodological quality if all criteria were assessed as adequate; otherwise, we coded it as having inadequate methodological quality.

Data synthesis

Data management of individual studies

In our primary analyses, we used similar coding of conflicts of interest and recommendations to the included studies. If an ordinal scale was used to grade recommendations, for example highly positive, positive, neutral, negative, and highly negative, we recoded recommendations into two categories: favourable versus neutral or unfavourable.

If a study included different types of documents (such as both clinical guidelines and research papers), we included the study in our pooled analyses only if we had separate data for the types of documents relevant for our review.

In our analyses on clinical guidelines, we included one study that investigated 13 guidelines that each included recommendations on 24 different drugs.12 To allow for this type of panel data, we used Poisson generalised estimating equations to calculate effect estimates, which we could include in our pooled analyses.13

In our analyses on advisory committee reports, we included studies with two types of analysis units: committee members and their individual votes (individual level) and advisory committee reports and the overall voting outcome (meeting level). In our primary analysis, we analysed data at meeting level, as this level of analysis was most comparable with recommendations in the other types of documents (eg, clinical guidelines).

In some cases, the same document was included in two separate studies. When we had access to unpublished data, it was possible to remove the duplicate documents, and we chose to remove it from the study with the latest publication date. We included two studies that investigated the same FDA advisory committee reports1415 and removed duplicates from one of the studies.15 In our analyses on opinion pieces, we included two studies that investigated editorials published in some of the same oncology journals in overlapping periods1617 and removed duplicates from one of the studies.16

Primary analyses

Owing to expected clinical and methodological heterogeneity between the included studies, we used inverse variance random effects models to estimate relative risks with 95% confidence intervals. We compared recommendations between documents with and without conflicts of interest and ensured uniform directionality, so a relative risk value of more than 1 indicated that documents with conflicts of interest more often had favourable recommendations than documents without conflicts of interest. We analysed financial and non-financial conflicts of interests separately, and clinical guidelines, advisory committee reports, opinion pieces, and narrative reviews separately. We dealt with statistical heterogeneity using the I2 statistic and prediction intervals (supplementary appendix 5).

Using the methods for calculating a number needed to treat, we calculated a number needed to read for each document type (supplementary appendix 6).18 The number needed to read was defined as the expected number of documents with conflicts of interest needed to be read rather than documents without conflicts of interest for one additional document having a favourable recommendation. As it is difficult to describe the 95% confidence interval for number needed to read when the confidence interval of the relative risk crosses the boundary of no effect,19 we report the 95% confidence interval of the number needed to read in supplementary appendix 6.

Secondary analyses

We analysed advisory committee reports at individual level (ie, individual votes).

In a post hoc analysis, we combined all four types of documents (clinical guidelines, advisory committee reports, opinion pieces, and narrative reviews) in one analysis of financial conflicts of interest.

Subgroup and sensitivity analyses

Various subgroup analyses were undertaken, including stratification by different types of financial conflicts of interest, such as funding, honorariums, and gifts, and stratification by different degrees of financial conflicts of interest (≥50% vs <50% of the authors or committee members with financial conflicts of interest). In addition, we undertook various sensitivity analyses in relation to how information on conflicts of interest and recommendations were coded, using fixed effect models, and by excluding studies with authors who had conflicts of interest (supplementary appendices 7 and 8).

Analyses were conducted in either RevMan 5.4 or Stata 15.

Assessment of certainty of the evidence

Based on experience, using formal systems such as GRADE (Grading of Recommendations Assessment, Development and Evaluation) for assessing the certainty of evidence from methodological studies is challenging. We therefore focused on interpreting our results in the context of the statistical precision of our estimates (width of confidence intervals) and risk of confounding. Supplementary appendix 9 shows the GRADE assessments using a similar approach to both observational intervention studies and prognostic studies.2021

Patient and public involvement

We decided to undertake our study without patient or public involvement. Although our protocol was available in the public domain,7 we received no comments on it.

Results

Of 9973 records identified in the searches, 21 studies that analysed 106 clinical guidelines, 1809 advisory committee reports, 340 opinion pieces, and 497 narrative reviews were included in the review (fig 1).21214151617222324252627282930313233343536 No unpublished studies or protocols for planned studies were identified.

Fig 1
Fig 1

Flow chart of study inclusion

Table 1 presents the characteristics of the included studies. The 21 studies were published between 1998 and 2019. Eight studies analysed clinical guidelines (median 9 (range 2-50) guidelines), seven analysed advisory committee reports (376 (79-416) reports), six analysed opinion pieces (44 (8-131) opinion pieces), and five analysed narrative reviews (84 (7-213) narrative reviews). Sixteen studies investigated drugs, three investigated devices, and two investigated both drugs and devices.

Table 1

Characteristics of included studies

View this table:

Twenty studies investigated financial conflicts of interest only and one study investigated both financial conflicts of interest and specialty affiliations among guideline authors (non-financial conflicts of interest). None of the included studies reported industry funding, but six did not report funding information. Seven of the included studies that investigated documents with and without financial conflicts of interest were conducted by authors who themselves had financial conflicts of interest.

Unpublished data were received for 11 studies; full datasets (n=8)1415161729303135 and additional summary data (n=3).252728 No published protocols were found, and only two studies provided unpublished protocols.1432 No discrepancies were found between outcomes in these protocols and study publications. Nine of 21 author teams replied that no protocol existed for their study, and two author teams supplied reports that we did not consider to be protocols (supplementary appendix 4).

Methodological quality in included studies

In total, 20 studies were assessed as having overall inadequate methodological quality and one study as having adequate methodological quality (fig 2). Around half of the included studies had adequate methodological quality in the document inclusion process (n=10), and most had adequate methodological quality in the coding of conflicts of interest (n=15) and recommendations (n=17). Six studies were assessed as adequate for dealing with confounding and 15 as inadequate for dealing with confounding, because they included documents of different topics, such as various cancer drugs for different indications, or included documents on the same drug used for different populations, such as diabetes drugs used in adults, children, or pregnant women.

Fig 2
Fig 2

Methodological quality in included studies

Financial conflicts of interest: differences in recommendations

Clinical guidelines

Eight studies investigated a total of 106 clinical guidelines.1222232425323536 Data from four of these studies (86 clinical guidelines) could be included in the pooled primary analysis.12222535 The relative risk for the association between financial conflicts of interest and favourable recommendations in clinical guidelines was 1.26 (95% confidence interval 0.93 to 1.69, I2=0%; fig 3). The number needed to read for clinical guidelines was 9.1 (supplementary appendix 6). The remaining four studies had similar results to those of the pooled analysis (supplementary appendix 6).23243236

Fig 3
Fig 3

Meta-analysis of association between financial conflicts of interest and favourable recommendations for each type of document and documents combined. IV=inverse variance; COI=conflicts of interest

Advisory committee reports

Seven studies investigated a total of 1809 advisory committee reports.2141526272829 Data from five studies could be included in our primary or secondary pooled analyses.1415272829 In the primary analysis, including four studies of 629 advisory committee reports, the relative risk for the association between advisory committee reports with any member who had financial conflicts of interest and voting in favour of approving a drug or device was 1.20 (0.99 to 1.45, I2=24%; fig 3). The number needed to read for advisory committee reports was 7.7 (supplementary appendix 6). In the secondary analysis, including three studies of 17 816 votes, the relative risk for the association between financial conflicts of interest of individual advisory committee members and voting in favour of approving a drug or device was 1.14 (1.07 to 1.21, I2=35%; fig 4). The remaining two studies investigated voting behaviour among advisory committee members; one of these studies had similar results to our pooled analysis (supplementary appendix 6).2

Fig 4
Fig 4

Meta-analysis of association between financial conflicts of interest and favourable votes of committee members. IV=inverse variance; COI=conflicts of interest

Opinion pieces

Six studies investigated a total of 340 opinion pieces.161732333435 Data from four of these studies (284 opinion pieces) could be included in our pooled primary analysis.16173335 The relative risk for the association between financial conflicts of interest and favourable recommendations in opinion pieces was 2.62 (0.91 to 7.55, I2=78%; fig 3). The number needed to read for opinion pieces was 2.3 (supplementary appendix 6). The remaining two studies had similar results to our pooled analysis (supplementary appendix 6).3234

Narrative reviews

Five studies investigated a total of 497 narrative reviews.3031323335 Data from four of these studies (457 narrative reviews) could be included in our pooled primary analysis.30313335 The relative risk for the association between financial conflicts of interest and favourable recommendations in narrative reviews was 1.20 (0.97 to 1.49, I2=39%; fig 3). The number needed to read for narrative reviews was 8.3 (supplementary appendix 6). The remaining study had similar results to our pooled analysis (supplementary appendix 6).32

All document types

In a post hoc analysis, when all types of documents were combined, the relative risk for an association between financial conflicts of interest and favourable recommendations was 1.26 (1.09 to 1.44, I2=38%; fig 3). The number needed to read was 7.1 (supplementary appendix 6).

Non-financial conflicts of interest: differences in recommendations

One study investigated specialty interests and included 12 clinical guidelines on mammography screening.36 The focus was on whether the guideline author team included a radiologist. The relative risk for an association between having radiologists on the guideline panel and recommending routine screening for breast cancer was 2.10 (0.92 to 4.77). The number needed to read was 2.1 (supplementary appendix 6).

Subgroup and sensitivity analyses

No differences were found in effect estimates by the type of financial conflicts of interest or the degree of financial conflicts of interest for any document type (supplementary appendix 7).

Sensitivity analyses were robust in 20 of 23 analyses of financial conflicts of interest. In three analyses the association between financial conflicts of interest and favourable recommendations became stronger (supplementary appendix 8).

Assessment of certainty of the evidence

The evidence on financial conflicts of interest in all four types of documents and non-financial conflicts of interest in clinical guidelines should be interpreted with caution as most of the studies (15 out of 21) dealt inadequately with confounding and all effect estimates in our primary analyses lacked statistical precision. Using the GRADE approaches for intervention and prognostic studies resulted in low to very low certainty of the evidence depending on the type of document and the GRADE system used (supplementary appendix 9).

Discussion

In this systematic review we found an association between financial conflicts of interest and favourable recommendations of drugs and devices in clinical guidelines, advisory committee reports, opinion pieces, and narrative reviews. The four primary analyses resulted in effect estimates of a fairly similar magnitude and consistent direction, but each with varying degrees of statistical precision. The post hoc analysis in which all document types were combined confirmed these findings, and statistical precision was increased. Our findings on the impact of non-financial conflicts of interest on recommendations were limited to evidence from a single study of breast cancer screening guidelines and the involvement of radiologist authors, with statistically imprecise results. It is therefore uncertain whether specialty interests or other types of non-financial conflicts of interest have an effect on recommendations.

Strengths and limitations of this study

A major strength of our study is the inclusion of unpublished data from 11 of 21 studies. We retrieved eight full datasets and unpublished summary data for three additional studies, which ensured high data quality and comprehensive analyses thereby increasing statistical precision and minimising reporting bias. Furthermore, we searched grey literature for published and unpublished protocols. We only obtained two protocols,1432 and a comparison of outcomes in the protocols with outcomes in the study publications showed no evidence of selective outcome reporting.

Six of 21 included studies were, however, reported in a format that did not allow inclusion in meta-analysis. Four of these studies reported similar results to our meta-analysis. Two of the four studies combined different types of documents without stratifying results, with estimates (relative risk 1.69, 95% confidence interval 1.07 to 2.67, and 13.91, 1.99 to 96.97) in line with our primary analysis.3234 The other two of the four studies sampled a single pair of clinical guidelines with and without financial conflicts of interest, and in both cases only guidelines with conflicts were favourable.2324 The last two of the six studies (29% of all documents)226 sampled FDA committee reports from the same period as the studies included in our meta-analysis, implying a considerable risk of documents overlapping between the studies. The two studies reported no results for our primary analysis; if we had had access to the raw data we would likely have excluded a considerable proportion of the documents to avoid double counting. Thus, we find it unlikely that our result would have been qualitatively different had the six studies reported results in a format suitable for meta-analysis.37

Nevertheless, this review has some limitations. Firstly, the different types of documents were described using various terms in the included studies, and, despite using a comprehensive search strategy, we might have missed relevant studies. Furthermore, only four studies were included in each of our four primary analyses. Therefore, our effect estimates have some degree of statistical imprecision and none of our primary analyses were statistically significant at the conventional 5% level. The sizes of the effect estimates were, however, similar for clinical guidelines, advisory committee reports, and narrative reviews, and slightly higher for opinion pieces, and when we combined all document types in a post hoc analysis, including 13 studies, the statistical precision was increased and we found a statistically significant association with moderate heterogeneity.

Secondly, our criteria for assessment of the methodological quality of the studies for adequately dealing with confounding might be viewed as strict, and others might interpret the methodological quality of studies differently. Nevertheless, most of the studies were at risk of confounding because compared documents might differ in other factors than conflicts of interest (eg, documents on different drugs used for different patient groups). Although confounding could have influenced our estimates, the association between conflicts of interest and recommendations was fairly consistent across document types, despite some studies including comparable documents, such as clinical guidelines on efalizumab for the treatment of psoriasis,24 and others including different documents, such as advisory committee reports on a wide range of different drugs.27 Moreover, recommendations in guidelines and narrative reviews could have been influenced by conflicts of interest in the underlying evidence. For example, in certain clinical specialties such as oncology,38 conflicts of interest are common, which could have impacted the conclusions of clinical trials and systematic reviews56 and thereby indirectly affected guideline recommendations and potentially resulted in effect modification. Furthermore, how conflicts of interest in the primary clinical trials and systematic reviews underpinning a guideline are interpreted could be associated with the guideline authors’ conflicts of interest.

Thirdly, the number of authors with financial conflicts of interest might influence recommendations in a document. Our subgroup analyses of documents where a majority of the authors had financial conflicts of interest compared with those with a minority of authors found no difference in effect. However, the analyses were simplistic and based on few data, resulting in statistically imprecise results. Another important factor is the role of authors with financial conflicts of interest. For example, the chair of a guideline committee or the lead author of a narrative review could have a greater influence on recommendations than an author with a less prominent role. However, none of the included studies reported data that allowed such a comparison.

Fourthly, 11 of the 21 included studies relied solely on disclosed information in the included documents for coding conflicts of interest. This could have led to an underestimation of our effect estimates, as conflicts of interest are often underreported in various publication types, including clinical guidelines.3

Finally, the interpretation of our results can be debated. No published guidance is specifically tailored for summarising and interpreting evidence from methodological studies. One approach could be to use the GRADE system,20 but it is questionable whether using GRADE for observational intervention studies or prognostic studies is best suited for methodological studies, since the methodology of studies or the presence of conflicts of interest cannot be randomised. In our supplementary appendix 9, we reported assessments using both strategies and obtained low to very low certainty of evidence depending on the type of document and approach. Using the GRADE approach for intervention studies resulted in a more conservative interpretation of the certainty of the evidence.

Comparison with other studies or reviews

Other systematic reviews of financial conflicts of interest in different types of studies produced similar findings to those of our review. A recent Cochrane methodology review focusing on primary research studies, mainly trials, reported that industry funded studies more often had favourable conclusions than non-industry funded studies (relative risk 1.34, 95% confidence interval 1.19 to 1.51).5 Similarly, another recent Cochrane methodology review reported that systematic reviews with industry funding or by authors with financial conflicts of interest more often had favourable conclusions than systematic reviews without financial conflicts of interest (relative risk 1.98, 95% confidence interval 1.26 to 3.11).6

Financial conflicts of interest have also been investigated in relation to other industries than the drug and device industry. A systematic review reported that industry funded nutrition studies and reviews more often had favourable conclusions than non-industry funded nutrition studies and reviews (relative risk 1.31, 95% confidence interval 0.99 to 1.72).39

Meaning of the study

For our analyses, we included studies of four types of documents that are common and involved the authors’ interpretation of external evidence (involving methods less stringent than in a systematic review). Although we had anticipated potential differences between the document types, we found a fairly consistent association between financial conflicts of interest and favourable recommendations in clinical guidelines, advisory committee reports, opinion pieces, and narrative reviews. One reason could be that authors with conflicts of interest are more prone to confirm prior beliefs by selectively citing and interpreting the literature.40 This could also explain the somewhat stronger association found in opinion pieces, which to some degree allow authors more room for interpretation than narrative reviews, which undergo peer review, and clinical guidelines, which are increasingly done using standardised methods. On an absolute scale, the association between conflicts of interest and recommendations was particularly strong for opinion pieces and specialty interest in clinical guidelines with numbers needed to read of only 2.3 and 2.1, respectively, although the estimates had considerable statistical imprecision.

Our findings support conflicts of interest policies from major organisations that issue guidelines, such as the US Preventive Services Task Force, World Health Organization, and National Institute for Health and Care Excellence.414243 These policies aim to minimise the number and role of guideline authors with conflicts of interest. Similarly, some high impact journals manage conflicts of interest beyond disclosure—for example, The New England Journal of Medicine prohibits narrative reviews and editorials by authors with major financial conflicts of interest (>$10 000; >£7715; >€8540), and The Lancet prohibits commentaries, seminars, reviews, and series by authors with relevant stock ownership, employment, or company board membership.4445 Other journals should consider introducing such polices to minimise the influence of conflicts of interest on journal content.

In line with this, in 2008 the FDA introduced more stringent criteria on the types of conflicts of interest allowed by committee members.15 This might explain why a study29 that exclusively sampled committee reports from 2008 and onwards, found a weaker association between financial conflicts of interest and recommendations in advisory committee reports than the three other studies included in the pooled analysis.141528

Unanswered questions and future research

Ideally, future studies should try to minimise the risk of confounding by, for example, using a matched study design.46 However, identifying editorials commenting on the same study, or guidelines addressing the same question and developed using similar methods, might be a challenge. Furthermore, future research could focus on investigating whether specific types of financial conflicts of interest (eg, advisory board membership) or conflicts of interest related to specific companies (eg, drug manufacturer) have a greater impact than others. The included studies used various definitions of financial conflicts of interest and recommendations, and therefore use of a standardised terminology would be helpful.

Investigating the impact of non-financial conflicts of interest is challenging as no uniform definition exists. Nonetheless, a multitude of factors can be viewed as non-financial conflicts of interest, such as specialty interests, intellectual interests, personal beliefs, and personal relationships.4748 Labelling personal beliefs and theoretical schools of thoughts as conflicts of interest is problematic as no researcher is completely free from interest or from intellectual preconceptions.44950 Furthermore, the distinction between financial and non-financial conflicts of interest is not always clear. For example, in the included study on mammography screening guidelines36 it can be debated whether being a radiologist should be considered a purely non-financial conflicts of interest, as radiologists often have direct financial income (in the form of salary) from breast cancer screening. Future studies could focus on investigating the impact of the various types of non-financial conflicts of interest on favourable recommendations and on the impact of managing such interests using guideline panels with a broad range of skill sets, rather than mainly content area experts.

Conclusions

We interpret our findings to indicate that financial conflicts of interest are associated with favourable recommendations of drugs and devices in clinical guidelines, advisory committee reports, opinion pieces, and narrative reviews. Although the magnitude of effect is fairly consistent across document types, most studies had a risk of confounding and our individual analyses of each document type had some degrees of statistical imprecision. It is uncertain whether non-financial conflicts of interest influence recommendations.

What is already known on this topic

  • Clinical guidelines, opinion pieces, and narrative reviews are often written by authors with conflicts of interest related to the drug or device industry; similarly, members of advisory committees, such as regulatory drug advisory committees, often have conflicts of interest

  • Previous studies found that financial conflicts of interest are associated with favourable conclusions in primary research studies and systematic reviews

  • It is not known to what degree conflicts of interest affect recommendations in clinical guidelines, advisory committee reports, opinion pieces, and narrative reviews

What this study adds

  • The findings of this review indicate an association between financial conflicts of interest and favourable recommendations of drugs and devices in clinical guidelines, advisory committee reports, opinion pieces, and narrative reviews

  • The included studies were, however, at risk of confounding, and some degrees of statistical imprecision was found in individual analyses by document type

  • It is uncertain whether non-financial conflicts of interest influence recommendations

Ethics statements

Ethical approval

Not required.

Data availability statement

No additional data available.

Acknowledgments

This article is based on a Cochrane methodology review. The protocol is published in the Cochrane Database of Systematic Reviews 2019;10:14651858.MR000040.pub2. The review is expected to be published in the Cochrane Database of Systematic Reviews 2020;12:14651858.MR000040.pub3 (see www.cochranelibrary.com for information). Cochrane reviews are regularly updated as new evidence emerges and in response to feedback, and the Cochrane Database of Systematic Reviews should be consulted for the most recent version of the review.

We thank Gloria Won (UCSF Medical Center, Mount Zion) for developing our initial search strategy for an earlier version of the protocol; Herdis Foverskov (University Library, University of Southern Denmark) for help in developing and adjusting the search strategies; Ulrich Halekoh (Epidemiology, Biostatistics and Biodemography, University of Southern Denmark) for statistical guidance; Kristin M Aakre, Nyssa Ackerley, Giovanni M Bariani, Nicholas S Downing, Adam G Dunn, James N George, Christiane S Hartog, Michael J Hayes, Tatiana G Lerner, Peter Lurie, Susan L Norris, Genevieve Pham-Kanter, Gisela Schott, Henry T Stelfox, Ariadna Tibau, Amy T Wang, and Audrey D Zhang and their respective colleagues (authors of included studies) for clarifications and data sharing; and the Cochrane Methodology Review Group and peer reviewers for assistance and comments on the review protocol.

Footnotes

  • Contributors: AL conceived the study. CHN, AH, and AL primarily developed the protocol, with contributions from LB, KJJ, and AWJ. The protocol was based on a previous protocol developed by AL, AWJ, and LB.51 CHN and either AWJ or AL assessed studies for inclusion. CHN and either ML, AWJ, or AL extracted data and assessed studies for methodological quality. CHN analysed the data. All authors interpreted the data. CHN wrote the draft review of the manuscript and all authors revised the manuscript. CHN is guarantor. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

  • Funding: No specific funding.

  • Competing interests: All authors have completed the ICMJE uniform disclosure form at http://www.icmje.org/coi_disclosure.pdf and declare no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; and no other relationships or activities that could appear to have influenced the submitted work. LB is coauthor of one of the included studies. LB was not involved in the study inclusion, data extraction, and methodological quality assessment of any studies.

  • The lead author (CHN) affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned have been explained.

  • Dissemination to participants and related patient and public communities: No specific plan beyond dissemination through journal publication and news media.

  • Provenance and peer review: Not commissioned; externally peer reviewed.

http://creativecommons.org/licenses/by-nc/4.0/

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

References