Re: Content area experts as authors: helpful or harmful for systematic reviews and meta-analyses?
I thank PG and JI for raising this issue on the appropriate composition of a systematic review team. As they say in their paper “Content area experts as authors: helpful or harmful for systematic reviews and meta-analyses” (1), the effect of having content area experts as part of a systematic review team has not been rigorously studied. Thus we are left with indirect evidence and anecdotal and experiential evidence. As director of the U.S. Agency for Healthcare Research and Policy’s Evidence-based Practice Center Program, I have had the opportunity to review many systematic reviews written by a variety of different team structures. Below, I correct some of the misstatements in PG and JI’s article, share some additional considerations when considering the appropriate expertise on a systematic review team, and disclose my own conflicts of interest.
I would like to correct the misstatement “The Agency for Healthcare Quality and Research in the United States and Canada expects content experts to become authors of its evidence-based practice reports and guidelines.” Firstly, the Agency for Healthcare Research and Quality (AHRQ) Evidence-based Practice Center (EPC) Program does not author guidelines, and focuses only on the conduct of systematic reviews. These systematic reviews (which are freely available and intended for public use) may become the basis of a guideline, which would be authored by other public, private, or professional organizations, separate from the AHRQ EPC Program. Secondly, it is not that we expect content experts to become authors, but that we expect that the systematic review team has appropriate clinical expertise on the authorship team. This may not be the same as what PG and JI refer to as content area experts. Their definition of content experts as “a specialist clinician in the area under review or an author of an included study” or as having “inside knowledge of unpublished trials or data or about things that went wrong” is more akin to the role of the Technical Expert within the EPC Program. As PG and JI suggest, Technical Experts provide input during the review development and may comment on the report, but are not authors of the systematic review.
INVESTIGATOR WITH CLINICAL EXPERTISE VS CONTENT AREA EXPERT
The EPC program and the IOM standards (2) have emphasized the importance of having adequate expertise in the pertinent clinical content areas. We must distinguish between an individual with clinical expertise or experience and a content area expert as defined by PG and JI (or a Technical Expert within the EPC Program). As noted above, the content area expert may have financial or intellectual conflicts of interest, whereas an individual with clinical expertise may be a general practitioner without publications on the exact topic of the review. The EPC Program approach focuses first on the composition of the systematic review investigative team to ensure inclusion of adequate clinical expertise (i.e. an individual with clinical experience on the topic at hand) as well as the strong presence of methodological expertise. Indeed, the norm in the EPC program is that the primary authors are “generalists” with balanced content area expertise provided through Technical Expert Panel input. I agree with PG and JI’s concern for conflicts of interest by those who are heavily invested in a field, which is why the EPC program has in place a strict financial conflict of interest policy for both EPC authors as well as Technical experts. The EPC program explicitly restricts the role of any author of an included study and also strives to ensure balance among specialists given the often broad nature of EPC systematic review questions.
ADDITIONAL BENEFITS OF INCLUDING APPROPRIATE CLINICAL EXPERTISE
I would like to expand on some of the benefits of including the appropriate clinical expertise on the systematic review team which were not mentioned by PG and JI. Based on my experience reading reports authored by those with clinical experience as well as reports written by teams without adequate clinical experience, I have found that reviews by individuals with clinical experience are more often able to place the review findings in the context of current practice and able to present a synthesis of the findings in a way that is actionable and useful for those in practice. Systematic reviews written without having adequate clinical expertise may miss important nuances in interpreting the research within a given field or be unable to describe how the research applies to clinical dilemmas often confronted by clinicians in their daily practice.
BALANCING RIGOR AND UTILITY
I share my experiences and efforts in ensuring both the rigor as well as the utility of systematic reviews conducted by the EPC Program. As part of our mission, we are responsible to ensure that the reviews both rigorous and that the results are understandable and usable for application to clinical practice. The backbone of the EPC program is to ensure scientific rigor through adherence to published methodological guidance (3) and a rigorous scientific peer-review process. However, while we cannot ignore scientific rigor at the cost of utility, we can also not ignore utility at the cost of scientific rigor. Inclusion of individuals with appropriate clinical expertise adds to the utility of systematic reviews; inclusion of content area expertise in the form of Technical expert panels ensures that scientific rigor is not compromised. Balancing both of these requires careful attention to the potential downsides of emphasizing either side.
I note that both PG and JI are themselves well published in systematic review methodology. I disclose that my own research interests have also been in the field of systematic review methodology and that I am a general internist (both a “generalist” and one with clinical experience) by training.
1) Gøtzsche PC ,Ioannidis JPA. Content area experts as authors: helpful or harmful for systematic reviews and meta-analyses?. BMJ 2012;345:e7031
2) Institute of Medicine. Finding What Works in Health Care: Standards for Systematic Reviews (Report Brief). 2011. http://www.iom.edu/Reports/2011/Finding-What-Works-in-Health-Care-Standa.... Accessed November 2012.
3) Methods Guide for Effectiveness and Comparative Effectiveness Reviews. AHRQ Publication No. 10(12)-EHC063-EF. Rockville, MD: Agency for Healthcare Research and Quality. April 2012. Available at: www.effectivehealthcare.ahrq.gov/methodsguide.cfm. Accessed November 2012.
Competing interests: SC has published in systematic review methodology and is a general internist.