Endorsing performance measures is a matter of trustBMJ 2018; 360 doi: https://doi.org/10.1136/bmj.k703 (Published 23 February 2018) Cite this as: BMJ 2018;360:k703
All rapid responses
TO THE EDITORS:
As leaders of the National Quality Forum (NQF), with clinical expertise in emergency medicine and acute care, we are deeply concerned with several inaccuracies included in “Endorsing performance measures is a matter of trust”. This article discussed the Centers for Medicare & Medicaid Services’ (CMS) Severe Sepsis and Septic Shock Early Management Bundle (SEP-1) measure. The authors mischaracterized NQF’s disclosure of interest process for seating expert committee members, and presented a misleading account of the measure’s review process at NQF. Most importantly, in 2016, NQF completed a review of the measure specifications based on updated evidence and the measure no longer includes the issues discussed in the paper.
The authors imply that the co-chair of the committee that reviewed SEP-1 in 2013 was conflicted. The co-chair was an employee of AdvaMed, a medical technology trade association, which promotes global policies to advance medical technology.
Jaswal et al state that, “We know of no evidence that any person or company intended to or effectively influenced the forum’s 2013 decision to endorse SEP-1 and its requirement for central venous pressure and blood oxygen saturation but the commercial relations are not in accordance with the forum’s policy on potential conflicts.” This is inaccurate. The co-chair was thoroughly vetted through NQF’s standard disclosure of interest process, a robust and transparent process that offers an opportunity for NQF member and public comment. Among several requirements, the policy explicitly requires all prospective members to disclose direct and indirect relationships with several entities including biomedical device manufacturing or sales companies, health care measure development organizations, medical or other health care profession societies, hospitals, clinics, health information technology manufacturing or sales companies, and health plans.
Committee members with conflicts are recused from participating in any discussion or vote on the measure(s) of interest. In accordance with policy, the co-chair of the committee that reviewed SEP-1 was found not to have a direct or indirect relationship with the third party, nor did the co-chair have direct involvement in developing SEP-1 or any other measures under endorsement consideration. At the time of the 2013 SEP-1 review, the co-chair was responsible for conveying health care delivery policy related information to AvaMed’s member organizations. The fact that a prospective or current committee member has a relevant interest does not mean that a conflict of interest exists or that the individual cannot serve on a committee.
The authors’ assessment of NQF’s evaluation of the evidence is also misleading and incomplete. The author’s contend that certain components of the SEP-1 bundle failed to meet NQF’s criteria for Importance to Measure and Report during the 2012 endorsement of maintenance review. (The SEP-1 bundle passed NQF’s Importance to Measure and Report criterion and was recommended for endorsement by the review committee with 79 percent approval.) The evidence was based on the 2008 Sepsis Survival Campaign , and – like other areas of improving and changing medical evidence – has since evolved.. It is troubling that the authors did not mention the most recent maintenance of endorsement review of SEP-1 in 2016 included a synthesis of the literature for several updated components in the bundle. The updates are based on the Surviving Sepsis Campaign: International Guidelines for Management of Sepsis and Septic Shock 2016 and graded by the Sepsis and Septic Shock 2016 Guidelines. Monitoring central venous pressure and central venous oxygen saturation are no longer required or included in the new measure specifications as of this 2016 review of the literature and emerging evidence. This is an important, factual omission from the Jaswal et al paper.
The National Quality Forum (NQF) is an independent, not-for-profit, membership-based organization that brings together various public- and private-sector organizations to reach consensus on how to measure quality in healthcare to make it better, safer, and more affordable.
NQF is committed to achieving consensus and balancing interests through transparency and consistency and continuously evaluates and improves the endorsement process in response to the needs and concerns of our multi-stakeholder membership and the public. We are of course open to changes and improvements in our disclosure of interest or conflict of interest policies, but such changes must be based on fact, not false assertions or innuendo.
Elisa Munthali, MPH
Shantanu Agrawal, MD, MPhil
Melissa Mariñelarena, RN, MPA, CPHQ
Jaswal SJ, Natanson C, Eichacker PQ. Endorsing Performance Measures is a Matter of Trust. BMJ. 2018 Feb 23;360:k703. doi: 10.1136/bmj.k703.
National Quality Forum. Committee Guidebook for the NQF Measure Endorsement Process. Washington, D.C.; 2017. http://www.qualityforum.org/Measuring_Performance/Consensus_Development_.... Last accessed March 2018.
Competing interests: No competing interests
Trusting the Sepsis Measure “Analysis”
As original developers (ST, ER, ML, RPD) and stewards (ST, ER) for the Centers for Medicare and Medicaid Services’ (CMS) Severe Sepsis and Septic Shock Early Management Bundle (SEP-1) measure and as a former policy advisor for CMS (LT), we provide the following corrections for Jaswal and colleagues’ multiple false and erroneous statements in “Endorsing performance measures is a matter of trust” (1).
SEP-1 is a measure in the Inpatient Quality Reporting (IQR) program, which only requires hospitals to report measure data to CMS; it is not a performance-based program (2). No link exists between a hospital’s performance on SEP-1 and payment. CMS publishes detailed information about the IQR program that the authors may not have reviewed (2, 3). The CMS web page cited by the authors pertains to the health insurance market and does not mention SEP-1. The references do not support the authors’ premise that CMS uses SEP-1 “to regulate US healthcare” by withholding reimbursement (4).
The authors confuse the IQR program (which houses the SEP-1 measure) with the CMS value-based purchasing (VBP) programs (5). These programs operate under different rules that are described publicly and updated annually through the government rulemaking process, including notice and comment periods (5).
The authors also claim that poor performance on SEP-1 jeopardizes hospital accreditation status. This claim is false. Neither reporting nor performance on SEP-1 affects a hospital’s accreditation status (6).
The authors assert that with the “compelling incentives” dispatched above, CMS has “mandate[d]” care approaches in SEP-1. This claim is false. SEP-1 is a protocol-based strategy to measure sepsis care rendered in hospitals and does not “mandate” that a hospital or clinician take any specific intervention. There is no punishment, financial or otherwise, for failing to complete any action in SEP-1.
The authors mislead readers to believe that reassessment of volume status and tissue perfusion necessitates the use of central venous pressure (CVP) and central venous oxygen saturation (ScvO2) monitoring. The current publicly available specification manual from 2017 (version 5.3a) clearly states that these variables are not required (6). As now specified, the reassessment requirement is satisfied if a clinician states that a reassessment was performed. There is no need to document the means by which the clinician completed the reassessment (7). We note the authors relied on a legacy specification manual (version 5.0a). Although now 3 years out-of-date, even then the measurement of CVP and ScvO2 was a matter of clinician choice with alternatives to satisfy the measure, among which was simply a focused physical exam.
Also of concern, the authors advance the fallacy that because the measure previously held out CVP and ScvO2 as variables that could satisfy the reassessment requirement, the developers (or perhaps NQF, or CMS) promoted the sale of Edwards Lifescience’s proprietary oximetry catheter. In fact, the two physiologic measurements can be obtained from a central line made by any manufacturer. If one were to endorse the authors’ reasoning which includes allegations of inappropriate intent and industry contamination, then SEP-1 corruptly supports many industries: pharmaceuticals (by recommending the use of antibiotics), manufacturers of intravenous tubing and crystalloid vendors (by recommending intravenous fluids), point-of-care manufacturers (by including lactate measurement), and perhaps support laboratory supply vendors too (by requiring blood cultures). The authors not only make a false allegation, but reach an illogical conclusion. The SEP-1 measure does not endorse any private entity and solely exists to promote sepsis quality care improvement.
Finally, these authors have falsely claimed that at least one developer “continued to receive financial support from Edwards Lifesciences,” and have attempted to disparage two others by referring to their past publicly disclosed affiliations with the firm. The claim of any continuing relationship is false. The authors’ citation to a web page without an associated publication date does not validate their claim. The standard by which conflicts of interests are adjudicated in medicine is disclosure. In this case that standard has been met and exceeded as none of the developers had relationship with Edwards in 2012 when the measure first included hemodynamic variables.
The authors, from the National Institutes of Health, are familiar with the need to resolve conflicts of interest, and they should be cognizant of the damage false and defamatory statements can do to respected institutions and individuals alike (8).
Because of the egregious nature of these multiple false or erroneous claims, we believe this publication should be retracted.
Sean R. Townsend MD
Lemeneh Tefera MD MSc
Emanuel P. Rivers MD MPH
Mitchell M. Levy MD
R. Philip Dellinger, MD
1. Jaswal SJ, Natanson C, Eichacker PQ. Endorsing Performance Measures is a Matter of Trust. BMJ. 2018 Feb 23;360:k703. doi: 10.1136/bmj.k703.
2. Centers for Medicare & Medicaid Services. Hospital Inpatient Quality Reporting Program. 9 September 2017 Accessed at www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/... on 22 February 2018.
3. Social Security Act § 1886(b)(3)(B)(vii)-(viii); 42 CFR § 412.64(d)(2); 81 Fed. Reg. 56939 (August 22, 2016).
4. Centers for Medicare & Medicaid Services. Health Insurance Marketplaces. Accessed at https://www.cms.gov/CCIIO/Programs-and-Initiatives/Insurance-Programs/ on 28 February 2018.
5. Centers for Medicare & Medicaid Services. Hospital Inpatient Quality Reporting Program. 11 September 2017 Accessed at https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Inst... on 27 February 2018.
6. The Joint Commission. Specifications Manual for National Hospital Inpatient Quality Measures. Accessed at www.jointcommission.org/specifications_manual_for_national_hospital_inpa... on 22 February 2018.
7. QualityNet. Fact Sheets for Measure Changes. Accessed at https://www.qualitynet.org/dcs/ContentServer?c=Page&pagename=QnetPublic/... on 28 February 2018.
8. The Wall Street Journal. Blood-Study Author Didn’t Disclose Conflicts. 14 July 2008. Accessed at https://www.wsj.com/articles/SB121598988902149353 on 28 February 2018.
Competing interests: No competing interests