Mortality among US veterans after emergency visits to Veterans Affairs and other hospitals: retrospective cohort studyBMJ 2022; 376 doi: https://doi.org/10.1136/bmj-2021-068099 (Published 16 February 2022) Cite this as: BMJ 2022;376:e068099
- David C Chan, professor of health policy and investigator1 2,
- Kaveh Danesh, doctoral and medical student3 4,
- Sydney Costantini, doctoral student1 2,
- David Card, professor of economics3,
- Lowell Taylor, professor of economics5,
- David M Studdert, professor of medicine and law1 6
- 1Department of Health Policy, Stanford University, Stanford, CA, USA
- 2Department of Veterans Affairs, Palo Alto, CA, USA
- 3Department of Economics, University of California, Berkeley, Berkeley, CA, USA
- 4School of Medicine, University of California, San Francisco, San Francisco, CA, USA
- 5Heinz College, Carnegie Mellon University, Pittsburgh, PA, USA
- 6Stanford Law School, Stanford, CA, USA
- Correspondence to: D Chan
- Accepted 10 January 2022
Objective To measure and compare mortality outcomes between dually eligible veterans transported by ambulance to a Veterans Affairs hospital and those transported to a non-Veterans Affairs hospital.
Design Retrospective cohort study using data from medical charts and administrative files.
Setting Emergency visits by ambulance to 140 Veteran Affairs and 2622 non-Veteran Affairs hospitals across 46 US states and the District of Columbia in 2001-18.
Participants National cohort of 583 248 veterans (aged ≥65 years) enrolled in both the Veterans Health Administration and Medicare programs, who resided within 20 miles of at least one Veterans Affairs hospital and at least one non-Veterans Affairs hospital, in areas where ambulances regularly transported patients to both types of hospitals.
Intervention Emergency treatment at a Veterans Affairs hospital.
Main outcome measure Deaths in the 30 day period after the ambulance ride. Linear probability models of mortality were used, with adjustment for patients’ demographic characteristics, residential zip codes, comorbid conditions, and other variables.
Results Of 1 470 157 ambulance rides, 231 611 (15.8%) went to Veterans Affairs hospitals and 1 238 546 (84.2%) went to non-Veterans Affairs hospitals. The adjusted mortality rate at 30 days was 20.1% lower among patients taken to Veterans Affairs hospitals than among patients taken to non-Veterans Affairs hospitals (9.32 deaths per 100 patients (95% confidence interval 9.15 to 9.50) v 11.67 (11.58 to 11.76)). The mortality advantage associated with Veterans Affairs hospitals was particularly large for patients who were black (−25.8%), were Hispanic (−22.7%), and had received care at the same hospital in the previous year.
Conclusions These findings indicate that within a month of being treated with emergency care at Veterans Affairs hospitals, dually eligible veterans had substantially lower risk of death than those treated at non-Veterans Affairs hospitals. The nature of this mortality advantage warrants further investigation, as does its generalizability to other types of patients and care. Nonetheless, the finding is relevant to assessments of the merit of policies that encourage private healthcare alternatives for veterans.
The United States has many healthcare systems, spread across the private and public sectors. The Veterans Health Administration, operated by the US Department of Veterans Affairs (VA), is the largest, comprising 171 hospitals and 1112 clinics that provide care to more than nine million military veterans and their families.1 Care is financed by the federal government, delivered by federal employees, and offered essentially free of charge to enrolled veterans. In these respects, the VA system resembles the public healthcare systems operating in many developed countries, including the NHS in the United Kingdom.
Since its establishment in 1930, there has been debate over the quality and accessibility of care provided in the VA system. Debate has intensified in recent years, fueled by concerns about the system’s monolithic nature and the lack of choice veterans have about where to obtain their care. Congress and the Obama and Trump administrations have responded with reforms that enable veterans to opt to obtain care in the private sector.23 The reforms are based, at least in part, on a premise that veterans can obtain better care outside the VA healthcare system. Available evidence calls this premise into question.
Studies comparing the performance of VA and non-VA hospitals have generally found that VA hospitals compare favorably.45 For example, a recent systematic review of 69 studies concluded that the “VA often (but not always) performs better than or similarly to other systems of care with regard to the safety and effectiveness of care.”6 However, benchmarking across different health systems invites several methodological challenges. Most importantly, the usual design involves comparing veterans receiving care in the VA healthcare system with non-veterans receiving care in non-VA settings. Although many of these studies adjust for demographic and clinical characteristics of the two patient populations, veterans are a distinctive group, and available information sources (eg, claims data) likely do not allow researchers to control for important underlying differences in the health status of veterans and non-veterans. Resolving such differences is particularly important in cross system comparisons of mortality risk, an informative quality measure that relatively few studies have investigated.46
To address these longstanding concerns regarding comparability, we tracked mortality in a cohort of veterans aged 65 years or older who met two criteria: they were enrolled in both the Veterans Health Administration and the Medicare program, and their care was initiated by an ambulance ride to obtain emergency treatment. These veterans’ eligibility to receive care in both VA and non-VA hospitals, coupled with the emergency nature of their visit, reduced the potential for self-selection into one hospital type or the other. Our analyses also accounted for other factors—for example, the zip code of the veteran’s residence, previous care patterns, and the ambulance’s life support capabilities—that might skew comparisons. Our primary objective was to obtain a more balanced comparison of 30 day mortality risk after emergency care in VA and non-VA hospitals.
Results are reported in accordance with the STROBE (strengthening the reporting of observational studies in epidemiology) guidelines (table S1).7
We obtained information on veterans and their VA care from the Corporate Data Warehouse. The Corporate Data Warehouse is a system wide repository of administrative and clinical data for the VA. It includes electronic health records created during enrollee visits (to a VA clinic, hospital, or emergency department) as well as demographic information (eg, enrollees’ sex, age, race or ethnic origin, and residential address). For non-VA and ambulance care, we obtained data on Medicare claims associated with each veteran in the study sample. These data provided information on all clinical encounters—including ambulance rides and outpatient, inpatient, and emergency care—that were billed to Medicare.
Mortality data came from four sources: the Corporate Data Warehouse, Medicare claims, the Veterans Benefits Administration, and the Social Security Administration. The Veterans Benefits Administration and Social Security Administration capture deaths that occur outside healthcare settings. Research comparing deaths in the National Death Index to deaths identified through these four sources suggest that, collectively, they capture about 98% of all deaths.89
The study sample consisted of patients (aged ≥65 years) who were enrolled in the VA and Medicare programs and transported by ambulance to a hospital emergency department across 46 US states and the District of Columbia during the study period (1 January 2001 to 20 November 2018). Most hospitals in our sample are observed in all years (fig S1). The patients must have been enrolled in both programs on the day of the ambulance ride and for at least 12 months before it. We restricted the sample to emergency ambulance rides with lights and sirens that originated from 911 dispatch calls. The rides were identified in Medicare Part B claims data using methods described elsewhere.10
To reduce risks of selection bias, we made several exclusions aimed at narrowing the study sample to patients who could plausibly have been transported to either a VA or non-VA hospital. We excluded ambulance rides to hospitals located more than 50 miles from the center of the patient’s zip code of residence, rides originating in zip codes that did not have at least one VA hospital and non-VA hospital within 20 miles of the zip code center, and rides originating in zip codes in which fewer than 5% of the rides went to a VA or non-VA hospital during the study period. After applying these eligibility criteria, we constructed an analytic dataset at the level of ambulance rides. We allowed eligible patients to contribute more than one ride, but to avoid overlapping follow-up periods we excluded rides that occurred within 30 days of a prior ride.
To determine which hospitals patients in the sample were transported to, we linked patients’ Medicare claims for ambulance rides with records of visits to the emergency department initiated within 24 hours of the ride at VA hospitals (Corporate Data Warehouse records) and non-VA hospitals (Medicare claims data). We assigned patients to the hospital to which they were taken; fewer than 1% of rides in our sample involved patients who were transferred to another hospital within a day.
To account for differences in health status between patients transported to VA hospitals and non-VA hospitals, our analyses adjusted for characteristics of patients and the ambulance ride. With respect to patients, we adjusted for zip code of residence; demographic characteristics (age in two year bands, race or ethnic origin, and sex); six binary variables indicating receipt of VA or non-VA primary care, emergency care, and inpatient care in the 12 months before the ride; and previous medical diagnoses, specified as 31 indicators for Elixhauser comorbidities recorded in the 12 months before the ride.11 We created a measure of predicted mortality risk for the patient in each ambulance ride from a linear regression of 30 day mortality on the above characteristics, controlling for whether the patient was transported to a VA hospital.
With respect to the ambulance ride, we adjusted for the origin of the ride (residence; residential, domiciliary, or custodial facility; skilled nursing facility; or scene of accident or acute event); time (day of the week, month-year interactions); life support capabilities, classified according to categories for basic and advanced life support specified in the Healthcare Common Procedure Coding System codes; and primary diagnosis made during the ride, coded according to ICD-9 (international classification of diseases, 9th revision). Table S3 describes these control variables in greater detail.
We used ordinary least squares regression analysis to estimate the association between type of hospital and mortality risk. Specifically, we fit linear probability models in which the outcome was mortality within 30 days of the index ambulance ride, and the predictor of interest was a binary variable indicating whether the patient was transported to a VA or non-VA hospital. The models adjusted for the characteristics of patients and ambulance rides enumerated above and elaborated in table S3.
As mentioned above, our focus on dually eligible veterans taken by 911 initiated ambulance rides, and our restriction to rides that could have plausibly been taken to either VA or non-VA hospitals, limits the potential for self-selection. Doyle et al10 rely on a similar intuition in the ambulance setting, but more formally rely on an instrumental variable technique based on the quasi-random assignment of ambulances. Although this approach strengthens causal inferences, its application to our study substantially reduces the analytic sample, making it infeasible to estimate mortality effects within the various subgroups of interest.
To examine whether mortality differences varied by patient characteristics, we stratified the model by patients’ sex, age group (65-74, 75-84, and ≥85 years), race or ethnic origin (non-Hispanic white, non-Hispanic black, Hispanic, or other), predicted mortality risk (divided into three equal groups corresponding to low, medium, and high risk), Elixhauser comorbidities, and prior hospital use. The prior use subanalyses separated patients according to the type of care they had received in the previous year (outpatient, emergency, and inpatient) and whether that care was obtained in the same VA or non-VA facility to which they were transported.
Our analysis was restricted to observations with complete data on all variables. We conducted sensitivity analyses that controlled for different combinations of control variables, added controls for hospital characteristics, relaxed our study eligibility criteria, examined a logit specification of mortality, and extended our mortality time window to 1.5 years. Analyses were conducted in Stata/MP version 15.1, and figures were created in R version 4.0.2.
Patient and public involvement
Veterans from the VA Palo Alto Veterans Engagement Council provided feedback on our research question during the preparation of the proposal for the grant that supported this work. Since the inception of our project, we have liaised with VA operational leadership, including the Office of Community Care and the National Lead for Emergency Medicine.
The study sample consisted of 583 248 patients who had 1 470 157 emergency ambulance rides (mean 2.52 rides per patient); 231 611 (15.8%) of the rides went to a VA hospital and 1 238 546 (84.2%) went to a non-VA hospital (table S4). Details are provided in figure 1. Patients who went to VA hospitals had similar characteristics to those who went to non-VA hospitals, but with some noteworthy differences (table 1). For example, patients taken to VA hospitals were more likely to be black and have used VA care in the prior year; they were also more likely to have mental health and substance use disorders (table S5). The proportion of ambulance rides that occurred on weekends (27.0%) was close to two sevenths, consistent with the emergency nature of these rides. Predicted 30 day mortality was slightly higher on average for patients taken to VA hospitals. The distribution of predicted 30 day mortality was similar between patients taken to VA and non-VA hospitals (fig S2).
Frequency and rates of mortality
In total, 9.3% of the patients taken to VA hospitals died within 30 days of their ride, compared with 11.7% of the patients taken to non-VA hospitals. In adjusted analyses, the 30 day mortality rate was also lower among patients taken to VA hospital than among those taken to non-VA hospitals (9.32 (95% confidence interval 9.15 to 9.50) v 11.67 (11.58 to 11.76) deaths per 100 patients). This absolute difference of 2.35 (95% confidence interval 2.16 to 2.54) deaths per 100 patients corresponds to a 20.1% lower mortality rate among patients taken to VA hospitals (fig 2).
Mortality differences by patient subgroups
The difference in adjusted mortality risk was similar for men and women (fig 2). However, compared with the overall estimate, the mortality advantage was disproportionately large for Hispanic patients (22.7% lower; 8.53 deaths per 100 patients taken to VA hospitals v 11.04 deaths per 100 patients taken to non-VA hospitals), black patients (25.8% lower; 7.78 v 10.49), patients aged 65-74 years (27.1% lower; 6.37 v 8.74), and patients who arrived with relatively low mortality risk (31.6% lower; 3.91 v 5.72).
Figure 3 groups patients according to the types of care, if any, that they received at the same hospital in the previous year. In four of the five groups shown, patients taken to VA hospitals had lower mortality rates than patients taken to non-VA hospitals. The VA mortality advantage was especially large for patients who had previously received outpatient and inpatient care at the same hospital, and for those who had previously received outpatient, emergency, and inpatient care there. Prior inpatient care at the index hospital was a common feature of the groups that showed the largest mortality advantage for patients treated at VA hospitals. The only group in which VA patients did not have lower mortality rates were patients who had not received treatment in the past year at the hospital to which they were taken. However, there were relatively few patients taken to VA hospitals in that group, and this adjusted mortality estimate had wide confidence intervals.
The lower mortality rate among patients taken to VA hospitals was consistent across patient groups defined by the presence of pre-existing comorbidities (fig 4) and across years in our sample (fig S3). For 29 of 31 Elixhauser comorbidities, patients taken to VA hospitals had lower adjusted mortality rates than those taken to non-VA hospitals, and for 22 comorbidities the 95% confidence interval included the overall estimate for the study sample (that is, a reduction of 2.35 percentage points among patients taken to VA hospitals v those taken to non-VA hospitals). The two notable outliers from this consistent pattern were patients with metastatic cancer and patients with lymphoma—the adjusted mortality rates for patients with these comorbidities did not differ appreciably according to the type of hospital they were taken to, and the confidence intervals for both differences included zero (indicating no difference).
Temporal nature of mortality difference
Over the 30 days after the ambulance rides, lower mortality rates were evident within a few days for patients taken to VA hospitals (table S9). This difference then increased, peaking midway through the period when mortality was 2.41 percentage points lower for patients taken to VA hospitals, before subsequently decreasing through day 30.
Our main results were robust to various sensitivity analyses. First, alternative model specifications that used different combinations of the control variables produced estimates of mortality rates among patients taken to VA hospitals that were 1.43 to 2.43 percentage points lower than those among patients taken to non-VA hospitals (table 2). Second, to probe whether systematic differences in the structure of VA and non-VA hospitals confounded the association of interest, we added covariates indicating characteristics of the destination hospitals (eg, trauma level, presence of cardiac catheterization, and teaching hospital status), but the mortality difference estimated in this model was virtually identical to that estimated in the main model (table S10). Third, we examined the extent to which the series of exclusions made to create a balanced study sample (fig 1) affected the mortality difference, and we considered alternative samples that were either more restrictive in the treatment of prior ambulance rides or less restrictive in the treatment of missing data. In all these samples, the mortality advantage of patients taken to VA hospital was statistically significant and qualitatively similar (tables S11 and S12). Fourth, we considered an alternative specification of mortality with a logit model, rather than the linear probability model used to compute our main estimates. This specification produced qualitatively similar results (table S14).
Finally, extending the window of analysis beyond 30 days showed that the gap between mortality rates for patients taken to VA hospitals and non-VA hospitals continued to narrow gradually, before closing 11 months after the index ambulance ride. This trend continued for as long as we followed it; at the 15 month mark, the mortality rate for patients taken to VA hospitals had become significantly higher than for those taken to non-VA hospitals (fig S4). These higher mortality hazards in the longer term among veterans presenting to VA hospitals appear to accord with the conclusion of previous studies that these veterans tend to have lower underlying health than veterans who present elsewhere for care.121314151617 This finding also suggests that our design did not achieve complete randomization of site of care, or fully control for health differences. However, the likely consequence of this residual imbalance in the comparison groups is that our results underestimate the true reduction in mortality due to VA care.
In this study of older adult veterans transported by ambulance to hospitals for emergency treatment, our findings indicate that those taken to VA hospitals were more likely than those taken to non-VA hospitals to be alive 30 days after the initial ambulance ride. The mortality advantage was consistent across men and women, patient age groups, and different types of pre-existing conditions, but the advantage was particularly pronounced among black and Hispanic patients and patients who had previously received care at the same hospital to which they were transported. Of the 50 subgroups of patients examined, none had significantly lower mortality rates at non-VA hospitals than at VA hospitals.
Comparison with other studies
Many previous studies have compared quality and outcomes of care in VA and non-VA settings.4 Studies of processes of care have consistently shown superior VA performance, but results from studies analyzing mortality outcomes are mixed.6 For example, Nuti et al found a lower 30 day mortality among men aged 65 years or older admitted to VA hospitals than among a comparable group admitted to non-VA hospitals,18 but other studies have found higher mortality among patients in VA hospitals after pancreatectomy,19 kidney transplantation,20 coronary artery bypass graft,21 and general surgery.22
A recognized explanation for these mixed findings relates to differences in the health status of the comparison groups.6 Most of these studies compare veterans receiving VA care with non-veterans receiving non-VA care. But if veterans tend to have worse health, this approach could falsely credit non-VA hospitals with superior performance. To avoid this bias, a few studies have restricted their comparisons to veterans, taking advantage of the fact that some veterans obtain care outside the VA system. Homogenizing the comparison groups in this way reduces the risk of bias.
Strengths and limitations of this study
Our study sought to mitigate risks of biased comparison in two ways: by focusing exclusively on veterans who were statutorily eligible for both VA and non-VA care, and by tracking outcomes of episodes of care that were initiated by emergencies. Previous studies have not sought to balance comparator groups in this way. In addition, with 583 248 patients followed into treatment at 2762 hospitals in 46 US states, our study is among the largest comparisons of VA and non-VA care conducted to date. It also examines all cause mortality, an outcome that captures effects of multiple dimensions of care on a wide range of medical conditions.
Our study also had several limitations. First, although our design mitigates the risk of confounding, it does not necessarily eliminate it. Systematic differences could still have existed between the patients taken to VA and non-VA hospitals and be associated with mortality risk. However, our sensitivity analyses indicated that the main results were robust to the inclusion of different sets of patient controls and hospital characteristics (eg, trauma center level, presence of cardiac catheterization). However, we detected slightly higher predicted mortality among patients taken VA hospitals, and our longer run analyses of mortality differences also suggested that those patients were slightly more ill than those taken to non-VA hospitals. The most plausible effect of this persistent underlying difference in health status on our study estimates is to render them an underestimate of the true reduction in mortality associated with VA care.
Second, for the analysis examining effects of previous care, we compared groups that differed in the location of care before the emergency visit of interest, potentially introducing some confounding. Third, our results might not be generalizable to younger veterans or to episodes of care not initiated by emergency treatment. Finally, although our results hint at several explanations for how VA hospitals achieved lower mortality rates, the study was not designed to illuminate mechanisms, and we can only speculate on what they could be.
The mortality advantage that we identified among patients treated at VA hospitals was consistent across medical conditions but much greater for patients with a history of care at the hospital to which they were taken. This pattern of findings is consistent with distinctive strengths of VA care that have been previously described—in particular, information technology and integration of care.23 The VA has long provided integrated healthcare, supported by an advanced health information technology system, whereas movement toward electronic health records at non-VA hospitals has been substantially delayed.24 Although the gap has narrowed, the VA system continues to lead many private healthcare delivery systems in health information technology capabilities.2526 Other distinctive features of the VA system include organization around primary care, minimal cost sharing for veterans, and a salaried approach to physician payment that avoids incentives to over-treat or under-treat that are common in private payment models. VA and non-VA hospitals might also differ in beds, staffing, and wait times in the emergency department; however, adjustment for differences in hospital characteristics did not substantively change our results (table S10).
Widely publicized concerns about quality and capacity of the VA system, the largest public healthcare delivery system in the US, have fueled public perceptions that the VA health system is falling short of providing good care to the many veterans who depend on it.2728 Our findings join those from other studies in suggesting that, for the system overall, those perceptions do not match reality. This conclusion has important implications for health policy. Enabling or encouraging veterans to obtain care outside the VA system could lead to worse—not better—health outcomes, particularly for veterans with established care relationships at VA facilities. The extent to which VA hospitals outperform other types of hospitals, and in what specific facets of care, should continue to be studied. At the same time, increasing evidence of superior performance justifies a redoubling of efforts to understand how the VA system achieves this. As well as helping the VA to improve care processes and outcomes, those insights could produce valuable lessons for healthcare delivery systems globally.
What is already known on this topic
Available evidence suggests Veterans Affairs hospitals perform comparably to or better than non-Veterans Affairs hospitals on process related measures of quality, but studies comparing mortality rates have produced mixed findings
Most of these studies compare veterans receiving care in the Veterans Affairs system with non-veterans receiving care elsewhere—an approach that might produce biased results if these patient populations are fundamentally different in nature
What this study adds
With a study design aimed at reducing risks of biased comparisons, veterans transported by ambulance to VA hospitals were found to have better 30 day survival than veterans transported to non-VA hospitals
The survival advantage was greater for Hispanic and black patients, and for patients with a history of receiving care at the same hospital to which they were taken
The study was approved by the institutional review board at Stanford University and the scientific review subcommittee at the VA Palo Alto Health Care System.
Data availability statement
No additional data are available.
We thank Steve Asch, Chad Kessler, Todd Wagner, and Anita Vashi for their feedback on this research project; and Noah Boden-Gologorsky, Johnny Huynh, Kevin Kloiber, Uyseok Lee, Matthew Merrigan, and Kevin Todd for their research assistance.
Contributors: DCC led the study and acts as the guarantor. All authors were involved in the study design, data analysis, and revision of the manuscript. All authors read and approved the final manuscript. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.
Funding: DCC was supported by a US National Institutes of Health Director’s Early Independence Award (1DP5OD019903-01). This project was supported by a Veterans Affairs Health Services Research and Development Merit Award (I01 HX002631-01). The funders had no input in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication.
Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/disclosure-of-interest/ and declare: support from the US National Institutes of Health for the submitted work; no financial relationships with any organizations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.
The lead author affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned have been explained.
Dissemination to participants and related patient and public communities: Following publication, we plan to work with veteran representatives in the Palo Alto Veterans Engagement Council and operational leadership (Veterans Affairs National Lead for Emergency Medicine and the Veterans Affairs Office of Community Care) to disseminate the results of this study.
Provenance and peer review: Not commissioned; externally peer reviewed.
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.