How do elderly patients decide where to go for major surgery? Telephone interview survey
BMJ 2005; 331 doi: https://doi.org/10.1136/bmj.38614.449016.DE (Published 06 October 2005) Cite this as: BMJ 2005;331:821
- Lisa M Schwartz, associate professor of medicine1,
- Steven Woloshin, associate professor of medicine (steven.woloshin{at}dartmouth.edu)1,
- John D Birkmeyer, professor of surgery2
- 1 VA Outcomes Group (111B), VA Medical Center, 215 N Main Street, White River Junction, VT 05009, USA,
- 2 Department of Surgery, 2101 Taubman Center, 1500 E Medical Center Drive, Ann Arbor, MI 48109-0346, USA
- Correspondence to: S Woloshin
- Accepted 8 August 2005
Abstract
Abstract Objective To learn how patients in Medicare, the US medical insurance programme that covers elderly patients, made decisions about where to undergo major surgery and how they would make future decisions.
Design National telephone interview study.
Setting United States.
Participants 510 randomly selected Medicare beneficiaries who had undergone an elective, high risk procedure about 3 years earlier—abdominal aneurysm repair (n = 103), heart valve replacement surgery (n = 96), or resection of the bladder (n = 119), lung (n = 128), or stomach (n = 64) for cancer. Response rates were 48% among eligible survivors and 68% among those able to participate.
Results Although all participants could choose where to have surgery, only 55% said there was an alternative hospital in their area where they could have gone. Overall, only 10% of respondents seriously considered going elsewhere for surgery. Few respondents (11%) looked for information to compare hospitals. Almost all respondents thought their hospital and surgeon had good reputations (94% and 88%, respectively), beliefs mostly determined by what their referring doctors said. When asked how much various factors would influence their advice to a friend about choosing where to go for major surgery, surgeon reputation was the most influential (78% said it would influence their advice “a lot”), followed by the hospital having “nationally recognised” surgeons (63%), and then various performance data (surgeon volume (58%), nurse:patient ratios (49%), number of operations carried out by the hospital (48%), and hospital operative mortality (45%)). Forty per cent said they would act on mortality data, indicating that they would switch from their initial choice of hospital to a different one if its mortality was a percentage point lower (that is, 3% v 4%).
Conclusion Some respondents claimed they would switch hospital for elective surgery on the basis of mortality data. Since most respondents relied on their referring physician's opinion to decide where to have surgery, surgical performance data ought to be accessible to referring physicians.
Introduction
There is growing interest in providing patients with surgical performance data to help them select the best surgeons and hospitals. New York State, for example, has released hospital mortality data for cardiac surgery to the public since the early 1990s.1 In recent years several US state departments of public health, proprietary health quality rating firms, patient advocacy groups, and purchaser coalitions have launched new public reporting initiatives.2 3 In Britain hospitals,4 surgeon organisations,5 and recently the Guardian newspaper6 have published operational mortality data for individual heart surgeons. These initiatives share the basic assumption that patients will use such data to select higher quality hospitals for their surgery.
However, it is not clear to what extent patients know about and value such information. Several studies have found minimal changes in hospital caseloads after the public reporting of mortality data.7–9 In the only published study to question surgical patients about their decision making, only 12% of patients undergoing coronary artery bypass in Pennsylvania in 1996 were aware of a publicly released report on surgical mortality before their operation.10 Performance data have since become more widely available (particularly with the growth of the internet) and, with increasing media attention on patient safety, more visible to the public. None the less, the usefulness of performance data to patients remains unknown.
To better understand how patients make decisions about where to have surgery, we conducted a national survey of patients in the Medicare programme, the US federal government insurance programme that covers hospital costs for almost all US citizens aged 65 years and older, who had undergone an elective, high risk procedure. In addition to learning how the patients made their original decisions, we assessed whether they thought performance data to be relevant and to what extent such data would be useful in their future decisions.
Methods
Sample selection
Our goal was to learn how patients choose where to go for major surgery and whether performance data are likely to affect this decision. We focused on elective procedures because choice of hospital or surgeon was theoretically possible. We interviewed a random sample of Medicare beneficiaries who had undergone one of five major elective operations (abdominal aneurysm repair, heart valve replacement, or resection of the bladder, lung, or stomach because of cancer). We surveyed Medicare patients because Medicare covers well over half of all patients undergoing these procedures in the United States.11
Our goal was to get 100 interviews for each operation (we chose this number to ensure a confidence interval of at most ±10%). The five procedures differ considerably in procedure frequency and long term survival after surgery, so we sampled some operations more than others. The sampling fractions of patients for each operation were 4.2% for lung resection, 10.8% for gastrectomy, 14.5% for cystectomy, 1.7% abdominal aneurysm repair, and 1.1% for valve replacement.
Figure 1 details the steps of the sampling procedure. Using data provided by the Center for Medicaid and Medicare Services, we selected a random sample of beneficiaries (stratified within each type of surgery) who had a claim for one of the five elective operations in 2000. The centre then provided us with a list of the names and addresses of 2114 beneficiaries (eight of whom subsequently reported they had not had surgery and were considered ineligible). Of these, 1055 were alive in January 2004. To obtain telephone numbers, we sent the sample list of names and addresses to Telematch, an independent company that provides this service. We called Directory Assistance for those cases when Telematch failed to find telephone numbers. We were able to obtain contact information for 828 people (12 had non-US mailing addresses). We attempted to interview 785 patients who were not hospitalised or in a nursing home and who spoke English and were able to hear adequately. A total of 510 individuals without cognitive impairment (such as Alzheimer's disease) completed the telephone interview. We concluded people could not be contacted only after at least six attempts at different times of day and three further attempts two weeks later.
We calculated response rates using the two methods recommended by the American Association of Public Opinion Research.12 With the number of eligible survivors as the denominator, the response rate was 48% (510/1055). With the number of those who were able to participate, the response (or cooperation) rate was 68% (510/751). There was no significant difference in response rates across the five operations.
Interview protocol
Development—To learn what people thought about the decision of where to have surgery, we conducted two focus groups with people who had recently undergone major surgery. We conducted the focus groups and all subsequent survey development in collaboration with the Center for Survey Research, a professional survey research firm affiliated with the University of Massachusetts. We developed a draft survey instrument based on the focus groups' conclusions and on a previously published survey of patients who had undergone coronary bypass surgery in Pennsylvania.10 Experienced interviewers conducted five cognitive interviews with patients who had undergone surgery at Dartmouth Hitchcock Medical Center within the previous three years to ensure that the questions were understood and that the answers were meaningful. After revising the draft instrument, we conducted a pilot test of the telephone survey with 25 patients. These interviews were audiotaped and then coded to identify questions that were difficult for interviewers to read or for respondents to answer. We revised the survey on the basis of these results.
Instrument—The final survey had three sections. The first asked about experiences with major surgery (such as how the respondent decided where to have surgery, what factors influenced this choice, the respondent's perceptions of the hospital and the surgeon). The second section focused on respondents' knowledge and reaction to surgical performance data, specifically information on work volume (number of operations performed by individual surgeons or hospitals), patient mortality, and nurse:patient ratios. The third section asked for respondents' reactions to two scenarios—firstly, what advice they would give to a friend who needed major surgery and, secondly, their reactions to Medicare publishing a list of best hospitals for different operations.
Administration—In December 2003 potentially eligible respondents were sent a notification letter (as required by the Center for Medicaid and Medicare Services) and a second letter two weeks later stating that the Center for Survey Research would be calling. Interviews were conducted by professional interviewers at the Center for Survey Research's telephone facility from January through February 2004. All interviewers received special training on the purposes and procedures of this particular study, all underwent monitoring for quality control and feedback from a supervisor. The interviews took an average of 21 minutes (range 12-46 minutes). Answers to the survey were directly entered into the computer assisted telephone interviewing system by the interviewers.
Statistical analysis
We weighted the results for the five operations to account for the different probabilities of selection into our sample and the slightly different response rates. The weighted results differed by only 1% or 2% from the unweighted results. Because the weighted results assume that non-respondents would answer questions similarly to respondents, we chose to present the unweighted results. We calculated 95% confidence intervals using the binomial Wald function. We performed all analyses using Stata statistical software (version 9, StataCorp, College Station, TX).
Results
Table 1 shows the 510 respondents' characteristics. They had a mean age of 78 years (range 68-93), two thirds were men, and 91% were white. Most reported lower socioeconomic status: 38% reported a total household income less than $25 000, and only 25% had graduated from college or graduate school. Self reported health was low: only 7% rated their health as excellent.
Characteristics of 510 Medicare beneficiaries who had undergone elective, high risk surgery about three years earlier. Values are numbers (percentages) of 510 respondents unless stated otherwise
How respondents made their surgery decision
Most respondents had had time to consider where to go for surgery, were aware of other hospitals in their area to choose between, and felt involved in the decision making process. Specifically, 84% said they had at least a week, and 30% had more than a month, between being told they needed surgery and undergoing the surgery. Two thirds said they were involved in selecting the hospital, 41% making the decision equally with their doctor, and 24% deciding mainly on their own or with their family.
Since all respondents were Medicare beneficiaries, all had options with regard to where they had surgery; nonetheless, only 55% said there were other local hospitals where they could have gone. Overall, only 10% seriously considered going elsewhere for surgery (this percentage was the same for those who said there was another local hospital to go to). Few respondents (11%) looked for information to compare hospitals, most commonly turning to friends and family, their primary doctor, or the internet. Most stayed at local hospitals, 73% reporting their travel time to be less than an hour.
Rather than seeking quantitative information, most seemed to rely on hospital or surgeon reputation in deciding where to have surgery. Almost all respondents thought their hospital and surgeon had good reputations (94% and 88% respectively). Thirty one per cent said their hospital was “the best” in the area, 40% said “better than most,” 22% said “about the same,” and only 1% said it was “worse than most.” Eighty per cent and 79% respectively said that surgeon and hospital reputation were “extremely” or “very” important to their decision (fig 2). Other factors influencing patient decisions included having had prior care at the hospital (rated important by 42%) and the recommendations of family and friends (rated important by 28%). When asked why they thought their hospital had a good reputation, 64% of respondents said it was because of what their referring doctor had said, and 31% said so because of what family or friends had said (table 2). These results did not vary importantly across the five surgical procedures.
Importance of different factors in respondents' decisions about where to have major surgery.
Responses of 510 Medicare beneficiaries who had undergone elective, high risk surgery about three years earlier to questions about their decision making for their surgery and reactions to surgical performance to data
How respondents would advise others
We asked respondents how much various factors would influence the advice they would give to a friend about where to go for major surgery (fig 3). Of these factors, surgeon reputation was rated the most influential (78% said it would influence their advice “a lot”), followed by the hospital having “nationally recognised” surgeons (63%). When we asked whether performance data could influence decisions, substantial proportions of the respondents said the following items would influence them “a lot”—surgeon's work volume (58%), nurse:patient ratios (49%), hospital work volume (48%), and hospital operative mortality (45%). Moreover, 40% of respondents said they would switch from their chosen hospital if the surgical mortality for another hospital was a percentage point lower (that is, 3% v 4%).
We also sought to learn what people know about hospital work volume (a measure increasingly used as a proxy for mortality) and found that most people accepted the intuitive notion that “practice makes perfect”: 80% of respondents thought that the chance of surviving an operation was better at a high volume hospital than a low volume one. However, only 11% of respondents had heard of volume standards (that is, that there is a minimum number of a certain operation a hospital needs to perform each year to do them well). After we explained this concept, 82% of respondents said they would recommend their friend go to a different hospital if their chosen hospital did not meet the appropriate standard.
To learn if, and how, respondents would like to learn about hospital performance data, we told them to imagine that Medicare planned to create a list of the best hospitals for various surgeries. Most (59%) respondents thought that Medicare would create such a list to help patients receive better quality care, but 23% believed that the list's main purpose would be to help the government save money. Seventy one per cent of respondents said they might consult such a list if it existed (49% said they would be “very likely” to consult such a list). Only 2% said they would like to receive information about the best hospitals directly; 40% wanted such information only from their doctor, and 55% wanted it from their doctor and from other sources as well.
Discussion
Most respondents in our study said they relied primarily on the opinions of their referring physician or family and friends in choosing where to have surgery, rather than on surgical performance data. Few respondents seriously considered going to another hospital or looked for any kind of performance data. However, they considered such data, including indirect measures such as a surgeon's or hospital's work volume, to be meaningful and relevant, and thought these data could influence their decision making. But they did not want such data directly; they wanted to learn about the best hospitals from their physicians.
If patients are not using clinical performance data this could explain why attempts to direct patients to high quality hospitals and surgeons have had only minimal impact.1 8 9 13 For example, public reporting of mortality data for cardiac surgery in New York state failed to redirect patients to higher quality hospitals: in 1989, 9% of all patients were treated at hospitals with risk-adjusted mortality figures significantly higher than the state average, but in 1993, after the public release of hospital mortality figures, the proportion remained 10%.14 More recently, a study in California of patients undergoing diskectomy found that hospitals with below average complication rates experienced only modest, transient increases in surgical caseloads as a result of a public reporting initiative.8
Limitations of study
Given our target population—Medicare beneficiaries who had had major elective surgery three years previously—achieving a good response was extremely challenging. Since our sample did not include patients who died within three years of having surgery, the views of the sickest patients may be under-represented. It seems unlikely, however, that the sickest patients would have looked harder for hospital performance information than the less sick. Our response rate was good (48% among eligible survivors and 68% among those able to participate), but systematic bias between respondents and non-respondents is still possible. However, our respondents' demographics closely matched those of all Medicare patients who had undergone the five surgical procedures considered (including those who did not survive the three years) (data not shown).
Recall bias is a risk in a survey of people's recollections of decision making processes that occurred three years earlier. However, since the purpose of our questions was to draw on the insights of experienced patients, we think that the fact that they had had time to process and learn from what happened (and to reflect on what they might have done differently) is a strength of our study.
Implications of results
Although the respondents considered hospital performance data to be important, few looked for such information when deciding where to undergo high risk, elective surgery—a situation for which the data are particularly relevant. This finding is consistent with those of other surveys looking at how patients choose a health plan,15 16 select a hospital (for any services),17 or decide where to have coronary bypass surgery10: most patients agreed that performance information was important, but it had little influence their decisions.
There are four possible explanations for why performance data have so little influence in practice. Firstly, patients may not be aware that such data are available. That is, although the concept of “practice makes perfect” makes sense in the abstract, patients may not appreciate how this might be used when selecting a surgeon (such as looking for data on surgeon work volume).
Secondly, patients might not have ready access to the data when they are needed. In 2000 (when our respondents had surgery), individual hospitals' work volume and mortality data, based on Medicare claims, were available on the website healthgrades.com for most of the surgeries we included, and several states, such as New York,14 publicly reported risk adjusted performance data for individual hospitals and surgeons for cardiac surgery (mainly coronary artery bypass grafting but also valve surgery). However, patients might not have known that these data were available or how to access them.
Thirdly, even if they accessed the data, some patients might not have understood the data18 or might not have believed them. A focus group study including patients with recent inpatient experiences in the UK NHS found substantial mistrust in the government's hospital rankings.19 The credibility of health information from the internet, the media, or government has consistently ranked far below that of information from physicians, family, and friends in US and European surveys.20 21
The fourth (and most likely) reason that performance data had so little influence on our respondents is that many people, particularly those who are elderly,22 rely heavily or completely on the judgment of their referring physician.20 21 Many patients probably assume that their physician uses performance data when recommending a surgeon or hospital.
How to make better use of performance data
Some might interpret our results as evidence of the ineffectiveness of public reporting strategies. However, we think the problem lies in assuming that patients should be the primary target for surgical performance data. One alternative would be to have purchasers of health services take responsibility for ensuring that patients are referred to the best hospitals and surgeons. For example, the Leapfrog Group, a large coalition of employers and insurers, has targeted five surgical procedures for “evidence based hospital referral.”3 Although it also encourages public reporting, the Leapfrog Group's strategy includes selective contracting and payment-based mechanisms for directing more patients to high quality medical centres. Such strategies would no doubt be more effective in redistributing patients, but they would not allow for patient preferences. Some patients may have strong feelings about receiving care locally, even if this means forgoing care at a regional hospital with better performance data.23
Instead, we think performance data should be directed at referring physicians. If patients generally decide where to have surgery according to their referring physician's recommendation it is essential that referring doctors know how to choose well. The little that is known about how referring physicians choose surgeons for their patients does not suggest that performance data play a central part. For example, surgeons identified by their peers as “best doctors” for coronary artery bypass surgery did not have the lowest operative mortality.24 Nor do the attributes that create good peer reputations—training at a prestigious institution or a long practice record—correlate well with lower operative mortality.25 Surgeons' reputations may be strongly influenced by such prestige factors or by traits such as bedside manner or availability. We know of only two studies examining referring physicians' awareness, comprehension, or use of performance data in choosing surgeons. A survey of cardiologists in New York state found that they had read and found accurate the state's mortality data for coronary artery bypass surgery; moreover, over a third said the information influenced their referral patterns.26 In contrast, a survey of cardiologists and cardiac surgeons in Pennsylvania found that, although most were aware of the state's report of mortality data for coronary artery bypass surgery, few discussed these data with patients and few said the report influenced their referrals.9
What is already known on this topic
There is growing interest in providing patients with surgical performance data to help them select the highest quality surgeons and hospitals
However, publication of such performance data seems to have minimal impact
What this study adds
A survey of elderly US citizens who had undergone elective, high risk surgery found that most had relied on the opinion of their referring physician in choosing where to have surgery
Although few had looked for surgical performance data, they felt these data could influence their decision making—some saying they would switch hospitals to one with lower mortality figures
Respondents did not want such data on their own; they wanted to learn about the best hospitals from their doctor, highlighting the importance of making this information accessible to referring physicians
For referring physicians to be expected to identify the best surgeons for their patients, several issues must be addressed—ensuring the physicians appreciate the importance of performance data, making the relevant data easily available, identifying the barriers that inhibit physicians from using these data, and helping physicians understand how to communicate performance data to patients.
LMS and SW are associate professors of medicine and community and family medicine, Dartmouth Medical School, Hanover, NH 03755, USA. They are the joint first authors of this paper, and the order of their names is arbitrary. We thank Emily Finlayson for her work on the pilot survey for this project, and H Gilbert Welch for helpful comments on earlier drafts of this manuscript.
Footnotes
-
Contributors All authors participated in the conception and design of the study, analysis and interpretation of data, and drafting and revising the article. SW is guarantor for the study.
-
Funding LMS and SW were supported by Veterans Affairs advanced research career development awards in health services research and development, and by Robert Wood Johnson generalist faculty scholar awards. This study was supported by a grant from the Agency for Health Care Research and Quality (R03 HS13049-01).
-
Competing interests JDB is a paid consultant and chair of the expert panel on evidence based hospital referral for the Leapfrog Group.
-
Ethical approval This project was approved by the institutional review boards at Dartmouth Medical School, Hanover, NH, the University of Massachusetts, Boston and the Center for Medicaid and Medicare Services.