Long term effect of depression care management on mortality in older adults: follow-up of cluster randomized clinical trial in primary care

BMJ 2013; 346 doi: http://dx.doi.org/10.1136/bmj.f2570 (Published 5 June 2013)
Cite this as: BMJ 2013;346:f2570

Recent rapid responses

Rapid responses are electronic letters to the editor. They enable our users to debate issues raised in articles published on bmj.com. Although a selection of rapid responses will be included as edited readers' letters in the weekly print issue of the BMJ, their first appearance online means that they are published articles. If you need the url (web address) of an individual response, perhaps for citation purposes, simply click on the response headline and copy the url from the browser window.

Displaying 1-6 out of 6 published

I am still waiting for the authors to provide a simple table of the distribution of deaths and missing data across the three sites and the practices nested within them. These data should be readily available and would allow independent evaluation of whether the statistical controls that the authors claimed they applied are even possible.

Unless they can produce such data, we need to consider seriously that statistical controls were not possible and the results that are reported are erroneous or simply spurious.

Competing interests: None declared

James C Coyne, Professor

University Medical Center, Groningen, 1032 Spruce St, Apt 200, Philadelphia, PA 19107 USA

Click to like:

I appreciate the willingness of Professor Gallo and colleagues to discuss further their fascinating claims that a collaborative care intervention for depression in primary care decreased mortality among depressed persons, and that survival effects were concentrated in patients with cancer. However, the reply of Professor Gallo and colleagues raises some issues that are probably best resolved with making some basic data available to readers in a table.

Professor Gallo reports that adjustments were made for unmeasured practice characteristics and case mix and that they additionally statistically accounted for clustering. Nowhere in the BMJ article or in numerous other reports of outcomes for PROSPECT was there any indication how this could have been done or what effects any adjustments had on the results that were reported. There were substantial demographic differences across the three sites for PROSPECT--Pittsburgh, Philadelphia, and Cornell--as well as the nonrandom sample of practices nested within these three sites. Site and practice differences could be expected to have substantial effects on outcomes, particularly in a study in which the intervention group, but not the control group had depression treatment reimbursed.

Highly influential baseline differences are thus quite likely. Given the relatively small number of deaths that are being explained, especially the deaths due to cancer, it is difficult to comprehend how these features of a multisite study with a nested, nonrandom sample of practices could be controlled in a way that meaningful inferences could be made about the patterning of deaths. In particular, uneven dispersion of a relatively small number of already diagnosed cases of across these sites and practices might preclude inferences about the effects of assignment to the intervention condition on subsequent mortality.

I encourage Professor Gallo and colleagues to make available to the readers of BMJ a simple table of the distribution of diagnoses of cancer across sites and practices. It would also be useful to include the distribution of missing outcome data, which cannot be assumed to be random. Only then, could readers independently decide if there are substantive findings to continue to discuss, or only much being made of spurious findings.

Competing interests: None declared

James C Coyne, Professor

University Medical Center, Groningen, P.O. Box 196, 9700 AD Groningen, The Netherlands

Click to like:

Comments provide an opportunity to make several important points. The project reported was reviewed by two NIH study sections and funded by the National Institute of Mental Health to specifically look at all-cause mortality. The design incorporated several features that make the design better than most. In addition to depressed patients, the original study followed a representative sample of patients who did not meet criteria for major or minor depression, allowing us to compare mortality and other outcomes among patients from the same practices. Doing so adjusts for unmeasured practice characteristics and case-mix. In addition, we statistically accounted for clustering within practice as well as patient-level characteristics consistent with the analysis of a group-randomized trial. The public health significance of chronic medical conditions and depression has come to the fore with the recognition that most persons with depression die from the adverse consequences of cardiovascular disease, diabetes, and cancer, through multiple pathways. Most primary care physicians, often providing medical care with limited resources, would disagree with the notion that their care of patients amounts to “nothing.” The World Health Organization, the Institute of Medicine, professional organizations of physicians, and others have recognized the need for integrating primary health care and mental health. We were careful to state that the observations we report may not be due to the intervention effect, and provided our reasons either way.

Competing interests: None declared

Joseph J. Gallo, Professor

with coauthors Knashawn Morales, Hillary Bogner, Patrick J. Raue, Jarcy Zee, Martha L. Bruce, Charles F. Reynolds III

Johns Hopkins University, Baltimore, Maryland

Click to like:

Gallo and colleagues [1] conducted post hoc analyses of mortality in PROSPECT, a clinical trial that evaluated collaborative care for suicidal ideation among older primary care patients [2]. They interpret their findings as indicating the complex intervention reduced mortality by almost a quarter, although the confidence intervals for the hazard ratio for this reduction included 1.0. The authors nonetheless identified the source of this effect in reduced mortality among depressed cancer patients and declared “changes in the financing and integration of primary care and mental healthcare being considered worldwide take on new urgency with the demonstration that depression care management saves lives.”
Their analyses are inappropriate to their study design and their interpretation of results ignore a key feature of the design of the original PROSPECT study. These analyses are nonetheless consistent with equally inappropriate strategies used in a number of reports of outcomes from this clinical trial. Basically, the study was a multisite randomized clinical trial with substantial differences among the three institutions conducting the trial, and individual primary care practices associated with these sites were the unit of randomization. The validity of any analyses of outcomes depends on taking this multisite character of the trial into account, along with the nesting of primary care practices. Failure to do so increases substantially the probability of a Type 1 error in which main site differences or interactions are misinterpreted as effects of the intervention. Kraemer [3] has shown that failure to take a multisite character to a clinical trial can lead to spurious results. As a result of Simpson’s paradox and other statistical artifacts, significant effects can be obtained when they were no significant effects for the individual sites, and no significant effects can be obtained when there were significant effects at each of the individual sites.
I was Principal Investigator on a research core of an NIMH Advanced Center for Intervention and Services Research (ACISRs) that was involved in the planning of the intervention for the clinical trial. An early article on which I was an author [4] further indicates why the analyses by Gallo in his colleagues and also used in other evaluations of the outcome of PROSPECT were inappropriate, particularly when taken with another generally ignored early paper from the project [5]. We noted that the clinical trial involved recruiting primary care practices in the immediate area of the ACISRs of the three different institutions, the University of Pittsburgh, the University of Pennsylvania, and Cornell University. Conveying information that is not readily available elsewhere, we also noted that each ACISR
recruited six of the primary care practices participating in the ambulatory care networks. The resulting total of 18 practices including four of the academic type and 14 of the community type; eight were located in urban and 10 in suburban settings; they served a wide-ranging racial/ethnic patient population. The six practices in each IRC [ACISR] were paired so as to share similar characteristics. A practice in each pair then was randomized to the study’s intervention arm and the other practice in the usual care arm. Consequently, three practices in each of the geographic areas were designed as intervention sites and three practice in each of their geographic areas as usual care sites (Bruce and Pearson, 1999).
On a priori basis, it should have been decided that analyses should have taken into account the contribution of the multisite design and of the practices being nested in the three sites. The Bruce and Pearson article [5], which is essentially a statement of the study’s protocol, contains a table breaking down practice characteristics by site that underscores this necessity. Table 1 indicates that two of the Cornell sites were large academic practices and two of the Pittsburgh sites were academic, but smaller; all of the other practices were small private practices. The table also indicates that the only two rural practices were associated with Pittsburgh. There were substantial differences in the percentage of minority patients, with four of the Pennsylvania practices having 58 to 86% minority patients. Only two of the Cornell practices even come close with 50 to 60% minority patients, and all of the rest of the practices in the study had from 2 to 21% minority patients.
Subsequent analyses of the PROSPECT data taking into account sociodemographic characteristics and yet more evidence of the importance of taking site into account, if any further justification is needed. One report [6] emphasized the importance of education in moderating the effects of the intervention on depression. For instance, the intervention increased rates of antidepressant use by 14.2% in the college group compared to a lack of effect in the no-college group. Another report [7] indicated the importance of financial strain, with patients reporting the greatest financial strain having an intervention having a differential intervention effect of -4.5 Points on the Hamilton Depression Rating Scale. Similarly, neighborhood poverty also moderated effects of the intervention. Taken together with the strong demographic differences associated with site noted in the Bruce and Pearson paper [5] these findings suggest we should expect to find strong site effects and that any interpretations of the overall effects of the trial should take these into account. Basically we are dealing with the necessity of considering an interaction effect where reports of results of a study have focused only on main effects for the intervention.
Any substantive interpretation of the outcome of the trial, particularly any put to use in drawing clinical and public health implications, should also take into account that patients assigned to the intervention group received free treatment, whereas patients assigned to the control group had to pay for treatment. This difference was associated with patients in the intervention group being much more likely to get either medication or psychotherapy, and undoubtedly in them getting more visits with primary care physicians.
The PROSPECT was unusual in providing depression care for an entire two-year period. Although effects of the intervention on suicidal ideation and depression had largely dissipated by 12 months, patients assigned to the intervention group nonetheless got free visits to their primary care physician for depression care for the two-year period, whereas patients assigned to the routine care control group had to pay for any visits. There were marked group differences in receipt of both medication and psychotherapy, and the issue can be raised whether the “routine care” group was simply inadequate care or, for many patients, no depression care. If so, there would be implications for interpreting the effects of an intervention that only had to provide more than inadequate or nothing.
A visit to a primary care physician is an opportunity to be asked and to volunteer information about other health issues that the patients assigned to routine care did not have. This difference in service utilization, rather than specifically depression care, might account for any mortality differences observed in the sample. It is notable the trial did not have mortality as a primary outcome. It was actually designed with suicidal ideation as the primary outcome, but was soon discovered that there was so little suicidal ideation in the sample that the outcome measure had to be dichotomized with no suicidal ideation being compared to any item being endorsed. This distinction lacks practical clinical significance. Moreover, there were only one patient in the intervention group and one patient in the control group who committed suicide, and so suicide was not a significant source of mortality.
A systematic review, meta-analysis, and meta-regression of trials of collaborative care revealed significant overall effects of these enhancements of care for depression [8]. However, effects were limited to studies conducted in the United States, with collaborative care interventions not having an impact on depression outcomes in Europe. This may point to the importance of health disparities and lack of medical care in the United States determining the efficacy of improving care for depression, particularly when the collaborative care interventions involves free care.
In summary, it is always hazardous to attempt to interpret post-hoc the outcomes of clinical trials that were not designed with the particular outcome as either a primary or secondary outcome, and all the more so when the confidence intervals for the post-hoc effect include 1.0. It is also inappropriate to analyze the results of clinical trials without taking into account their multisite nature and the nesting of practices, the unit of randomization, within sites. If one nonetheless wants to proceed, ignoring complications, one should at least take into account the crucial, perhaps most crucial difference between the intervention and control group in terms of effects on mortality, the payment for treatment. At any rate, the claim of Gallo and colleagues that their study demonstrates a new urgency in implementing collaborative care programs for depression seems at least a bit exaggerated.
1. Gallo JJ, Morales KH, Bogner HR, Raue PJ, Zee J, Bruce ML, et al. Long term effect of depression care management on mortality in older adults: follow-up of cluster randomized clinical trial in primary care. BMJ 2013;346:f2570.
2. Bruce ML, Pearson JL. Designing an intervention to prevent suicide: PROSPECT (Prevention of Suicide in Primary Care Elderly: Collaborative Trial). Dialogues Clin Neurosci 1999;1(2):100-12.
3. Kraemer HC. Pitfalls of multisite randomized clinical trials of efficacy and effectiveness. Schizophrenia Bulletin 2000;26(3):533-541.
4. Schulberg HC, Bryce C, Chism K, Mulsant BH, Rollman B, Bruce M, et al. Managing late-life depression in primary care practice: a case study of the health specialist’s role. Int J Geriatr Psychiatry2001;16:577-84.
5. Bruce, M. L., & Pearson, J. L. (1999). Designing an intervention to prevent suicide: PROSPECT (Prevention of suicide in primary care elderly: Collaborative trial). Dialogues in Clinical Neuroscience, 1(2), 100.
6. Bao YH, Alexopoulos GS, Casalino LP, Ten Have TR, Donohue JM, Post EP, et al. Collaborative Depression Care Management and Disparities in Depression Treatment and Outcomes. Archives of General Psychiatry;68(6):627-636.
7. Gilman SE, Fitzmaurice GM, Bruce ML, Ten Have T, Glymour MM, Carliner H, et al. Economic Inequalities in the Effectiveness of a Primary Care Intervention for Depression and Suicidal Ideation. Epidemiology;24(1):14-22.
8. Bower P, Gilbody S, Richards D, Fletcher J, Sutton A. Collaborative care for depression in primary care - Making sense of a complex intervention: systematic review and meta-regression. British Journal of Psychiatry 2006;189:484-493.

Competing interests: I served as Principal Investigator of the Research Core on a NIMH Advanced Center for Intervention and Services Research (1P30MH066270-01; http://search.engrant.com/project/GXwQU7/coreprincipal_research_core) that designed parts of the original PROSPECT trial. I was not involved in designing analyses of outcome data in subsequent reports.

James C Coyne, Professor

University Medical Center, Groningen, P.O. Box 196, 9700 AD Groningen, The Netherlands

Click to like:

Recently, Gallo et al. (1) reported that 214 older primary care patients with major depression randomized to a depression care management intervention in the PROSPECT trial had significantly lower risk of mortality after a median follow-up period of 98 months compared to 182 patients with major depression in usual care (hazard ratio = 0.76, 95% confidence interval [CI] 0.57 to 1.00). They also reported that patients with major depression in usual care were more likely to die than patients without depression (hazard ratio = 1.90, 95% CI 1.57 to 2.31), but that this was not the case for patients with major depression compared to patients without depression in the intervention arm of the trial (hazard ratio 1.09, 95% CI 0.83 to 1.44). A number of factors, however, suggest that these results should be interpreted with caution and that there is substantial risk that they will not be replicated in other patient samples.

First, Gallo et al. wrote, “Our study was designed to focus on all cause mortality” (p. 4). However, neither a 1999 article that described the trial protocol (2) nor the trial registration (NCT00000367) mentions this outcome. As described in the trial protocol, all planned outcomes were related to depression and suicide. Gallo et al. did not report how many patients died of suicide, but a 2007 PROSPECT publication (3) stated that, after a median follow-up of 53 months, only 1 patient in the intervention arm and none in the usual care arm died of suicide. The mortality outcome appears to be a post hoc outcome that was added at some point after initiation of the trial.

Second, whereas the published study protocol (2) emphasized the importance of including patients with both major and minor depression in analyses, and previous PROSPECT reports provided shorter-term mortality results for the entire group of depressed patients (3, 4), Gallo et al. only provided mortality outcome results for patients with major depression in the intervention versus major depression in the control arm, patients with major depression in the intervention arm versus non-depressed patients in the intervention arm, and patients with major depression in the usual care arm versus non-depressed patients in the usual care arm. Thus, the analyses reported were a series of subgroup analyses, but only a portion of the subgroup analyses that could have possibly been reported.

Third, Gallo et al. reported results after adjusting for 12 covariates, but did not report unadjusted trial results. Unadjusted hazards ratios cannot be generated from the information reported by Gallo et al., but unadjusted relative risk of mortality can be calculated. As shown in Table 1, there is a higher rate of overall mortality in the intervention arm (35%) compared to the usual care arm (31%), and there are no statistically significant differences between trial arms. Gallo et al. emphasized that patients with major depression in the intervention arm were not more likely to die than non-depressed intervention patients, but that in the usual care group the risk was almost double. However, the rate of death for patients with major depression was the same in the intervention and usual care trial arms and only slightly elevated compared to non-depressed patients. The only notable difference is that non-depressed patients in the intervention arm of the trial died at a somewhat higher rate (33%) than non-depressed patients in the usual care arm (28%). It appears that liberal covariate adjustment may have turned a set of null results into statistically significant findings.

Related to this last point, PROSPECT study investigators have used a number of different analytical approaches over time, the choice of which has influenced the results that they have reported. The longer-term follow-up that was recently published in BMJ utilized a P < 0.05 inclusion criterion for covariates. In the first PROSPECT publication on mortality (3), Gallo et al. similarly used a P < 0.05 criteria for inclusion of covariates. They reported that 5-year all-cause mortality was reduced in patients with major depression, but not in those with either minor depression or no depression and that effects in patients with major depression were “limited to deaths due to cancer” (p. 689). A subsequent PROSPECT publication (4) that used the same patient data and follow-up period switched from the P < .05 covariate entry criterion to a P < 0.10 entry criterion and included a somewhat different group of covariates. The second study did not mention the findings of the first study, but reported that the intervention reduced 5-year all-cause mortality among diabetic patients with major or minor depression, but not among depressed patients without diabetes. Cancer was no longer mentioned. Both articles reported that the intervention did not have a significant effect on mortality prior to adjustment for covariates. The article by Gallo et al. published recently in BMJ did not report unadjusted results, but there is reason to be concerned that they would similarly be non-significant.

Unplanned, exploratory analyses can play an important role in generating hypotheses and identifying areas that may warrant additional research. However, posthoc subgroup analyses carry a high risk of spurious findings and are generally unlikely to be confirmed in subsequent studies (5). Gallo et al. did not provide information in their article to alert readers that they were reporting unplanned, exploratory analyses that, typically, do not replicate well. The authors of the PROSPECT study have now published 3 such articles on mortality outcomes with PROSPECT data, and none has provided this information – or even disclosed the results from the other articles that also reported morality outcomes from PROSPECT.

Substantially greater reporting transparency is needed.

References
1. Gallo JJ, Morales KH, Bogner HR, Raue PJ, Zee J, Bruce ML, et al. Long term effect of depression care management on mortality in older adults: follow-up of cluster randomized clinical trial in primary care. BMJ 2013;346:f2570.
2. Bruce ML, Pearson JL. Designing an intervention to prevent suicide: PROSPECT (Prevention of Suicide in Primary Care Elderly: Collaborative Trial). Dialogues Clin Neurosci 1999;1(2):100-12.
3. Gallo JJ, Bogner HR, Morales KH, Post EP, Lin JY, Bruce ML. The effect of a primary care practice-based depression intervention on mortality in older adults: a randomized trial. Ann Intern Med 2007;146(10):689-98.
4. Bogner HR, Morales KH, Post EP, Bruce ML. Diabetes, depression, and death: a randomized controlled trial of a depression treatment program for older adults based in primary care (PROSPECT). Diabetes Care 2007;30(12):3005-10.
5. Moher D, Hopewell S, Schulz KF, Montori V, Gotzsche PC, Devereaux PJ, et al. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ 2010;340:c869.

Competing interests: None declared

Brett D. Thombs, Associate Professor

Jewish General Hospital, Montreal, Quebec, H3T 1E4

Click to like:

7 June 2013

Life is lonely, and there are two kinds of loneliness: the absence of loved ones, and the presence of unloved ones. The former is haunting, and the latter is daunting. What should we do? We should cultivate a reliable relationship with ourselves, because we are the only loved one who will always be with us.

Competing interests: None declared

Hugh Mann, Physician

Retired, Eagle Rock, MO, USA

Click to like:

THIS WEEK'S POLL