- Scott R Weingarten, president ()a,
- James M Henning, associate directorb,
- Enkhe Badamgarav, clinical project leadera,
- Kevin Knight, research managera,
- Vic Hasselblad, associate research professorc,
- Anacleto Gano Jr, research associatea,
- Joshua J Ofman, enior vice president researcha
- aZynx Health, 9100 Wilshire Boulevard 665E, Beverly Hills, CA 90212, USA
- bTAP Pharmaceutical Products, Lake Forest, IL, USA
- cDuke Clinical Research Institute, Duke University, Durham, NC, USA
- Correspondence to: S Weingarten
- Accepted 6 September 2002
Objective: To systematically evaluate the published evidence regarding the characteristics and effectiveness of disease management programmes.
Data sources:Computerised databases for English language articles during 1987-2001.
Study selection: 102 articles evaluating 118 disease management programmes.
Main outcome measures: Pooled effect sizes calculated with a random effects model.
Results: Patient education was the most commonly used intervention (92/118 programmes), followed by education of healthcare providers (47/118) and provider feedback (32/118). Most programmes (70/118) used more than one intervention. Provider education, feedback, and reminders were associated with significant improvements in provider adherence to guidelines (effect sizes (95% confidence intervals) 0.44 (0.19 to 0.68), 0.61 (0.28 to 0.93), and 0.52 (0.35 to 0.69) respectively) and with significant improvements in patient disease control (effect sizes 0.35 (0.19 to 0.51), 0.17 (0.10 to 0.25), and 0.22 (0.1 to 0.37) respectively). Patient education, reminders, and financial incentives were all associated with improvements in patient disease control (effect sizes 0.24 (0.07 to 0.40), 0.27 (0.17 to 0.36), and 0.40 (0.26 to 0.54) respectively).
Conclusions: All studied interventions were associated with improvements in provider adherence to practice guidelines and disease control. The type and number of interventions varied greatly, and future studies should directly compare different types of intervention to find the most effective.
What is already known on this topic
Disease management programmes have gained popularity in recent years as a means of improving the quality and efficiency of care of patients with chronic diseases
A limited number of trials have documented the effectiveness of disease management in specific situations, but uncertainty remains about its overall value and which interventions are most effective
What this study adds
Programmes using education, feedback, or reminders for healthcare providers produced significant improvements in provider adherence to care guidelines
Programmes using the provider strategies or education, reminders, or financial incentives for patients improved disease control
Further study is needed to assess the relative effectiveness of the different strategies
Chronic diseases account for billions of dollars in annual medical expenditures. In the United States asthma, depression, and diabetes are estimated to account for $5.1bn (£3.4bn, €5.2bn), $12.4bn, and $44bn respectively, in annual direct medical costs.1–3 Loss of work time and decreased worker productivity contribute to indirect costs. Unsurprisingly, therefore, there has been much interest in systematically improving the quality and reducing the cost of caring for patients with chronic illness.
Disease management programmes have proliferated recently as a means of improving the quality and efficiency of care for patients with chronic illness. Ellrodt et al defined disease management as a multidisciplinary approach to care for chronic diseases that coordinates comprehensive care along the disease continuum across healthcare delivery systems.4 Epstein defined disease management as a population based approach to health care that identifies patients at risk, intervenes with specific programmes of care, and measures outcomes.5 These programmes may represent an important improvement in the quality and value of health care for patients with chronic illnesses. However, disease management programmes can be costly to develop, implement, and evaluate. According to the Disease Management Association of America, an estimated $1bn was spent in 1999 to develop and implement disease management programmes.6 Despite the investment, evidence supporting the effectiveness of disease management is sparse. A limited number of published trials have documented the effectiveness of disease management in specific situations, but uncertainty remains about its overall value. Understanding which interventions are most effective could guide the development of disease management programmes.
Several qualitative reviews have described the effects of interventions such as educational programmes, giving feedback to healthcare providers, and patient financial incentives to promote adherence to practice guidelines. Oxman et al reviewed 102 trials and concluded that a wide range of interventions may improve practice but that there are no “magic bullets.”7 Davis et al showed that educational interventions improved physician performance and possibly patient outcomes.8 Mugford found provider feedback effective when part of an overall implementation strategy.9 There have also been some qualitative descriptions of interventions to implement practice guidelines.10–12 Although these reviews covered a wide range of interventions, no review focused on interventions used in disease management programmes for patients with chronic illness. Additionally, there have been only limited evaluations of the quantitative effects of specific types of interventions to complement qualitative and descriptive information. This study reviews the types of interventions used in published trials of disease management programmes and provides quantitative and qualitative evaluation of the evidence regarding the effectiveness of different types of intervention.
We performed a systematic review of the medical literature to identify studies evaluating the effectiveness of disease management programmes in improving care or reducing costs for patients with common chronic conditions. In collaboration with a librarian experienced in searching computerised bibliographic databases, we conducted a search of the Medline, HealthStar, and Cochrane databases for English language articles published between January 1987 and June 2001. The search used the following medical subject heading (MeSH) terms: patient care team, patient care planning, primary nursing care, case management, critical pathways, primary health care, continuity of patient care, guidelines, practice guidelines, disease management, comprehensive health care, ambulatory care, and the title words “disease state management” and “disease management” (see appendix 1 on bmj.com). Hand searches of bibliographies from relevant articles and reviews and consultations with experts in the subject yielded further references.
Our working definition of disease management was “an intervention designed to manage or prevent a chronic condition using a systematic approach to care and potentially employing multiple treatment modalities.” We defined a guideline or systematic approach to care as systematically developed statements to assist practitioner and patient decisions about appropriate health care for a specific clinical circumstance. 4 13 To determine whether a programme incorporated a systematic approach to care, we searched for keywords such as guidelines, protocols, algorithms, quality improvement programmes, care plans, and standardised patient and provider education. We excluded programmes aimed exclusively at evaluations of single treatment methods (such as psychotherapy or specific drugs) and drug compliance programmes. We rejected articles if they included only paediatric cases or if they were reviews, case reports, editorials, letters, or meeting abstracts. We also rejected articles if they did not use acceptable experimental or quasi-experimental study designs as defined by the Cochrane Effective Practice and Organisation of Care (EPOC) Group's criteria for acceptable study designs,14 or if they did not report sufficient information to allow for estimation of at least one relevant measure of programme effect and its variance. Failure to meet this criterion could be due to inadequate reporting of results or to lack of an appropriate comparison group.
Based on these explicit inclusion and exclusion criteria for titles, abstracts, and articles, two reviewers trained in health services research and the principles of critical appraisal independently reviewed random samples of titles, abstracts, and reviews. Inter-rater agreement was assessed using the κ statistic, and reviews were split between reviewers if a sufficient level of agreement was achieved (κ>0.7). Results from accepted articles reporting results for disease management programmes were used to address study objectives.
Classification of interventions
Intervention components were classified as follows:
Provider education—Materials or instruction given to healthcare providers regarding appropriate care for patients with the condition targeted by the programme
Provider feedback—Information given to healthcare providers regarding the specific care or results of care received or experienced by their patients
Provider reminders—Prompts given to providers to perform specific patient care tasks
Patient education—Materials and instructions issued to patients providing information on their condition and how it could be managed
Patient reminders—Prompts given to patients to perform specific tasks related to care for their condition
Patient financial incentives—Payments (direct monetary payments, discounts, or services) made to patients for achieving specific treatment related goals.
We also searched for programmes using provider financial incentives but found no such interventions. The proportions of interventions using each of the different interventions were described, as were the proportions with one, two, three, or four components (no intervention included more than four components).
Classification of process measures or outcomes
Our analyses focused on measures of provider adherence to guidelines as key processes of care and on measures of disease control as key outcomes of care.
Extraction of data
Using a standardised abstraction form, we collected data describing interventions, components used, study design, population characteristics, sample size, intervention strategies, and measures of programme effects on processes and outcomes of care from unmasked articles that met our inclusion criteria. For each disease and condition, we had a defined method for selecting each disease specific process or outcome indicator as a measure of disease control (see appendix 2 on bmj.com). We treated multiple reports of results from the same study as a single programme evaluation. Reports with multiple intervention arms contributed one observation for each. When separate results were reported for mutually exclusive subgroups, each subgroup contributed an observation, and the overall results were not included.
When appropriate, we used changes over baseline values rather than follow up values in our analyses. Variances for changes over baseline were often not reported, in which case we assumed that the variance for the change was equal to the average of the variances of the baseline and follow up distributions if both were given. If the baseline variance was not provided, we assumed that the variance for the change was equal to the variance of the follow up distribution.
We calculated effect sizes—defined as a difference between the means of the treatment and control arms divided by the pooled estimate of the standard deviation (continuous variables) or the log odds ratio multiplied by a constant variance term (binary outcomes)15—for each study outcome to allow pooling of similar outcomes.16–19 We constructed effect sizes so that positive numbers denoted treatment benefit. The method of interpreting the clinical importance of different effect sizes has been reported previously.20
We used the more conservative random effects, empirical bayesian method of Hedges and Olkin to pool the estimated effects.18 We pooled results for interventions incorporating each intervention component for disease control measures.21 For measures of provider adherence to care, we pooled results for interventions containing each intervention component except patient education (we concluded a priori that patient education would have minimal effect on provider adherence to guidelines). We also pooled results for each of these measures for interventions containing one, two, three, and four intervention components. When more than one process or outcome result within a category was reported for the same group of subjects, we used the one associated with the smallest effect size. Results are reported as pooled effect sizes with 95% confidence intervals.
Our initial search strategy identified 16 917 references published between January 1987 and June 2001 (figure). We accepted 2963 titles for further screening, and 581 abstracts met our explicit inclusion criteria. Of these abstracts, 493 (85%) failed to meet our inclusion criteria when the articles were reviewed. Bibliographic hand searches and expert consultation yielded an extra 51 articles for review, of which 14 were accepted. In total 102 studies met our criteria for inclusion. Multiple reports of the same intervention reduced the count of studies by 17.
The 102 accepted studies evaluated 118 discrete intervention programmes and reported 352 estimates of programme effect. In 67 (19%) cases, these estimates were selected from two or more reported results within a single subdomain based on rules delineated in the methods section. Selected programmes were heterogeneous with regard to the types of intervention they used (table 1). Of the 118 programmes, 92 used patient education, 47 used provider education, 32 used provider feedback, 28 used patient reminders, 19 used provider reminders, and only six used financial incentives for patients.
Patient education was an integral part of most programmes studied (9/9 for congestive heart failure, 3/3 for chronic pain, 6/7 for back pain, 6/7 for chronic obstructive pulmonary disease, and 21/26 for diabetes). However, provider education was widely used for only a few conditions—depression (17/25), diabetes (9/26), hypertension (5/8), and hyperlipidaemia (4/7). The six programmes that used financial incentives for patients were for asthma, depression, hyperlipidaemia, and hypertension. Patient reminders were often used in programmes for patients with congestive heart failure (4/9), coronary artery disease (3/6), and diabetes (9/26).
The number of different interventions included in each disease management programme varied, with 48 programmes using a single intervention, 41 using two, 22 using three, and seven using four interventions (table 2
Impact of programmes with provider interventions
We identified 32 programmes addressing six conditions that included provider education and evaluated disease control (table 3). 22–42 51 54 56 57 76 85 Twelve of these (38%) produced significant improvements in disease control. 24 30 31 33 34 36–42 Disease management for depression and diabetes had the highest percentage of programmes that produced significant benefits (40% (6/15) and 25% (2/8), respectively). Overall, programmes that included provider education showed a modest but significant improvement in disease control (effect size 0.35 (95% confidence interval 0.19 to 0.51)).
Twenty three programmes addressing asthma, depression, diabetes, renal disease, hyperlipidaemia, and hypertension included provider feedback and evaluated disease control (table 3). 23 24 27 29 31–33 42–52 54 84 Of these, nine (39%) showed significant improvements in disease control (eight of which were depression programmes). 24 31 33 42 44–47 84 Overall, programmes with provider feedback produced a small but significant improvement in disease control (effect size 0.17 (0.1 to 0.25)).
Fourteen programmes incorporating provider reminders evaluated disease control (table 3). 29–31 36 39 43 50 53–55 85 Of these, six (43%) significantly improved disease control. 30 31 36 39 53 55 Programmes with provider reminders were effective for patients with diabetes and hypelipidaemia. Overall, the programmes made a small but significant improvement in disease control (effect size 0.22 (0.1 to 0.37)).
Provider adherence to guidelines
Twenty four programmes that included provider education assessed measures of provider adherence to guidelines, 24 25 28–33 41 42 54 56–61 of which 12 (50%) significantly improved these measures (table 4). 28 30 31 33 41 42 58–61 Disease management for depression had the highest percentage of programmes with significant benefit (9/14). Overall, programmes with provider education components significantly improved provider adherence to guidelines (effect size 0.44 (0.19 to 0.68)).
Sixteen programmes with provider feedback evaluated provider adherence to guidelines (table 4). 24 29 31–33 4246 49 54 58 59 62 63 Nine of these (56%) significantly improved provider adherence, 31 33 42 46 58 59 with programmes for depression being most successful (8/12). Overall, these programmes were effective in improving provider adherence (effect size 0.61 (0.28 to 0.93)).
Ten programmes with provider reminders examined provider adherence (table 4). 29–31 54 59 63 64 Six (60%) significantly improved adherence, 30 31 59 64 with programmes for depression being most successful (5/8). Overall, these programmes were also effective in improving provider adherence (effect size 0.52 (0.35 to 0.69)).
Impact of programmes with patient interventions on disease control
Fifty five programmes included patient education and evaluated disease control (table 5). 22–27 29–31 33 34 36 37 42 44–47 49 50 52 53 56 65–88 91 Of these, 24 (44%) significantly improved disease control. 24 30 31 33 34 36 37 42 44–47 53 66 67 69 70 73 74 79 82–84 88 The highest percentage of programmes producing significant improvements were among those for depression (10/18), asthma (2/3), and hypertension (2/2). Overall, patient education produced a small but significant improvement in disease control (effect size 0.24 (0.07 to 0.40)).
Sixteen programmes including patient reminders evaluated disease control (table 5). 22 34 43 46 47 49 53 54 68 71 81 82 87 89 90 Six (38%) significantly improved disease control. 34 46 47 53 82 89 The highest percentage of programmes producing significant improvements were among those for depression (2/3) and diabetes (4/7). Overall, programmes with patient reminders produced a small but significant improvement in disease control (effect size 0.27 (0.17 to 0.36)).
Four programmes incorporating patient financial incentives evaluated disease control (table 5). 33 52 83 91 Three, including one depression programme33 and two hypertension programmes, 83 91 significantly improved disease control. Overall, these programmes seemed effective in improving disease control (effect size 0.40 (0.26 to 0.54)).
Our study showed that many different interventions—including provider education, provider feedback, provider reminders, patient education, patient reminders, and patient financial incentives—were associated with improvements in provider adherence to guidelines and patient disease control. However, since existing studies do not directly compare different interventions, less is known about which interventions produce the greatest relative improvements in care.
There should be careful consideration of the most effective strategies for changing provider and patient behaviour, since it is difficult to postulate any improvements in patient care if both provider and patient behaviour remains unchanged. Our study showed that many different types of intervention are being used in disease management programmes, with patient education being the commonest. Also, 59% of disease management programmes used two or more interventions, possibly because multiple interventions are thought to be more likely to be successful than single interventions. The wide variety interventions used may reflect the paucity of available information to guide programme development and to define an optimal strategy. It is ironic that disease management programmes are designed to reduce unexplained variations in care, yet there are large and unexplained variations in the design, development, and implementation of disease management programmes.
Because of the promise that disease management holds for improving patient care, about $1bn is invested in disease management programmes in the United States each year.4 The National Committee on Quality Assurance (NCQA) in the United States requires health plans to submit data on two disease management programmes each year for consideration of accreditation. However, this investment in disease management should be guided by information on how to optimise the benefits of these programmes. Unfortunately, disease management programme developers have had limited qualitative or quantitative information about which interventions achieve the greatest benefits, and programmes are therefore highly variable in design.
Strengths and limitations of study
To the best of our knowledge, our study is the first comprehensive attempt to evaluate the effectiveness of different disease management programmes for patients with chronic illness. Our study brought together disparate information of disease management, to allow for qualitative and quantitative interpretation. We evaluated 16 917 different article titles and identified 102 different disease management studies. We evaluated multiple potential implementation strategies for many different diseases and conditions, and we evaluated both the process of care (provider adherence to guidelines) and the outcome of care (disease control). Disease control measures were carefully selected and related to the key clinical goals of the treatment of each disease. In addition, our study provided both qualitative and quantitative information to assess the effectiveness of different interventions; most other studies have evaluated only qualitative findings. 4 7 8 10–13 Therefore, in addition to integrated information on the effect sizes of interventions on patient disease control and adherence to guidelines,20 we have provided detailed descriptions of each study (see authors' website http://www.zynx.com/research/disease_management.htm).
Our study has several limitations, most importantly the quality, quantity, and heterogeneity of the original studies. The studies included great variation in interventions used, patient populations, provider populations, and measured processes and outcomes of care. Many provided insufficient detail in the methods section for us to understand the quality of the interventions and the intensity or duration of each intervention. For example, a study might report that provider education was used, but provide insufficient information for readers to understand how the educational process was performed and how to replicate the process.
The clinical significance of effect sizes may be unclear and need to be interpreted with caution and related to the measured clinical effects reported in the trials.20 Few studies directly compared the effectiveness of different interventions, and without direct comparisons of interventions in trials it is difficult to evaluate each intervention's relative effectiveness.
The available published literature shows that most disease management programmes directed at providers and patients are associated with improvements in care. However, little is known about the relative effectiveness and costs associated with different implementation strategies, and few studies have directly compared intervention strategies. Further research is needed to determine the effectiveness and costs of different implementation strategies that could be used in disease management programmes. These studies should adhere to methodological standards and be described in peer reviewed literature in sufficient detail to enable others to understand and reproduce the results in different patient populations, and to understand the relative effectiveness of different disease management interventions for improving the care of patients with chronic diseases.
We thank David Nash, David Shulkin, and Mark Zitter for providing extra references for our review.
Contributors: SRW, the principal investigator and guarantor for the study, initiated and designed the study, contributed to data analysis and interpretation, and oversaw the study. JMH advised on and helped with the study design, contributed to data interpretation, discussed core ideas, and obtained funding for the study. EB helped design data collection protocols, coordinated data collection, discussed core ideas, was data manager, analysed data, and contributed to data interpretation. KK contributed to the study methods, advised on research measures, discussed core ideas, commented on data analysis, and helped with data interpretation. VH guided and conducted the statistical analysis and commented on data analyses and interpretation. AG Jr helped with data collection and management and data analyses and interpretation. JJO conceived the study and design, obtained funding, advised on and helped with study design, discussed core ideas, designed data collection protocols, and helped with data interpretation. SRW, JMH, KK, JJO, and EB jointly wrote, revised, and edited the paper.
Funding This work was partially supported by a research grant from TAP Pharmaceutical Products
Competing interests SRW has been reimbursed for speaking at conferences on the topic of disease management and was a member of the NCQA Disease Management Advisory Committee. At the time of the study Zynx Health was owned by Cedars-Sinai Health System and affiliated with the UCLA School of Medicine; it is now a subsidiary of Cerner Corporation. At no time has it derived revenue from selling or providing services for disease management programmes.
Further details of this review appear on bmj.com and details of the studies reviewed appear on the authors' website