Measuring multimorbidity beyond counting diseases: systematic review of community and population studies and guide to index choiceBMJ 2020; 368 doi: https://doi.org/10.1136/bmj.m160 (Published 18 February 2020) Cite this as: BMJ 2020;368:m160
All rapid responses
Re: Measuring multimorbidity beyond counting diseases: systematic review of community and population studies and guide to index choice
We read with great interest the manuscript entitled “Measuring multimorbidity beyond counting diseases: systematic review of community and population studies and guide to index choice”(1). This is an important and challenging subject for researchers. We also welcome more precise multimorbidity measurement by better incorporating mental health conditions and moving beyond disease count.
We note three major concerns. First, we are concerned about the need for a more comprehensive and deeper appraisal of the literature prior to making inferences about the quality and validity of measures and whether they are recommended for use. In our opinion, the authors failed to accomplish this goal.
Validation and direct comparisons with existing metrics are critically important for the development of a new measure. Several investigators published subsequent articles to their index papers to demonstrate generalizability and validity. Many studies were added to an appendix but incompletely incorporated into the manuscript results and recommendations. Further, many articles were omitted or inadequately summarized. Thus, conclusions in sections such as “Usage, performance, and validation” and “Recommendations for index selection” are based on incomplete or incorrect data.
Second, additional criteria should be factored in “Recommendations for index selection” such as:
• Participant characteristics - age (all vs. older adults), sex (to include sex-specific conditions), sample size (for robust estimates of less prevalent and underreported conditions), nationally-representative sample for generalizability.
• Number and type of conditions assessed - most measures include prevalent conditions while a few encompass a large inventory including less common but highly impactful conditions.
• Time period conditions assessed – to account for secular trends, therapeutic and diagnostic advances spanning 50 years for measures included in this review (1968-2017).
• Measurement of outcomes assessed – validity, specific measure (e.g., health-related quality of life (HRQOL) and physical functioning also have numerous measures whose strengths and limitations can be debated).
• Distribution of multimorbidity captured by the measure – we reported left-censoring of mortality-based comorbidity measures generalized to multimorbidity measurement in young through older community-dwelling adults(2) and Medicare beneficiaries aged 65 years and older with high multimorbidity(3).
These criteria would help researchers understand context, intended utility, and potential harm when measures are extended to outcomes and populations beyond the development cohorts. For example, measures developed to predict inpatient mortality(4) have been generalized to HRQOL(5) and physical functioning despite being poorly suited for this(2, 6).
Without defining and fully incorporating key criteria to assess multimorbidity measures, one risks oversimplifying and overlooking key differences. For example, among recommended measures in Figure 3 the following should be noted:
• The single measure recommended for physical functioning was developed in 146 women >65 years (mean age 75) and included 25 conditions(7).
• The Tooth(8) measure recommended for mortality, HRQOL, healthcare use and daily functioning was developed in women >65 years, included 17 conditions, and was noted “Needs further evaluation in sample including men.”
• For mortality, Desai(9) included 10 conditions from hospitalized patients, mean age 78.7 years, and Charlson(4) was recommended despite subsequent updates(10).
Third, the authors state that direct comparisons between measures are important. However, several measures have been directly compared but were incompletely summarized and not considered in the final recommendations.
Applying the above three concerns to the measure we know best, the multimorbidity-weighted index (MWI)(11):
1. MWI was erroneously not cited as having been “compared with an existing measure,” “validated elsewhere” and “compared with other indices,” or “tested at predicting different or additional outcomes to those in the original index.” MWI has been validated and predicts future physical functioning, cognitive functioning and rates of cognitive decline, basic and instrumental activities of daily living (ADLs, IADLs), all scales of the Short Form (SF)-36 HRQOL, social participation, all-cause and suicide mortality, influenza vaccine uptake, and hospitalized days in different sample populations from the original cohorts including nationally-representative US adults and Medicare beneficiaries (2-3, 12-19). Validity assessment and comparisons with other indices(2-3, 12-16) were omitted despite being published and/or available online prior to the systemic review search in October 2018 and author reply to peer review in November 2019.
2. MWI is among the most comprehensive modern and validated indices developed from 612,592 observations from 216,890 young through older community-dwelling adults, with 81 common and rare but impactful conditions weighted by the validated SF-36 physical functioning with repeated measures of exposures and outcomes 1992-2008(11).
3. Compared with Charlson(4), Elixhauser(20), Mukherjee(21), and disease count, MWI had the best model fit and strongest associations with future physical functioning(2-3) and mortality(2) and spanned the widest distribution of multimorbidity(2-3) but was not recommended for any outcome in this review.
We have also identified apparent errors in the appendix that impacted the manuscript tables, figures, and recommendations. In Table e10, first row (multimorbidity-weighted index):
• “Physical functioning” should be listed under the column “Performance (original outcomes)” while grip strength, gait speed, TICS-m, ADL and IADL limitations should appear under “Additional outcomes tested in external validation”(12).
• Outcomes are missing under “Additional outcomes tested in external validation” including future SF-36 physical functioning, cognitive functioning and rates of cognitive decline (TICS-m, immediate and delayed recall, working memory), HRQOL, all-cause and suicide mortality(2-3, 13-15). If abstracts were included, future disability, social participation, and hospitalized days(17-19) would also be listed.
• The “Predictive accuracy” column was listed as “None” but we reported C-statistics for mortality(2-3), correlations, coefficients of determination for physical functioning, and other measures to assess distribution compared with other measures(2-3, 12, 14-15).
We recommend that details from measures in this review be corrected to accurately reflect analyses performed. Otherwise, present conclusions about which multimorbidity measures to use are based on an incomplete review of the literature and may be detrimental to the future science and application of multimorbidity measures.
Finally, the authors conclude multimorbidity measurement is “at risk of saturation” but we encourage continued efforts to develop and improve upon existing measures. There are more and better data with unique linkages to existing data such as personal wearable devices, advanced methods including machine learning for big datasets, minority and high-risk populations underrepresented in current studies, and complexities of multimorbidity to be elucidated such as disease interactions and patterns of progression, all in the dynamic landscape of modern epidemiology and medicine.
Melissa Y. Wei, MD, MPH, MS
Assistant Professor of Internal Medicine
Division of General Medicine, Department of Medicine, University of Michigan, Ann Arbor, Michigan, USA
Kenneth J. Mukamal, MD, MPH, MA
Associate Professor of Internal Medicine
Division of General Medicine, Beth Israel Deaconess Medical Center and Harvard Medical School, Boston, Massachusetts, USA
1. Stirland LE, González-Saavedra L, Mullin DS, Ritchie CW, Muniz-Terrera G, Russ TC. Measuring multimorbidity beyond counting diseases: systematic review of community and population studies and guide to index choice. BMJ. 2020 Feb 18;368:m160. doi: 10.1136/bmj.m160.
2. Wei MY, Mukamal KJ. Multimorbidity, mortality and long-term physical functioning in three prospective cohorts of community-dwelling adults. Am J Epidemiol. 2018;187(1):103-112. doi: 10.1093/aje/kwx198.
3. Wei MY, Ratz D, Mukamal KJ. Multimorbidity in Medicare beneficiaries: performance of an ICD-coded multimorbidity-weighted index. J Am Geriatr Soc. 2020, https://doi.org/10.1111/jgs.16310. 21
4. Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. Chronic Dis. 1987;40(5):373-83. https://doi-org.proxy.lib.umich.edu/10.1016/0021-9681(87)90171-8.
5. Wahlgren T, Levitt S, Kowalski J, Nilsson S, Brandberg Y. Use of the Charlson combined comorbidity index to predict postradiotherapy quality of life for prostate cancer patients. International journal of radiation oncology, biology, physics. 2011;81(4):997-1004.
6. Greene ME, Rolfson O, Gordon M, Garellick G, Nemes S. Standard Comorbidity Measures Do Not Predict Patient-reported Outcomes 1 Year After Total Hip Arthroplasty. Clinical orthopaedics and related research. 2015;473(11):3370-3379.
7. Bayliss EA, Ellis JL, Steiner JF. Subjective assessments of comorbidity correlate with quality of life health outcomes: initial validation of a comorbidity assessment instrument. Health Qual Life Outcomes. 2005;3:51. https://doi-org.proxy.lib.umich.edu/10.1186/1477-7525-3-51.
8. Tooth L, Hockey R, Byles J, Dobson A. Weighted multimorbidity indexes predicted mortality, health service use, and health-related quality of life in older women. J Clin Epidemiol. 2008;61(2):151-9. https://doi.org/10.1016/j.jclinepi.2007.05.015.
9. Desai MM, Bogardus ST Jr, Williams CS, Vitagliano G, Inouye SK. Development and validation of a risk-adjustment index for older patients: the high-risk diagnoses for the elderly scale. J Am Geriatr Soc. 2002;50(3):474-81. https://doi-org.proxy.lib.umich.edu/10.1046/j.1532-5415.2002.50113.x
10. Quan H, Li B, Couris CM, Fushimi K, Graham P, Hider P, Januel JM, Sundararajan V. Updating and validating the Charlson comorbidity index and score for risk adjustment in hospital discharge abstracts using data from 6 countries. Am J Epidemiol. 2011 Mar 15;173(6):676-82. doi: 10.1093/aje/kwq433.
11. Wei MY, Kawachi I, Okereke OI, Mukamal KJ. Diverse cumulative impact of chronic diseases on physical health-related quality of life: implications for a measure of multimorbidity. Am J Epidemiol. 2016;184(5):357-365. doi: 10.1093/aje/kwv456.
12. Wei MY, Kabeto M, Langa KM, Mukamal KJ. Multimorbidity and physical and cognitive function: performance of a new multimorbidity-weighted index. J Gerontol A Biol Sci Med Sci. 2018;73(2):225-232. doi: 10.1093/gerona/glx114.
13. Wei MY, Kabeto MU, Galecki AT, Langa KM. Physical functioning decline and mortality in older adults with multimorbidity: joint modeling of longitudinal and survival data in the Health and Retirement Study. J Gerontol A Biol Sci Med Sci. 2019;74(2):226-232. doi: 10.1093/gerona/gly038.
14. Wei MY, Mukamal KJ. Multimorbidity and mental health-related quality of life and completed suicide. J Am Geriatr Soc. 2019;67(3):511-519. doi: 10.1111/jgs.15678.
15. Wei MY, Levine DA, Zahodne LB, Kabeto MU, Langa KM. Multimorbidity and cognitive decline over 14 years in older Americans. J Gerontol A Biol Sci Med Sci. 2019, doi: 10.1093/gerona/glz147.
16. Harrison SM, Wei MY, Lamerato LE, Petrie JG, Martin ET. Multimorbidity is associated with uptake of influenza vaccination. Vaccine. 2018;36(25):3635-3640. doi: 10.1016/j.vaccine.2018.05.021.
17. Wei MY, Tilton N, Mukamal KJ. Quantifying the burden of hospitalized days in Medicare beneficiaries with multimorbidity. Innov Aging. 2019;3(Suppl 1):S924. doi:10.1093/geroni/igz038.3365
18. Wei MY, Kabeto M, Langa KM. Multimorbidity and long-term limitations with ADLs and IADLs in older adults. Innovation in Aging. 2017;1(Suppl 1):929. doi:10.1093/geroni/igx004.3330.
19. Luster J, Ratz D, Wei MY. Multimorbidity and social participation among middle-aged and older Americans. J Gen Intern Med. 2019;34(Suppl 2):S99-867. https://doi.org/10.1007/s11606-019-05007-5.
20. Elixhauser A, Steiner C, Harris DR, Coffey RM. Comorbidity measures for use with administrative data. Medical care. 1998;36(1):8-27.
21. Mukherjee B, Ou HT, Wang F, Erickson SR. A new comorbidity index: the health-related quality of life comorbidity index. J Clin Epidemiol. 2011;64(3):309-19. doi: 10.1016/j.jclinepi.2010.01.025.
Competing interests: No competing interests
Re: Measuring multimorbidity beyond counting diseases: systematic review of community and population studies and guide to index choice
We have read with interest the systematic review of Stirland L.E. et al. evaluating different community and population scores of multimorbidity (1). There are three issues we would like to comment to the authors.
Firstly, the taxonomy, terminology and features of multimorbidity are evolving issues, and they evolve in parallel with the epidemiological evolution of this population. In last years of the previous century the focus was centered in progressively severe chronic diseases (heart failure, cirrhosis, respiratory diseases...); but nowadays, with increased life expectancy, those patterns share protagonist role with aging and health-care processes (frailty, sarcopenia, polypharmacy, immobility, functional decline, solelyness...) in the same extent. Hence, we fully agree with authors in the importance of considering other measures beyond 'standard' diseases (mental, functional, social, familiar) in the holistic evaluation of multimorbidity (1). That's why, new concepts in this taxonomy like polypathological patients or patients with complex chronic conditions should be also incorporated (2).
Secondly, we have some doubts about the authors' recommended indices, particularly the Charlson index. We fully agree about its historical value (it was pioneer, and could be considered even legendary); but nowadays we have serious concerns regarding its precision (calibration and discrimination power) on the basis of comparisons performed with other current indices (3). As a matter of fact, according to its dimensions' relative weights, Charlson index will probably systematically overestimate death risk. This consideration is relevant if we are going to establish our aims with the patient, and make decisions about treatments and interventions. We would be in danger of denying opportunities to patients with a really better life prognosis, and could fall into a somehow nihilist practice.
At last, with respect to the PROFUND index, its generalizability has already been established in terms of reproducibility in the original validation cohort (3); as well as in terms of historical, geographic, spectrum, and follow-up transportability (4-8). In all these evaluations, the PROFUND index has maintained its accuracy, establishing an easy-to-use risk stratification model for patients with multimorbidity in many clinical scenarios (Table 1).
In conclusion multimorbidity is an evolving issue, dyed nowadays with different aging processes, geriatric syndromes, and health-care aspects. This paradigm raises the need to open our focus beyond 'standard' diseases. Because of these reasons, we have to reassess risk-stratification indices, and eventually develop new and accurate tools, adapted to the above mentioned population changes.
1. Stirland LE, González-Saavedra L, Mullin DS, Ritchie CW, Muniz-Terrera G, Russ TC. Measuring multimorbidity beyond counting diseases: systematic review of community and population studies and guide to index choice. BMJ 2020;368:m127. http://dx.doi.org/10.1136/bmj.m127.
2. Ollero-Baturone M, Bernabeu-Wittel M, Espinosa-Almendro JM, et al. Integrated care process for polypathological patients. Public Health Direction of Andalusian Government Ed. Sevilla, 2018. https://www.juntadeandalucia.es/export/drupaljda/salud_5af1956d9925c_ate.... Last accessed: February 22th, 2020.
3. Bernabeu-Wittel M, Ollero-Baturone M, Moreno-Gaviño L, et al. Development of a new predictive model for polypathological patients. The PROFUND index. Eur J Intern Med 2011;22:311-7.
4. P. Bohorquez Colombo; M.D. Nieto Martín; B. Pascual de la Pisa; M. J. García Lozano; M. A. Ortiz Camuñez; Bernabeu-Wittel M. Validation of a prognostic model for polypathological patients (PP) in Primary Health Care: PROFUND STUDY-AP. Atención Primaria 2014; 46 (Suppl 3): 41-48.
5. Bernabeu-Wittel M, Moreno-Gaviño L, Ollero-Baturone M, Barón-Franco B, Díez-Manglano J, Rivas-Cobas C, Murcia-Zaragoza J, Ramos-Cantos C, Fernández-Moyano A; PROFUND researchers. Validation of PROFUND prognostic index over a four-year follow-up period. Eur J Intern Med. 2016; ;36:20-24.
6. Díez-Manglano J, Cabrerizo García JL, García-Arilla Calvo E, Jimeno Saínz A, Calvo Beguería E, Martínez-Álvarez RM, Bejarano Tello E, Caudevilla Martínez A. External validation of the PROFUND index in polypathological patients from internal medicine and acute geriatrics departments in Aragón. Intern Emerg Med 2015; 10:915-26.
7. López-Garrido MA, Antequera Martín-Portugués I, Becerra-Muñoz VM, Orellana-Figueroa HN, Sánchez-Lora FJ, Morcillo-Hidalgo L, Jiménez-Navarro MF, Gómez-Doblas JJ, de Teresa-Galván E, García-Pinilla JM. Prevalence of comorbidities and the prognostic value of the PROFUND index in a hospital cardiology unit. Rev Clin Esp 2017; 217:87-94.
8. Martín-Escalante MD, Quirós-López R, Martos-Pérez F, Olalla-Sierra J, Rivas-Ruiz F, Aguilar-García JA, Jiménez-Puente A, García-Alegría J. Validation of the PROFUND index to predict early post-hospital discharge mortality. QJM 2019; 112:854-60.
Table 1. Risk stratification groups of PROFUND index in different populations of polypathological patients.
RISK-GROUP PROFUND IN HOSPITAL SETTING$ PROFUND IN PRIMARY CARE SETTING&
1-YEAR DEATH RISK 4-YEAR DEATH RISK 2-YEAR DEATH RISK
0-2 points 12,1%-14,6%* 52% *11 % - 8,5%
3-6 points 21,5%-31,5%* 73,5% *18% - 21,6%
7-10 points 45%-50%* 85% *26,8% - 29,5%
11 or more points 68% -61%* 92% *41,8% - 43,7%
$ At discharge after hospital stay; & Patients with no hospital stays in previous 3 months; * Derivation-validation cohorts
Competing interests: No competing interests