Doctors and patients don't agree: cross sectional study of patients' and doctors' perceptions and assessments of disability in multiple sclerosisBMJ 1997; 314 doi: https://doi.org/10.1136/bmj.314.7094.1580 (Published 31 May 1997) Cite this as: BMJ 1997;314:1580
- P M Rothwell, research fellowa,
- Z McDowell, medical studenta,
- C K Wong, visiting fellowa,
- P J Dorman, MRC training fellowa
- Correspondence to: Dr P M Rothwell Department of Clinical Neurology, Radcliffe Infirmary, Oxford OX2 6HE.
- Accepted 11 March 1997
Objectives: To compare the judgments of clinicians on which domains of health in the short form questionnaire (SF-36) would be most important to patients with multiple sclerosis with the opinions of patients themselves; to compare assessment of physical disability in multiple sclerosis by a clinician using Kurtzke's expanded disability status scale and a non-clinically qualified assistant using the Office of Population Census and Surveys' (OPCS) disability scale with self assessment of disability and other domains of health related quality of life by patients using the SF-36 and the EuroQol questionnaire; and to compare the scores of patients for each domain of the SF-36 with control data matched for age and sex.
Design: Cross sectional study.
Setting: Clinical department of neurology, Edinburgh.
Subjects: 42 consecutive patients with multiple sclerosis attending a neurology outpatient clinic for review or a neurology ward for rehabilitation.
Main outcome measures: Scores on the SF-36; EuroQol; Kurtzke's expanded disability status scale; the OPCS disability scale.
Results: Patients and clinicians disagreed on which domains of health status were most important (χ2=21, df=7, P=0.003). Patients' assessment of their physical disability using the physical functioning domain of the SF-36 was highly correlated with the clinicians' assessment (r=-0.87, P<0.001) and the non-clinical assessment (r=-0.90, P<0.001). However, none of the measures of physical disability correlated with overall health related quality of life measured with EuroQol. Quality of life correlated with vitality, general health, and mental health in the SF-36, each of which patients rated as more important than clinicians and for each of which patients scored lower than the controls.
Conclusions: Patients with multiple sclerosis, and possibly those with other chronic diseases, are less concerned than their clinicians about physical disability in their illness. Clinical trials in multiple sclerosis should assess the effect of treatment on the other elements of health status that patients consider important, which are also affected by the disease process, are more closely related to overall health related quality of life, and may well be adversely affected by side effects of treatment.
Patients can accurately assess their own physical disability
Physical disability may not always be the main determinant of overall health related quality of life
Patients and clinicians differ in their assessments of the relative importance of different elements of health related quality of life
The opinions of patients should be taken into account in the selection of outcome measures for clinical trials
Optimal assessment of the efficacy of a clinical intervention depends on the natural course of the disease under study. Sometimes it may be reasonable simply to measure the effect of treatment on case fatality or the risk of major sequelae, such as stroke or myocardial infarction. Many conditions, however, are neither fatal nor characterised by acute sequelae. In these circumstances assessment of outcome usually entails either direct measurement of the extent and severity of the disease or assessment by a physician of the physical impairment or disability caused by the disease. However, both these approaches tend to be expensive, time consuming, and prone to bias if blinding of assessors to treatment allocation is imperfect,1 and neither takes account of the potential adverse effects of treatment on other aspects of health. Self assessment by patients avoids bias by an external assessor, allows measurement of the effect of treatment on overall health related quality of life, and may not necessarily be less informative than clinical assessment with regard to physical disability.
Recent trials of interferon beta have highlighted the uncertainty about how best to measure outcome in clinical trials in multiple sclerosis.2 3 4 5 Treatment reduces the frequency of new lesions on magnetic resonance brain scans and perhaps the number of relapses in patients,2 3 and it may reduce neurological impairment and the rate of progression of disability as measured by neurologists.4 5 However, the effect of treatment on other domains of health, which may be as or more important to patients, has not been measured. Indeed, little information exists on which aspects of health are considered important by patients with multiple sclerosis or other disabling conditions and, consequently, whether it is necessary to measure the effect of treatment on more than just the physical manifestations of the disease. Moreover, it is unclear whether assessment of physical disability requires a neurologist or whether self assessment by patients is adequate.
We therefore compared the perceptions of patients with multiple sclerosis and clinicians as to the relative importance of the eight different domains of health related quality of life measured by the short form 36 (SF-36),6 7 and for each domain we compared the observed quality of life of patients with that expected for general population controls matched for age, sex, and locality. We further assessed the correlation between patients' self assessment of physical disability using the physical function domain of the SF-36 and physical disability measured by a neurologist and a non-medically qualified assistant (using different scales), and we examined the correlation between each measure of physical disability and the overall health related quality of life estimated by patients.
Over eight weeks we studied all patients with multiple sclerosis, either admitted to the neurology ward for rehabilitation or attending the neurology outpatient clinics at the Western General Hospital, Edinburgh. Patients were eligible if they gave informed consent and fulfilled the following criteria: had a clinically definite or laboratory supported clinically probable diagnosis of multiple sclerosis according to Poser et al's criteria8; had had no acute neurological relapse during the previous month; were resident in Lothian region; and knew that they had multiple sclerosis. A neurologist measured neurological impairment by using Kurtzke's expanded disability status scale.9 10 An independent non-clinically qualified assistant administered the disability questionnaire of the Office of Population Censuses and Surveys (OPCS),11 and patients completed the SF-366 7 and the EuroQol health related quality of life questionnaire.12 The order of clinical assessment and questionnaire administration was not specified, and both investigators were blind to each other's findings.
After completion of the questionnaires, patients were given a standard written description of each of the eight elements of health related quality of life assessed in the SF-36 and were asked which three elements were the most important determinants of their overall quality of life. Using the same descriptions, clinicians working in the department of clinical neurosciences (senior and junior neurologists or neurosurgeons) were asked which three domains of the SF-36 they thought were the most important determinants of health related quality of life for patients with multiple sclerosis.
Age and sex stratified tabular control data for each of the domains of the SF-36 were obtained from a survey of health related quality of life in over 6000 people resident in Lothian in 1993.13 The data were in the form of mean scores for men and women in five year age bands. The expected score for each case was taken as the corresponding age and sex matched mean control value.
Kurtzke's scale was treated as an ordinal scale, and scores on this scale were correlated with other scores by using the Spearman rank method with a two tailed test of significance. The OPCS scale and the SF-36 were treated as interval scales. All analyses were performed with the statistical software package SPSS for Windows (version 3.0).
Of the 47 eligible patients who attended the department during the study period, 42 (89%) agreed to participate. Median age was 41 (range 28-68) years; 28 patients were women, and 33 were outpatients. The median score on Kurtzke's scale was 5.5 (range 1-8). The frequency distribution of domains of the SF-36 reported to be important by patients and clinicians (fig 1) differed from those expected by chance alone (patients: χ2=16.2, df=7, P=0.02; clinicians: χ2=44.7, df=7, P<0.0001) and from each other (χ2= 21.4, df=7, P=0.003). The clinicians were significantly more likely than the patients to rate physical functioning and physical role limitations as important and significantly less likely to be concerned with mental health and emotional role limitations. The mean score among the patients was less than that expected on the basis of the age and sex matched control data from the general population for physical functioning, physical role limitations, general health, mental health, and vitality in the SF-36 (table 1).
Physical disability measured by the non-medically qualified assistant using the OPCS disability scale was correlated with the score on Kurtzke's scale obtained by the neurologist (r=0.84, P<0.0001). Physical disability measured by the patients using the physical functioning domain of the SF-36 was highly correlated with both the score on Kurtzke's scale and the overall OPCS disability score (fig 2). However, none of the scores of the other domains of the SF-36 were correlated with the score on Kurtzke's scale, and only the score for vitality was correlated with the OPCS disability score (table 2). Neither the score on Kurtzke's scale nor the OPCS score was correlated with overall health related quality of life estimated by the patients using the visual analogue scale in the EuroQol questionnaire (fig 3). Overall health related quality of life was, however, correlated significantly with vitality, general health, mental health, and physical role limitation in the SF-36 (table 2).
Concerns of patients
Doctors are not good at estimating the overall quality of life of their patients.14 15 O'Boyle and colleagues argued that a valid measure of quality of life should quantify the level of functioning of each individual in those areas of life that he or she believes to be most important.16 We have shown, at least in multiple sclerosis, that doctors' assessments of the relative importance of the domains of health related quality of life differ from those of patients. The pattern of domains reported to be important by the patients and that reported by the clinicians both differed significantly from the distribution expected by chance alone. In other words, there was a degree of agreement within the patient group and within the medical group as to which domains were most important. However, there was significant disagreement between the two groups. Generally the clinicians were more concerned than the patients about the physical manifestations of disease; the patients were more concerned with less tangible quantities such as mental health and vitality.
Although we did not study the reliability of these assessments, we have produced some evidence of their importance. Firstly, the patients with multiple sclerosis had significantly lower scores than the general population controls for three of the four domains of the SF-36 (mental health, vitality, and general health) that they rated higher than the clinicians did. Secondly, these domains were highly correlated with their estimates of overall health related quality of life. It does not necessarily follow, however, that the patient's viewpoint is more important than that of the doctor. Doctors will usually have a better understanding of the natural course and possible clinical manifestations of a particular disease, and their opinions may be based on experience of treating many patients. Nevertheless, doctors should at least bear in mind that their concerns may not coincide with those of their patients when considering whether to prescribe treatment, and, as has been suggested recently, trialists should consider taking patients' views into account when designing trials.17 18
Assessment of physical disability
Irrespective of which domains of health related quality of life are important to patients with multiple sclerosis, physical disability should clearly still be measured in trials of treatments that potentially modify disease. Although Kurtzke's scale is the most commonly used measure of disability in multiple sclerosis, it is expensive and time consuming to perform as it requires a full clinical examination by a neurologist. Questionnaire based disability scales, such as the OPCS scale, have the advantage that they can be administered by non-clinical assistants, although this will still be relatively costly in a large multicentre trial. Moreover, both options require blinding of assessors to treatment allocation. Non-blinded assessment can significantly bias the results of multiple sclerosis trials in favour of treatment,1 and blinding is difficult to maintain in practice without going to considerable lengths. Self assessment by patients has the advantage of eliminating the potential for bias by an external assessor, although it does not avoid bias resulting from placebo effects experienced by patients. Postal or telephone follow up with the SF-36, or a similar instrument, would be easily standardised and relatively inexpensive.
Assessment of physical disability by a non-clinically qualified assistant using the OPCS disability scale was correlated with that obtained by a neurologist using Kurtzke's scale. The functional limitations profile has also been shown to be highly correlated with Kurtzke's scale.19 We have shown that self assessment of disability by patients, using the physical functioning domain of the SF-36, can provide closely similar information to that obtained with Kurtzke's scale or the OPCS disability scale. Patients could provide their self assessment by post or telephone, and, at least on a cross sectional basis, this would provide about 80% of the information obtained with Kurtzke's scale or the OPCS scale (r2=0.76 andr2=0.81 respectively). However, none of the measurements of physical disability was correlated with the patients' assessments of their overall health related quality of life based on the EuroQol questionnaire. Overall health related quality of life was correlated most significantly with vitality, general health, and mental health in the SF-36. Given that these domains were all thought to be important by patients and that the scores obtained from patients for each of the domains were lower than those expected on the basis of control data–that is, the scores had been affected by the disease process–the assessment of overall health related quality of life seems to have some validity. Other studies have shown that measurement of health related quality of life in multiple sclerosis can be reproducible.20
Side effects of treatment
The finding that patients rate variables such as vitality, general health, and mental health as important determinants of their overall health related quality of life is important when considering how best to incorporate the side effects of treatments into the results of clinical trials. Side effects are rarely incorporated into the overall trial result and are usually simply listed separately. Whether the benefits of treatment justify the side effects is therefore left for doctors to decide for themselves. However, as the concerns of doctors and patients may not coincide this may be inappropriate. Treatments given to patients with multiple sclerosis frequently cause side effects that are sufficiently severe to influence patients' quality of life.21 For example, interferon beta may cause reactions at the injection site, flu-like symptoms, nausea, myalgia, fever, depression, and malaise, 2 3 4 5 22 each of which is likely to have an adverse effect on the very domains of health related quality of life that patients consider more important than doctors. Measuring outcome with an overall measure of health related quality of life would at least record the patients' perspective on whether the treatment was worse than the disease itself.
We thank Dr John Forbes for access to the SF-36 population control data from the Lothian health survey and Professor Charles Warlow and Mr Jim Slattery for help and advice.
Funding: PJD was supported by a UK Medical Research Council (MRC) training fellowship; PMR was also funded by the MRC.
Conflict of interest: None.