Discrepancies between patients' assessments of outcome: qualitative study nested within a randomised controlled trialBMJ 2003; 326 doi: http://dx.doi.org/10.1136/bmj.326.7383.252 (Published 01 February 2003) Cite this as: BMJ 2003;326:252
- Rona Campbell (), lecturer in health services researcha,
- Brian Quilty, consultant in rheumatologyb,
- Paul Dieppe, directorc
- a Department of Social Medicine, University of Bristol, Bristol BS8 2PR
- b Department of Medicine, University of Bristol
- c MRC Health Services Research Collaboration, University of Bristol
- Correspondence to: R Campbell
- Accepted 7 August 2002
Assessments of therapeutic effectiveness should not rely exclusively on clinical data, but they should include patient based outcome measures. A plethora of generic and disease specific measures is now available to collect such data by questionnaire, and well developed methods for testing the precision of such measures exist. 1 2 Another method of collecting patient based outcome data is by in-depth interview. A randomised controlled trial to test the effectiveness of a package of physiotherapy treatment (nine treatment sessions involving patellar taping, seven different exercises, correction of posture, and advice on footwear) for patello-femoral osteoarthritis, which included a nested qualitative study of 20 participants randomised to the intervention arm, provided an opportunity to compare the two approaches to collecting outcome data: quantitatively by questionnaire and qualitatively by means of in-depth interview. 3 4
Participants, methods, and results
The primary outcome measure was pain in the worse knee, recorded on a 10 cm visual analogue scale in the presence of BQ. We used the function subscale of the Western Ontario and McMaster Universities' osteoarthritis index (WOMAC), a validated, disease specific, patient based measure, as a secondary outcome measure.5 An experienced interviewer undertook the in-depth interviews after the treatment but before the main follow up visit of the trial. Interviews were conducted in patients' homes, guided by a checklist of topics that ensured similar issues were explored. Interviews were taped, fully transcribed, and analysed independently by RC and PD, who were blind to the scores on the scales. For both questionnaire and interview data we recorded whether the patient reported worsening, improvement, or no change in pain and restriction of activities.
The level of concordance between the questionnaire and interview data was less than 50% (table). Questionnaire data indicated that three patients had increased pain and seven an increased restriction of activities. In contrast, data from the interviews showed that only one patient had increased pain and one increased disability. A similar trend was apparent in the number of patients reporting their symptoms as unchanged. This disparity was particularly marked in pain scores, where questionnaire data showed that pain was unchanged in three patients, whereas interview data indicated this was so for 10 patients. Some patients who showed worsening on pain and scores of the osteoarthritis index considered themselves better, and vice versa.
The way in which responses are elicited and the context in which data are recorded can affect reported outcomes in osteoarthritis, potentially leading to erroneous decisions about what interventions benefit patients. Randomised controlled trials need reliable and valid patient based outcome measures to determine whether an intervention is effective.
The lack of agreement in this study between standard patient based measurements and patients' narrative accounts is disquieting. The discrepancies are unlikely to be explained by differences in the timing of the quantitative assessments and interviews, and the investigators agreed completely in the interpretation of the interview data. Furthermore, topics included in the interview guide corresponded to those in the questionnaire of the osteoarthritis index. The most likely explanation comes from the context in which data were collected: the quantitative information was obtained in the trial clinic in the presence of a doctor, whereas the qualitative accounts were obtained by an independent interviewer (who was not a healthcare professional) in the patient's home.
We thank all the patients who participated in the research; Marion Tucker, who provided the physiotherapy intervention; and Jenny Donovan, who commented helpfully on a draft of this paper.
Contributors: RC initiated the study. PD and BQ devised the trial from which the patients were recruited, and Maggie Evans conducted the qualitative interviews. BQ undertook the analysis of the data from the quantitative trial and RC and PD the analysis of the data from the in-depth interviews. RC wrote the first draft, with all authors contributing to the writing of subsequent drafts. RC and PD will act as guarantors.
Funding RS is funded by South and West regional Research Directorate. Both the trial and the qualitative study were funded by grants from the South and West Regional Research Directorate. Paul Dieppe is director of the MRC Health Services Research Collaboration, which is based in the Department of Social Medicine at the University of Bristol.
Competing interests None declared.
Examples of interviews with patients appear on bmj.com