Jump to: Page Content, Site Navigation, Site Search,
You are seeing this message because your web browser does not support basic web standards. Find out more about why this message is appearing and what you can do to make your experience on this site better.
Published 31 October 2008, doi:10.1136/bmj.a1864
Cite this as: BMJ 2008;337:a1864
Preference Collaborative Review Group
Correspondence to: H Tilbrook, Research Fellow, York Trials Unit, Department of Health Sciences, University of York, York YO10 5DD het2{at}york.ac.uk
Data sources Citation search using Science Citation Index and Google Scholar and search of the main electronic databases (Medline, CINAHL, Embase, and AMED) with a combination of key words.
Study selection Fully randomised patient preference trials that compared treatments for any clinical condition were included. Other types of preference trials and crossover trials were excluded. Other inclusion criteria: participants aged 16 years and over; primary, self-reported outcomes measured on a continuous numerical scale. From 167 studies identified and screened, 17 were identified as fully randomised patient preference trials.
Data synthesis Of the 17 trials identified, 11 authors provided raw data for the meta-analysis. Data collected were baseline and follow-up data for the main outcome, randomised allocation data, preference data, and demographic data. Baseline and first post-intervention follow-up data for the main outcome were standardised. To improve homogeneity, data for only the eight musculoskeletal trials (n=1594) were combined. To estimate the effects of preferences on outcomes and attrition, three groups were compared: patients who had a preference and were randomly allocated to their preferred treatment; patients who had a preference and were randomly allocated to the treatment they did not prefer; and patients who had no preference.
Results Patients who were randomised to their preferred treatment had a standardised effect size greater than that of those who were indifferent to the treatment assignment (effect size 0.162, 95% confidence interval 0.011 to 0.314; P=0.04). Participants who received their preferred treatment also did better than participants who did not receive their preferred treatment (effect size 0.152, –0.035 to 0.339), although this was not statistically significant (P=0.11). Participants allocated to their undesired treatment had outcomes that were no different from those who were indifferent. Participants who were allocated to their undesired treatment were less likely to be lost to first follow-up compared with indifferent participants (odds ratio 1.70, 1.076 to 2.693; P=0.02). No difference was found in attrition between patients allocated to their preference and those who were indifferent.
Conclusions Preferences among patients in musculoskeletal trials are associated with treatment effects. In open randomised trials, preferences should be ascertained before randomisation.
Patients with strong preferences may decline to participate; in a trial in which strong preferences exist and a large number of patients refuse randomisation, the external validity will be adversely affected. When this occurs, generalisability of the results to a wider population will be limited.4 If patients with preferences consent to randomisation, this may also affect its internal validity.
In a randomised controlled trial, patients may have a preference either for the standard treatment or for the new treatment being evaluated or may be indifferent to both treatments. If patients with preferences consent to be randomised then some patients will get their preferred treatment and others will not. Those who receive their preferred treatment might be better motivated and comply better with the treatment programmes and report better outcomes.5 Patients who do not receive their preferred treatment may experience "resentful demoralisation,"6 may be less motivated, may not comply with the treatment programme, may not report accurately during follow-up, and may even drop out of the trial and thereby introduce bias that affects the internal validity of the trial.5 Effects of preference are likely to be more apparent in unblinded trials in which patients are aware of the treatment they are receiving and the outcome measure is subjective and self reported by the patient.
As well as the direct effects of patients preferences on compliance and motivation, a "therapeutic effect" of patients preferences can occur.7 These are the psychological effects that influence outcomes and are similar to the placebo effect.7 The therapeutic effects of patients preferences are problematic for trials because these seek to determine or assess the physiological or pharmacological effects of an intervention. Disentangling the psychological effects of patients preferences from the physiological effects is complex.8
One approach to dealing with patients preferences is the partially randomised preference design also known as a patient preference trial or comprehensive cohort design.2 5 Patients without strong preferences for a treatment are randomised, and those with strong preferences are given their choice, resulting in a four armed trial.2 Such a design enables comparisons between patients with and without a preference and an exploration of patients characteristics associated with preference. An important disadvantage of this design is that the outcome may be affected by uncontrolled confounders in the non-randomised groups,9 which may bias the results.10 Whether the randomised group should be compared with the non-randomised group is therefore debatable.9 6
An alternative approach is a standard randomised controlled trial in which, after participants have given consent in the usual way and before randomisation, patients preferences are recorded: this design would be a fully randomised preference trial.6 These preferences are then taken into account in the analysis of the trial. Such an approach has been used in some trials,w1 and at least one apparently observed a preference-treatment interaction.w2 This interaction, however, was not statistically significant. A potential criticism of this approach is that ignoring patients preferences and proceeding with randomisation is unethical. However, a counter argument is that ignoring preferences may be ethical if all the treatments being offered are believed to be effective and patients give fully informed consent.w3
Detecting patients preference-treatment interactions within trials is difficult, as trials are not sufficiently powered to detect them.10 4 However, Torgerson and Sibbald suggested standardising the way in which information on preferences is obtained in trials so that the trials could be used in meta-analysis to assess the effect of preferences on outcomes.11
King and colleagues recently systematically reviewed the effects of participants preferences in randomised controlled trials.5 12 They investigated the effects of preferences on outcomes and recruitment to trials and the influence of preferences on attrition. Their review identified 34 randomised controlled trials, but only two of these were fully randomised patient preference trials.13 w1 The review, which used evidence from partially randomised preference trials, found no evidence that preferences influenced attrition. It found some evidence of an effect on outcomes in some of the preference trials and no effect on outcomes in the two conventional randomised controlled trials. The trials and studies included in the review were very heterogeneous, and the authors reported that they were therefore unable to "reach definitive conclusions for any particular clinical field."5
Here we report the results of a systematic review and an individual patient data analysis of fully randomised patient preference trials. The aim of the review was to assess whether preferences had an interaction with effectiveness of treatment and whether attrition from the study was different between patients who received their preferred treatment, those who did not, and those who had no preference. This review differs from the previous review by King and colleagues in that we were interested only in fully randomised preference designs and we also sought patient level data for our analysis.
We included only fully randomised preference trials.6 Other inclusion criteria were participants 16 years and over and primary self reported outcomes measured on a continuous scale. We included only trials with self reported outcomes, as trials with outcomes reported by clinicians (or another assessor) are unlikely to be influenced by patients preferences. Additionally, we chose to focus on only one self reported primary outcome and exclude secondary outcome measures to avoid type I error. If trialists had used more than one primary outcome measure, we used the measure that was most common to all the studies, as this would tend to ameliorate the statistical heterogeneity between the studies. We selected outcomes measured on a numerical scale, as these would be more powerful than dichotomous outcomes. We excluded trials in which patients were given their choice of treatment and crossover trials. We checked the generation of the randomisation sequence, concealment of randomisation allocation, and systematic differences in attrition of participants between the comparison groups to ensure the methodological quality of the included studies.
We initially approached the authors of the identified trials by email to ask if they would be willing to release their data for the study. If they expressed an interest, we sent them the protocol for this study with a request for the raw data. Because the studies used different outcome measures, we calculated standardised baseline and outcome scores for all patients by dividing their scores by the baseline standard deviation for the sample population for that trial. In addition, most trials had multiple follow-up points. We therefore decided to use the first follow-up point after treatment had finished, as this would tend to minimise loss of data and be most likely to show an effect of preference if one existed.
Analysis
To analyse if patients preferences influence outcomes, we used multiple regression with dummy variables for categorical predictors. We used SPSS version 14. The dependent variable was the standardised first follow-up score, and the predicting variable was patients preference for treatment. We adjusted the analysis for baseline scores and categorical variables of trial and treatment allocation, which adjusted for treatment effects of individual interventions. In the regression analyses, we treated study as a fixed effect. We checked this assumption with a heterogeneity test and found no evidence of heterogeneity. For calculating the odds ratios for response to first follow-up questionnaires, we used logistic regression analysis. The dependent variable was missing data at first follow-up, and in the analysis we adjusted for age, sex, trial, and severity of condition at baseline. We compared the outcomes of three preference groups: patients with a preference who got what they wanted, those with a preference who did not get what they wanted, and those who had no preference (that is, they were indifferent). We first compared the two preference groups with the indifferent group; then we compared patients with a preference who were randomised to their desired treatment with those who were randomised to their undesired treatment and the indifferent group. We assessed the methodological quality of the trials and found them to be adequate, so we did not need to include this in the analysis.
|
|
|
|
2=6.02, df=8; P=0.64). Within the musculoskeletal trials, we examined whether patients with a preference were statistically significantly different in terms of age, sex, and standardised baseline scores from participants who were indifferent. We found no differences in standardised baseline scores (mean 0.261 for preference group; 0.267 for indifferent group; 95% confidence interval for mean difference –0.068 to 0.177; P=0.38) but a difference in age (mean 48.57 for preference group; 46.99 for indifferent groups; 0.21 to 2.96; P=0.024). Women were more likely to have a preference, although this was not significantly different (46.3% (n=439) for women; 42.3% (n=271) for men; –1% to 9%; P=0.12).
After adjustment for baseline score, trial, and treatment allocation, in terms of treatment effect overall, patients who received their preference had significantly greater improvements than did those who were indifferent (mean effect size 0.162, 95% confidence interval 0.011 to 0.314; P=0.036). In contrast, contrary to expectations, those patients with a preference who did not receive their preferred treatment had only a slight, non-significant difference favouring an improvement relative to the indifferent group (effect size 0.011, –0.142 to 0.164; P=0.89). When we compared patients randomised to their preferred treatment with those randomised to their unpreferred treatment, we found an effect size of 0.152 (–0.035 to 0.339), which was not statistically significant (P=0.11). The total number of participants included in the analysis to compare the effects of preferences on outcomes was 1398 (table 4
).
|
|
Strengths and limitations
By pooling the data from the eight musculoskeletal trials, we obtained a relatively large sample size (n=1398 to investigate the effect of patients preferences on treatment outcomes and n=1583 to investigate the effect of patients preferences on attrition). The design of this study—in which we made comparisons between patients who received their preference, patients who did not receive their preference, and those with no preference—is, as far as we can determine, novel for detecting effects of preference.
In most of the trials included in the analysis, the questionnaires had been completed "alone" by the participants and therefore were unlikely to have been influenced by any treatment preference held by health professionals or researchers involved in the trials. Where questionnaires had been completed in the presence of health professionals or researchers, procedures had been put in place by the trialists to minimise the influence of their treatment preferences on the participants answers to the questionnaires.
Our study does have some limitations. Firstly, we restricted our meta-analysis to musculoskeletal trials, so our results may not be applicable to other areas of health care. Secondly, within each of the individual component trials participants with a very strong preference for usual care will tend not to be recruited into the studies. However, several studies did recruit participants who had a preference for treatments that were available outside of the trial setting. Thirdly, we interpreted missing data at the first follow-up after the intervention as attrition. We did not investigate if data were present at the final follow-up. Fourthly, eight of the 17 studies were identified through the personal knowledge of one of the authors (DT), who was a co-investigator/author on some reports and who had found others through general reading of trial reports. Although four of these studies were unpublished at the time of the search, this suggests that other similar trials exist that our search strategy did not identify. Finally, confounding could be present—for example, actual treatment associated with preferences causing an effect. We acknowledge that an intermediate confounding variable may be responsible for the association, but we cannot find any evidence of a variable in such an analysis.
Comparison with other studies
This is the first study that we are aware of in which a patient level meta-analysis has been done to investigate the effects of patients preferences on treatment outcomes and attrition. Our review differs from the earlier review by King and colleagues,5 in that we sought only trials in which all participants had been randomised irrespective of their preferences. Also our study was restricted to musculoskeletal trials. The search in the review by King and colleagues covered the period 1966 to September 2004 and identified only two fully randomised patient preference trials.13 w1 Most of the trials identified in our study were published after this time, indicating that fully randomised patient preference trials are becoming more widely used. By using these designs, we can ensure that when we compare patients who receive their preferred treatment with those who did not receive their preferred treatment they are similar in terms of potential baseline confounders.
Implications
In terms of treatment effects, we did see an increased treatment effect size among participants who were randomised to their preferred treatment compared with those who were indifferent to the treatment allocation. Preferences seemed to affect outcome despite the fact that patients with and without a preference were balanced in the most important variable at baseline (that is, baseline score). Whether preferences will materially affect the overall outcome of any one trial will depend on the proportion of participants with a preference, the effect of the preference itself, and the effect size of the intervention. We did not look at the origin of preferences, which may involve the person having had that particular treatment before and knowing that the treatment has helped and what the treatment involves. In this review, we have shown that about 57% of the participants have a preference and those who are allocated to their preference show about 0.162 of a standard deviation increase in effect above those who were indifferent. This might have an important effect on the outcomes of a trial. If, for example, a treatment had an effect size of 0.25 among indifferent participants, then our results suggest that for those with a preference the effect would be about 0.41 (that is, a 64% increase in effect size). This increased effect is not necessarily a bias—it might be a genuine effect in the sense that the treatment works better among patients who desire and receive their preferred treatment. It could, however, change the cost effectiveness of a treatment. A treatment may not be cost effective overall but could be among a subgroup of participants who prefer that treatment.14 However, we cannot assume this to be true for people with strong preferences for a treatment and who consequently did not take part in the trials included in this study. The behaviour of this group of people is unknown—for example, they may become easily disappointed if the treatment does not meet their expectations. Consequently, we would conclude that in trials—particularly when a novel therapy is available only within the context of the study and outcomes are subjective—pre-randomised preferences should be identified and recorded so that they can be accounted for in any analysis. Although an effect size of 0.162 seems to be modest, we need to set this in the context of generally modest effect sizes from effective treatments for musculoskeletal problems in general. For example, the UK BEAM trial, which evaluated manipulation with or without exercise for the treatment of low back pain, found an effect size of 0.34 for exercise after three months and an effect size of 0.39 for manipulation.15 Our results provide no evidence that "resentful demoralisation" leads to a reduction in effect size if people are allocated to their undesired treatment.
Stratifying by preferences would help the power of a trial if either the trial was small or the proportion of patients with a preference was small; if, for instance, 10% of participants had a preference and the trial only had 100 participants, all 10 (or at least eight or nine) participants with a preference might end up in one group. Consequently, we would be unable to control for the possible effects of preference in a regression analysis. Therefore, in small trials or trials in which the proportion of patients with a preference is small, trialists might consider using preference as a stratification variable.
Conclusions
This review shows that treatment preferences affect outcomes in a sample of musculoskeletal trials but that they are not detrimental to attrition rates. The fully randomised preference trial seems to be more widely used now, which would enable further work to determine the treatment effects of patients preferences in different clinical conditions.
|
Cite this as: BMJ 2008;337:a1864
Contributors: JMB provided advice on the statistical analysis and conceived the framework of the statistical analysis. HET did the systematic review as part of her MSc dissertation, did the analysis, and wrote the first draft of the paper. DJT had the idea for the review, supervised HET, identified some papers for inclusion in the review, and helped to write the first draft of the paper. SJA, EMH, REJ, GTJ, HK, JAKM, GJM, HMacP, SMcL, LN, CS, and ET contributed data to the study. SJA, JMB, EMH, GTJ, JAKM, GJM, HMacP, LN, CS, ET, and DJT critically reviewed versions of the manuscript. HET is the guarantor.
Competing interests: None declared.
Provenance and peer review: Not commissioned; externally peer reviewed.
![]()
CiteULike
Complore
Connotea
Del.icio.us
Digg
Reddit
StumbleUpon
Technorati What's this?
Read all Rapid Responses