- Preference Collaborative Review Group
- Correspondence to: H Tilbrook, Research Fellow, York Trials Unit, Department of Health Sciences, University of York, York YO10 5DD
- Accepted 27 August 2008
Objective To systematically review fully randomised patient preference trials and to explore the impact of preferences on attrition and outcome by meta-analysis of patient level data.
Data sources Citation search using Science Citation Index and Google Scholar and search of the main electronic databases (Medline, CINAHL, Embase, and AMED) with a combination of key words.
Study selection Fully randomised patient preference trials that compared treatments for any clinical condition were included. Other types of preference trials and crossover trials were excluded. Other inclusion criteria: participants aged 16 years and over; primary, self-reported outcomes measured on a continuous numerical scale. From 167 studies identified and screened, 17 were identified as fully randomised patient preference trials.
Data synthesis Of the 17 trials identified, 11 authors provided raw data for the meta-analysis. Data collected were baseline and follow-up data for the main outcome, randomised allocation data, preference data, and demographic data. Baseline and first post-intervention follow-up data for the main outcome were standardised. To improve homogeneity, data for only the eight musculoskeletal trials (n=1594) were combined. To estimate the effects of preferences on outcomes and attrition, three groups were compared: patients who had a preference and were randomly allocated to their preferred treatment; patients who had a preference and were randomly allocated to the treatment they did not prefer; and patients who had no preference.
Results Patients who were randomised to their preferred treatment had a standardised effect size greater than that of those who were indifferent to the treatment assignment (effect size 0.162, 95% confidence interval 0.011 to 0.314; P=0.04). Participants who received their preferred treatment also did better than participants who did not receive their preferred treatment (effect size 0.152, −0.035 to 0.339), although this was not statistically significant (P=0.11). Participants allocated to their undesired treatment had outcomes that were no different from those who were indifferent. Participants who were allocated to their undesired treatment were less likely to be lost to first follow-up compared with indifferent participants (odds ratio 1.70, 1.076 to 2.693; P=0.02). No difference was found in attrition between patients allocated to their preference and those who were indifferent.
Conclusions Preferences among patients in musculoskeletal trials are associated with treatment effects. In open randomised trials, preferences should be ascertained before randomisation.
The randomised controlled trial is acknowledged as the most scientifically rigorous study design for evaluating medical interventions.1 Although random allocation is intended to evenly distribute characteristics of participants that may affect outcome and remove selection bias, it may not deal with other potential biases. One of these is patients’ preferences. Random allocation will distribute patients with a preference for a given intervention between the intervention groups in similar proportions, as it does with physical characteristics such as weight or height. What it cannot do, however, is deal with the post-randomisation effects of these preferences on treatment outcomes. Because patients’ preferences are not dealt with in the randomisation process, they are viewed as a potential threat to the validity of trials.2 The effect of patients’ preferences on treatment outcomes in randomised controlled trials is, however, uncertain.3
Patients with strong preferences may decline to participate; in a trial in which strong preferences exist and a large number of patients refuse randomisation, the external validity will be adversely affected. When this occurs, generalisability of the results to a wider population will be limited.4 If patients with preferences consent to randomisation, this may also affect its internal validity.
In a randomised controlled trial, patients may have a preference either for the standard treatment or for the new treatment being evaluated or may be indifferent to both treatments. If patients with preferences consent to be randomised then some patients will get their preferred treatment and others will not. Those who receive their preferred treatment might be better motivated and comply better with the treatment programmes and report better outcomes.5 Patients who do not receive their preferred treatment may experience “resentful demoralisation,”6 may be less motivated, may not comply with the treatment programme, may not report accurately during follow-up, and may even drop out of the trial and thereby introduce bias that affects the internal validity of the trial.5 Effects of preference are likely to be more apparent in unblinded trials in which patients are aware of the treatment they are receiving and the outcome measure is subjective and self reported by the patient.
As well as the direct effects of patients’ preferences on compliance and motivation, a “therapeutic effect” of patients’ preferences can occur.7 These are the psychological effects that influence outcomes and are similar to the placebo effect.7 The therapeutic effects of patients’ preferences are problematic for trials because these seek to determine or assess the physiological or pharmacological effects of an intervention. Disentangling the psychological effects of patients’ preferences from the physiological effects is complex.8
One approach to dealing with patients’ preferences is the partially randomised preference design also known as a patient preference trial or comprehensive cohort design.2 5 Patients without strong preferences for a treatment are randomised, and those with strong preferences are given their choice, resulting in a four armed trial.2 Such a design enables comparisons between patients with and without a preference and an exploration of patients’ characteristics associated with preference. An important disadvantage of this design is that the outcome may be affected by uncontrolled confounders in the non-randomised groups,9 which may bias the results.10 Whether the randomised group should be compared with the non-randomised group is therefore debatable.9 6
An alternative approach is a standard randomised controlled trial in which, after participants have given consent in the usual way and before randomisation, patients’ preferences are recorded: this design would be a fully randomised preference trial.6 These preferences are then taken into account in the analysis of the trial. Such an approach has been used in some trials,w1 and at least one apparently observed a preference-treatment interaction.w2 This interaction, however, was not statistically significant. A potential criticism of this approach is that ignoring patients’ preferences and proceeding with randomisation is unethical. However, a counter argument is that ignoring preferences may be ethical if all the treatments being offered are believed to be effective and patients give fully informed consent.w3
Detecting patients’ preference-treatment interactions within trials is difficult, as trials are not sufficiently powered to detect them.10 4 However, Torgerson and Sibbald suggested standardising the way in which information on preferences is obtained in trials so that the trials could be used in meta-analysis to assess the effect of preferences on outcomes.11
King and colleagues recently systematically reviewed the effects of participants’ preferences in randomised controlled trials.5 12 They investigated the effects of preferences on outcomes and recruitment to trials and the influence of preferences on attrition. Their review identified 34 randomised controlled trials, but only two of these were fully randomised patient preference trials.13 w1 The review, which used evidence from partially randomised preference trials, found no evidence that preferences influenced attrition. It found some evidence of an effect on outcomes in some of the preference trials and no effect on outcomes in the two conventional randomised controlled trials. The trials and studies included in the review were very heterogeneous, and the authors reported that they were therefore unable to “reach definitive conclusions for any particular clinical field.”5
Here we report the results of a systematic review and an individual patient data analysis of fully randomised patient preference trials. The aim of the review was to assess whether preferences had an interaction with effectiveness of treatment and whether attrition from the study was different between patients who received their preferred treatment, those who did not, and those who had no preference. This review differs from the previous review by King and colleagues in that we were interested only in fully randomised preference designs and we also sought patient level data for our analysis.
We wanted to identify all trials that have used the fully randomised preference trial design. The extensive systematic review by King and colleagues (2005) searched for papers published between 1966 and September 2004 and identified only two that were fully randomised patient preference trials.5 To find additional papers we did a citation search using Torgerson et al (1996) and Torgerson and Sibbald (1998),6 11 which were the first papers to describe the fully randomised patient preference trial. We used Science Citation Index and Google Scholar to do the citation search. In Science Citation Index, we selected the two papers on which the citation search was to be based. The database then identified all articles that had cited the papers. When we used Advanced Scholar Search in Google Scholar, we selected the subject areas “Biology, life sciences, and environmental science” and “Medicine, pharmacology, and veterinary science,” and the date range 2000 to October 2006. We did our first search on the terms “random, trial, preference, Torgerson, Sibbald” and a second search on the terms “random, trial, preference, Torgerson, Klaber”. We also made searches in Medline, CINAHL, Embase 1996 to 2006, and AMED on the search terms “patient preference$ trial$”. We identified relevant studies initially by title and abstract and finally by full text. In addition, we included trials that were known to us through personal knowledge.
We included only fully randomised preference trials.6 Other inclusion criteria were participants 16 years and over and primary self reported outcomes measured on a continuous scale. We included only trials with self reported outcomes, as trials with outcomes reported by clinicians (or another assessor) are unlikely to be influenced by patients’ preferences. Additionally, we chose to focus on only one self reported primary outcome and exclude secondary outcome measures to avoid type I error. If trialists had used more than one primary outcome measure, we used the measure that was most common to all the studies, as this would tend to ameliorate the statistical heterogeneity between the studies. We selected outcomes measured on a numerical scale, as these would be more powerful than dichotomous outcomes. We excluded trials in which patients were given their choice of treatment and crossover trials. We checked the generation of the randomisation sequence, concealment of randomisation allocation, and systematic differences in attrition of participants between the comparison groups to ensure the methodological quality of the included studies.
We initially approached the authors of the identified trials by email to ask if they would be willing to release their data for the study. If they expressed an interest, we sent them the protocol for this study with a request for the raw data. Because the studies used different outcome measures, we calculated standardised baseline and outcome scores for all patients by dividing their scores by the baseline standard deviation for the sample population for that trial. In addition, most trials had multiple follow-up points. We therefore decided to use the first follow-up point after treatment had finished, as this would tend to minimise loss of data and be most likely to show an effect of preference if one existed.
To analyse if patients’ preferences influence outcomes, we used multiple regression with dummy variables for categorical predictors. We used SPSS version 14. The dependent variable was the standardised first follow-up score, and the predicting variable was patient’s preference for treatment. We adjusted the analysis for baseline scores and categorical variables of trial and treatment allocation, which adjusted for treatment effects of individual interventions. In the regression analyses, we treated study as a fixed effect. We checked this assumption with a heterogeneity test and found no evidence of heterogeneity. For calculating the odds ratios for response to first follow-up questionnaires, we used logistic regression analysis. The dependent variable was missing data at first follow-up, and in the analysis we adjusted for age, sex, trial, and severity of condition at baseline. We compared the outcomes of three preference groups: patients with a preference who got what they wanted, those with a preference who did not get what they wanted, and those who had no preference (that is, they were indifferent). We first compared the two preference groups with the indifferent group; then we compared patients with a preference who were randomised to their desired treatment with those who were randomised to their undesired treatment and the indifferent group. We assessed the methodological quality of the trials and found them to be adequate, so we did not need to include this in the analysis.
Our search identified 167 possibly relevant studies (figure⇓). After screening the abstracts and full papers, the searches identified 17 trials that met the inclusion criteria of this study (table 1⇓), and we were able to obtain the patient level data for 11 studies (table 2⇓).w1-w12 All the trialists had used simple preferences, whereby the participants indicated which treatment they preferred or if they were indifferent to the treatments. In eight of the 11 studies, the questionnaires had been completed “alone” (that is, not in the presence of a healthcare professional or researcher) by the participants. A researcher was present in the trial by Johnson and colleagues (2007) but was advised not to help the participants to complete the questionnaires.w7 The information the participants had available to them on the treatments was provided in information sheets. In the trial by Hay and colleagues (2003) and Thomas and colleagues (2004),w5 w6 patients were asked to complete the baseline questionnaires before the visit of a research nurse who was blind to treatment allocation, and patients did not complete follow-up questionnaires in the presence of any treating healthcare professionals. In the trial by Adamson and colleagues (2005),w3 patients were asked what their treatment preference was at the feedback session immediately before allocation of treatment; a standard description was used for the two treatments, and the trialists advised that they were scrupulous in establishing a sense of equipoise in the minds of all personnel involved in the trial.
The proportion of patients who had a preference in the 11 trials ranged from 16% for a trial of a solution finding approach to back pain to 85% for a trial comparing general practitioner’s care with hospital care for non-urgent skin problems (table 3⇓).w9 w11 The median preference rate was 56% (interquartile range 43-63%). The data show, therefore, that for this sample of fully randomised preference trials, most trial participants were not indifferent to the treatment to which they were going to be allocated.
Of the 11 studies with patient level data, eight (n=1594) were interventions for musculoskeletal conditions. Because the data we obtained were dominated by musculoskeletal studies whose reported outcomes were very much in accordance (that is, they reported outcomes on pain and disability), we decided to restrict our pooled analysis to these studies. Consequently, clinical heterogeneity would be reduced. We found no evidence of statistical heterogeneity between studies (χ2=6.02, df=8; P=0.64).
Within the musculoskeletal trials, we examined whether patients with a preference were statistically significantly different in terms of age, sex, and standardised baseline scores from participants who were indifferent. We found no differences in standardised baseline scores (mean 0.261 for preference group; 0.267 for indifferent group; 95% confidence interval for mean difference −0.068 to 0.177; P=0.38) but a difference in age (mean 48.57 for preference group; 46.99 for indifferent groups; 0.21 to 2.96; P=0.024). Women were more likely to have a preference, although this was not significantly different (46.3% (n=439) for women; 42.3% (n=271) for men; −1% to 9%; P=0.12).
After adjustment for baseline score, trial, and treatment allocation, in terms of treatment effect overall, patients who received their preference had significantly greater improvements than did those who were indifferent (mean effect size 0.162, 95% confidence interval 0.011 to 0.314; P=0.036). In contrast, contrary to expectations, those patients with a preference who did not receive their preferred treatment had only a slight, non-significant difference favouring an improvement relative to the indifferent group (effect size 0.011, −0.142 to 0.164; P=0.89). When we compared patients randomised to their preferred treatment with those randomised to their unpreferred treatment, we found an effect size of 0.152 (−0.035 to 0.339), which was not statistically significant (P=0.11). The total number of participants included in the analysis to compare the effects of preferences on outcomes was 1398 (table 4⇓).
After adjustment for severity of condition at baseline (that is, baseline scores), age, and sex, those who did not receive their preference were more likely to return their questionnaire at first follow-up than were those who were indifferent (odds ratio 1.70, 95% confidence interval 1.08 to 2.69; P=0.02); those who did receive their preference also had an increased response rate, but this difference was not significantly different from the indifferent group (odds ratio 1.26, 0.82 to 1.94; P=0.29). The comparison of those who did not get their preference with those who did yielded an odds ratio of 1.35, but this was not significant (95% confidence interval 0.78 to 2.33; P=0.29). The total number of participants included in the analysis to compare the effects of preferences on attrition was 1583 (table 5⇓).
In this review, we identified 17 fully randomised preference trials and obtained patient level data for 11 of these. These trials covered a range of conditions but were mainly in musculoskeletal medicine where patients are unblinded to their treatment and self report quality of life outcomes. In these types of trials, preferences are most likely to have an effect on outcomes and attrition. In contrast with an earlier review,5 we have found some evidence to suggest that preferences can modify treatment outcomes, although these effects were not all in the expected direction. In terms of increased study attrition, which is widely hypothesised to be affected by preference, we found the converse of what was expected: participants with a preference who are randomised to the opposite treatment are actually more likely to return follow-up data than those who were indifferent.
Strengths and limitations
By pooling the data from the eight musculoskeletal trials, we obtained a relatively large sample size (n=1398 to investigate the effect of patients’ preferences on treatment outcomes and n=1583 to investigate the effect of patients’ preferences on attrition). The design of this study—in which we made comparisons between patients who received their preference, patients who did not receive their preference, and those with no preference—is, as far as we can determine, novel for detecting effects of preference.
In most of the trials included in the analysis, the questionnaires had been completed “alone” by the participants and therefore were unlikely to have been influenced by any treatment preference held by health professionals or researchers involved in the trials. Where questionnaires had been completed in the presence of health professionals or researchers, procedures had been put in place by the trialists to minimise the influence of their treatment preferences on the participants’ answers to the questionnaires.
Our study does have some limitations. Firstly, we restricted our meta-analysis to musculoskeletal trials, so our results may not be applicable to other areas of health care. Secondly, within each of the individual component trials participants with a very strong preference for usual care will tend not to be recruited into the studies. However, several studies did recruit participants who had a preference for treatments that were available outside of the trial setting. Thirdly, we interpreted missing data at the first follow-up after the intervention as attrition. We did not investigate if data were present at the final follow-up. Fourthly, eight of the 17 studies were identified through the personal knowledge of one of the authors (DT), who was a co-investigator/author on some reports and who had found others through general reading of trial reports. Although four of these studies were unpublished at the time of the search, this suggests that other similar trials exist that our search strategy did not identify. Finally, confounding could be present—for example, actual treatment associated with preferences causing an effect. We acknowledge that an intermediate confounding variable may be responsible for the association, but we cannot find any evidence of a variable in such an analysis.
Comparison with other studies
This is the first study that we are aware of in which a patient level meta-analysis has been done to investigate the effects of patients’ preferences on treatment outcomes and attrition. Our review differs from the earlier review by King and colleagues,5 in that we sought only trials in which all participants had been randomised irrespective of their preferences. Also our study was restricted to musculoskeletal trials. The search in the review by King and colleagues covered the period 1966 to September 2004 and identified only two fully randomised patient preference trials.13 w1 Most of the trials identified in our study were published after this time, indicating that fully randomised patient preference trials are becoming more widely used. By using these designs, we can ensure that when we compare patients who receive their preferred treatment with those who did not receive their preferred treatment they are similar in terms of potential baseline confounders.
In terms of treatment effects, we did see an increased treatment effect size among participants who were randomised to their preferred treatment compared with those who were indifferent to the treatment allocation. Preferences seemed to affect outcome despite the fact that patients with and without a preference were balanced in the most important variable at baseline (that is, baseline score). Whether preferences will materially affect the overall outcome of any one trial will depend on the proportion of participants with a preference, the effect of the preference itself, and the effect size of the intervention. We did not look at the origin of preferences, which may involve the person having had that particular treatment before and knowing that the treatment has helped and what the treatment involves. In this review, we have shown that about 57% of the participants have a preference and those who are allocated to their preference show about 0.162 of a standard deviation increase in effect above those who were indifferent. This might have an important effect on the outcomes of a trial. If, for example, a treatment had an effect size of 0.25 among indifferent participants, then our results suggest that for those with a preference the effect would be about 0.41 (that is, a 64% increase in effect size). This increased effect is not necessarily a bias—it might be a genuine effect in the sense that the treatment works better among patients who desire and receive their preferred treatment. It could, however, change the cost effectiveness of a treatment. A treatment may not be cost effective overall but could be among a subgroup of participants who prefer that treatment.14 However, we cannot assume this to be true for people with strong preferences for a treatment and who consequently did not take part in the trials included in this study. The behaviour of this group of people is unknown—for example, they may become easily disappointed if the treatment does not meet their expectations. Consequently, we would conclude that in trials—particularly when a novel therapy is available only within the context of the study and outcomes are subjective—pre-randomised preferences should be identified and recorded so that they can be accounted for in any analysis. Although an effect size of 0.162 seems to be modest, we need to set this in the context of generally modest effect sizes from effective treatments for musculoskeletal problems in general. For example, the UK BEAM trial, which evaluated manipulation with or without exercise for the treatment of low back pain, found an effect size of 0.34 for exercise after three months and an effect size of 0.39 for manipulation.15 Our results provide no evidence that “resentful demoralisation” leads to a reduction in effect size if people are allocated to their undesired treatment.
Stratifying by preferences would help the power of a trial if either the trial was small or the proportion of patients with a preference was small; if, for instance, 10% of participants had a preference and the trial only had 100 participants, all 10 (or at least eight or nine) participants with a preference might end up in one group. Consequently, we would be unable to control for the possible effects of preference in a regression analysis. Therefore, in small trials or trials in which the proportion of patients with a preference is small, trialists might consider using preference as a stratification variable.
This review shows that treatment preferences affect outcomes in a sample of musculoskeletal trials but that they are not detrimental to attrition rates. The fully randomised preference trial seems to be more widely used now, which would enable further work to determine the treatment effects of patients’ preferences in different clinical conditions.
What is already known on this topic
The effect of patients’ preferences on treatment outcomes in randomised controlled trials is uncertain
Alternative, partially randomised, trials have been designed to overcome the potential problem of patients’ preferences in trials
An existing systematic review found no evidence that preferences influenced attrition and some evidence of an effect on outcomes in partially randomised trials
What this study adds
Patients’ preferences do affect treatment outcomes in randomised controlled trials in musculoskeletal medicine
“Resentful demoralisation” did not occur in participants who did not get their preferred treatment
Use of the standard randomised controlled trial, which collects preference data before randomisation, provides the opportunity to take preference effects into account in the analysis
Cite this as: BMJ 2008;337:a1864
This paper was a joint collaboration with the members of the Preference Collaborative Group: Simon J Adamson, J Martin Bland, Elaine M Hay, Ruth E Johnson, Gareth T Jones, Henry Kitchener, Jennifer A Klaber Moffett, Gary J Macfarlane, Hugh MacPherson, Sionnadh McLean, Linsey Nelson, Chris Salisbury, Elaine Thomas, Helen E Tilbrook, and David J Torgerson.
Contributors: JMB provided advice on the statistical analysis and conceived the framework of the statistical analysis. HET did the systematic review as part of her MSc dissertation, did the analysis, and wrote the first draft of the paper. DJT had the idea for the review, supervised HET, identified some papers for inclusion in the review, and helped to write the first draft of the paper. SJA, EMH, REJ, GTJ, HK, JAKM, GJM, HMacP, SMcL, LN, CS, and ET contributed data to the study. SJA, JMB, EMH, GTJ, JAKM, GJM, HMacP, LN, CS, ET, and DJT critically reviewed versions of the manuscript. HET is the guarantor.
Competing interests: None declared.
Provenance and peer review: Not commissioned; externally peer reviewed.
This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.