Impact of patient and public involvement on enrolment and retention in clinical trials: systematic review and meta-analysisBMJ 2018; 363 doi: https://doi.org/10.1136/bmj.k4738 (Published 28 November 2018) Cite this as: BMJ 2018;363:k4738
- Joanna C Crocker, research fellow1 2,
- Ignacio Ricci-Cabello, Miguel Servet research fellow3 4 5,
- Adwoa Parker, research fellow6,
- Jennifer A Hirst, senior researcher7,
- Alan Chant, patient partner2,
- Sophie Petit-Zeman, former director of patient involvement2,
- David Evans, professor in health services research8,
- Sian Rees, patient and public involvement, engagement and experience theme lead9
- 1Health Experiences Research Group, Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford OX2 6GG, UK
- 2National Institute for Health Research (NIHR) Oxford Biomedical Research Centre (BRC), John Radcliffe Hospital, Oxford, UK
- 3Balearic Islands Health Research Institute (IdISBa), Palma de Mallorca, Spain
- 4Primary Care Research Unit of Mallorca, Balearic Islands Health Service, Palma de Mallorca, Spain
- 5Ciber de Epidemiologia y Salud Publica (CIBERESP), Madrid, Spain
- 6York Trials Unit, Department of Health Sciences, University of York, York, UK
- 7Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK
- 8University of the West of England, Bristol, UK
- 9Oxford Academic Health Science Network, Oxford, UK
- Correspondence to: J Crocker
- Accepted 30 October 2018
Objective To investigate the impact of patient and public involvement (PPI) on rates of enrolment and retention in clinical trials and explore how this varies with the context and nature of PPI.
Design Systematic review and meta-analysis.
Data sources Ten electronic databases, including Medline, INVOLVE Evidence Library, and clinical trial registries.
Eligibility criteria Experimental and observational studies quantitatively evaluating the impact of a PPI intervention, compared with no intervention or non-PPI intervention(s), on participant enrolment and/or retention rates in a clinical trial or trials. PPI interventions could include additional non-PPI components inseparable from the PPI (for example, other stakeholder involvement).
Data extraction and analysis Two independent reviewers extracted data on enrolment and retention rates, as well as on the context and characteristics of PPI intervention, and assessed risk of bias. Random effects meta-analyses were used to determine the average effect of PPI interventions on enrolment and retention in clinical trials: main analysis including randomised studies only, secondary analysis adding non-randomised studies, and several exploratory subgroup and sensitivity analyses.
Results 26 studies were included in the review; 19 were eligible for enrolment meta-analysis and five for retention meta-analysis. Various PPI interventions were identified with different degrees of involvement, different numbers and types of people involved, and input at different stages of the trial process. On average, PPI interventions modestly but significantly increased the odds of participant enrolment in the main analysis (odds ratio 1.16, 95% confidence interval and prediction interval 1.01 to 1.34). Non-PPI components of interventions may have contributed to this effect. In exploratory subgroup analyses, the involvement of people with lived experience of the condition under study was significantly associated with improved enrolment (odds ratio 3.14 v 1.07; P=0.02). The findings for retention were inconclusive owing to the paucity of eligible studies (odds ratio 1.16, 95% confidence interval 0.33 to 4.14), for main analysis).
Conclusions These findings add weight to the case for PPI in clinical trials by indicating that it is likely to improve enrolment of participants, especially if it includes people with lived experience of the health condition under study. Further research is needed to assess which types of PPI work best in particular contexts, the cost effectiveness of PPI, the impact of PPI at earlier stages of trial design, and the impact of PPI interventions specifically targeting retention.
Systematic review registration PROSPERO CRD42016043808.
Poor recruitment and retention of patients in trials are major sources of research inefficiency because they delay the delivery of research, inflate its costs, and can lead to biased findings.12 The top inefficiency in the conduct of trials from recruitment of the first participant to publication of results is failure to meet recruitment targets.3 Directors of UK clinical trials units have identified “research into methods to boost recruitment in trials” and “methods to minimise attrition” as the top two priorities for trials methodology research.4 In the UK, only 56% of trials funded by the Health Technology Assessment programme recruit their originally specified target sample size, with 32% receiving an extension.56 Several initiatives aimed at improving recruitment and retention in clinical trials have been established, including the MRC START research programme and Trial Forge.78 Recruitment and retention interventions identified as meriting formal evaluation include patient and public involvement (PPI).9
In the UK, PPI (also known as “public involvement”) has been defined as “research being carried out ‘with’ or ‘by’ members of the public (including patients and carers) rather than ‘to’, ‘about’ or ‘for’ them.”10 Trials in the UK have experienced a recent surge in PPI activity, partly because the National Institute for Health Research (NIHR) now expects active PPI in the research it funds.11 Patients and members of the public are primarily involved in agenda setting, steering committees, ethical review, protocol development, and piloting.12 Many different types of involvement exist, from one person to many people or whole patient organisations, from one-off involvement in a particular aspect of the trial (for example, reviewing draft information for patients or recruiting participants from their communities) to involvement throughout the trial (for example, as members of a trial steering committee), and from involvement with no decision making power (for example, as advisers) to involvement in decision making as equal partners.
Two broad arguments are made for involving patients and members of the public in health research: the moral argument (those affected by, or paying for, research should have a say in what and how it is done) and the consequentialist argument (PPI should improve the quality, efficiency, and impact of research). Because clinical trialists and funders are steeped in a predominantly quantitative, evidence based culture, the consequentialist argument for PPI in clinical trials (for example, that it increases participant enrolment rates) is likely to play an important role in the adoption of meaningful PPI as routine, widespread practice. Hypotheses about how PPI could increase enrolment rates include greater access to potential participants, improved information sheets, more patient centred trial design, more relevant research questions, and peer endorsement of research.13141516 One observational study of 114 trials reported a doubled odds of successful recruitment associated with “consumer input,” but this did not attain statistical significance (odds ratio 2.00, 95% confidence interval 0.36 to 10.05).17 A more recent observational study reported a statistical association between PPI and success of recruitment among UK mental health research studies,13 but many potential confounding factors could not be controlled for, and information about the nature of PPI in the included studies was lacking. Exploring the effectiveness of PPI practices to improve recruitment and retention of trial participants has been identified as one of the top research priorities for PPI in trials.18
This review aimed to measure the impact of PPI interventions on recruitment (specifically participant enrolment) and retention in clinical trials. A secondary objective was to explore how this impact varies according to context (for example, patient population, recruitment setting, trial treatment/intervention) and the nature of the PPI intervention (for example, activities, involvement model, and other PPI characteristics).
Our systematic literature review followed the PRISMA statement.19 We did a systematic electronic search in the following databases (last updated October 2017): Medline, Science Citation Index, Social Science Citation Index, Embase, PsychINFO, Cochrane Library, CINAHL, and Health Expectations journal. We constructed the search strategy by combining keywords within four topic domains: clinical trials, PPI, enrolment or retention of participants, and potential outcomes/change (see appendix 1). In addition to the electronic database search, we searched the INVOLVE Evidence Library for any papers pertaining to the impact of public involvement on health or public health research,20 as well as the ClinicalTrials.gov and WHO ICTRP clinical trial registries.
Screening and study selection
We conceptualised PPI as a complex intervention,21 involving human behaviours and often multiple interactive components. We included papers that quantitatively evaluated the impact of a PPI intervention, compared with no intervention or another non-PPI intervention, on enrolment and/or retention rates in a clinical trial or trials in any patient population (see eligibility criteria in table 1 for further details). We defined “PPI intervention” as an intervention that was, or included as an active component, any form of PPI consistent with the INVOLVE definition of public involvement: “research being carried out ‘with’ or ‘by’ members of the public rather than ‘to’, ‘about’ or ‘for’ them,” where the term public includes patients, potential patients, carers, and people who use health and social care services, as well as people from organisations that represent people who use services.10 This included interventions not necessarily labelled or conceptualised as “PPI” by the study authors (for example, user testing, peer recruitment, and community based participatory research). We included interventions in which PPI was integrated with additional components inseparable from the PPI (such as involvement of other stakeholders) because this is consistent with the way patients are often involved in practice (for example, being part of an advisory group). Hereafter, we refer to such components as “non-PPI components” of interventions.
A review restricted to randomised controlled trials would give an incomplete summary of the impact of PPI, as many types of PPI interventions (for example, patient involvement in the early stages of trial design) are not amenable to randomisation; we therefore included non-randomised as well as randomised evaluations, with a plan for assessing risk of bias. We accepted all non-randomised study designs (provided there was a direct comparison group), including non-randomised controlled trials, controlled and uncontrolled before-after studies, and observational studies. Comparison groups were patients unexposed to the PPI intervention (for example, before its introduction) or patients exposed to an alternative intervention with no PPI (for example, recruitment via healthcare professionals). The evaluation did not have to be the study authors’ primary research question. We put no limits on publication date or language.
Initially, one reviewer (JC) screened all titles and abstracts for potentially eligible papers and subsequently assessed full text papers against the eligibility criteria. Another reviewer (SR) supervised this process and provided advice when there was uncertainty about eligibility. Later, we received funding for a second reviewer (IRC) to independently screen all records in addition to JC. At the end of this process, JC and IRC compared their results in terms of studies included and excluded. Discrepancies were discussed and the opinion of a third reviewer (AP) was sought when necessary to achieve consensus. We contacted authors to provide further information when confirmation of eligibility was needed.
AP and IRC also did forwards and backwards citation searches by hand searching reference lists of included studies and review articles and using the “cited by” function in Scopus. Any potentially eligible papers were double screened for eligibility by JC.
Using a standardised data extraction form in Microsoft Access, one of three reviewers (JC, AP, or IRC) extracted qualitative information from each paper about the context of the trial, the nature of PPI interventions, and the nature and findings of evaluations. This form was piloted and revised by JC and AP in the early stages. Two reviewers (JC and IRC) then independently extracted quantitative data from included papers on the primary outcomes (enrolment and retention), context, and PPI intervention into a standardised Microsoft Excel spreadsheet for the meta-analyses. For enrolment, we extracted the number of people invited, approached, or reached during the recruitment period (denominator) and the number who consented to take part in the clinical trial (numerator). We included the proxy denominator “total number of participants,” where the intervention targeted a subgroup within the trial population (such as a minority ethnic group or specific geographical region) and the evaluation compared subgroup proportions with and without the intervention. For retention, we extracted the number of people who consented to take part (denominator) and the number who adhered to the trial protocol and/or completed follow-up for the longest period of time investigated by the authors (numerator). We chose the context and intervention variables a priori (table 2) because we considered them to be potentially influential on enrolment and retention outcomes, they are sometimes or often reported in study publications, and, if categorical, they could be split into no more than two or three categories (owing to the small overall sample size). This is consistent with recommendations that systematic reviews of complex interventions include typologies of the structural characteristics of the intervention and, where few or no typologies exist, that face validity for categorisation be provided by experts working in the field.23 We considered theories of change underpinning interventions to be potentially important, but we could not categorise them appropriately for inclusion in this analysis. We are doing a realist analysis on the same sample of studies to shed light on the underlying theory and mechanisms of impact of the included interventions (to be published separately).
Discrepancies between the two data extractors (JC and IRC) were discussed, and the opinion of a third reviewer (AP) was sought if necessary to achieve consensus. We sought additional or accompanying papers where necessary to obtain the data we needed (for example, papers describing the contextual clinical trial or the development of the intervention) and contacted authors to provide further information when insufficient data were reported in available papers.
Risk of bias assessment
Two reviewers (JC and IRC) independently assessed the risk of bias of the studies included in meta-analyses by using the Cochrane Risk of Bias tool for randomised studies and the ROBINS-I tool for non-randomised studies (with pre-specified potential confounding domains of time, funder, and patient population).2425 Discrepancies were discussed and a third reviewer consulted if necessary to achieve consensus. The studies were assessed for risk of bias in relation to our review question, not the study authors’ primary research question (which often differed from ours, particularly for the non-randomised studies).
The only criterion for carrying out meta-analyses was the availability of sufficient outcomes. We took the view that any amount of statistical heterogeneity would be acceptable,26 and we considered that, even in the presence of high heterogeneity, an estimate of the average effect of PPI interventions across studies and the statistical significance of this effect were worth reporting. We did two separate meta-analyses to determine the average impact of PPI interventions on enrolment and retention. We combined the numbers of participants enrolled and retained with and without PPI by using a random effects DerSimonian and Laird meta-analysis to report odds ratios. We used the Hartung-Knapp-Sidik-Jonkman variance correction to calculate 95% confidence intervals reflecting the uncertainty in heterogeneity estimates.272829 We examined statistical heterogeneity by using the I2 statistic and by calculating approximate 95% prediction intervals (which indicate a predicted range for the true effect of a PPI intervention in an individual study)30 using methods reported by Higgins et al.31 Because of high methodological and statistical heterogeneity across non-randomised studies, we made a post-hoc decision to present findings from randomised studies only as our main analysis. We then did a secondary analysis including non-randomised studies as well as randomised studies. Where multiple non-PPI recruitment strategies had been used within a non-randomised study, we pooled the data for comparison with the PPI recruitment strategy. Where multiple PPI interventions had been compared within a study, we included both interventions as separate comparisons in the meta-analysis and split numbers of participants in the comparator group equally across the two intervention arms.
We did pre-planned subgroup analyses on all included studies (randomised and non-randomised combined) to explore the influence of context and characteristics of the PPI intervention on the association between PPI interventions and enrolment rates and to investigate sources of heterogeneity (table 2). We used univariable meta-regression to determine whether differences between subgroups were statistically significant.
We did sensitivity analyses on both the main analysis (randomised studies only) and the secondary analysis (randomised and non-randomised studies combined). These excluded studies at high risk of bias, studies with small sample sizes (n<100), PPI interventions that included additional non-PPI components, PPI interventions that were formal qualitative research (and therefore not universally classified as PPI), and studies using a proxy denominator to measure enrolment rate (see table 2).
We used Peters’ test to examine small study effects.3233 As only two included studies investigated the cost per participant enrolled of PPI versus non-PPI interventions, we did not do a meta-analysis for this outcome. We used Stata 14.0SE for all analyses, with a threshold of P<0.05 to determine statistical significance.
Patient and public involvement
The idea for this review emerged from meetings with an advisory panel for JC’s research fellowship in PPI impact assessment, which included two patient partners (including author AC). The patient partners were involved in the group to ensure that the research was relevant to, and informed by the perspectives of, patients and members of the public. They were chosen because of their long term experience of involvement in health research and their interest in impact assessment. The decision to do this review was in part due to our patient partners’ desire to quantitatively assess the impact of PPI, particularly on recruitment of patients to clinical trials, because “a trial that recruits more quickly will ultimately benefit patients more quickly.” While the review was underway, one patient partner (MO) retired and a third (RH) joined the group.
The patient partners provided input at six advisory group meetings and email correspondence in between meetings. As well as helping to decide on the review question, they helped to decide on our definition of PPI, which contextual and intervention characteristics to explore and how to categorise them, and which potential confounding factors to focus on in the risk of bias assessments. In addition to influencing these decisions, their enthusiasm and belief in the importance of this work helped to maintain the lead author’s motivation through what was a challenging piece of work. Working in partnership with patients has been a very positive experience for the researchers in the team, and we have not identified any negative effects on the research. Our current patient partners (AC and RH) report multiple positive aspects of their involvement, including being interested in the topic and endorsing its importance, feeling welcomed and respected as part of the project team, and feeling that their contributions are valued and responded to. Negative aspects have included difficulty following the conversation and contributing during teleconference meetings (sometimes necessary because of the long geographical distance between RH and the lead author) and having only a limited understanding of the mathematics of the meta-analysis.
Characteristics of studies included in systematic review
Our search results yielded 11 856 records. After excluding duplicates, two independent reviewers screened 6939 titles and abstracts and assessed 134 full text articles for eligibility. Twenty six studies met the criteria for inclusion in the review (fig 1).
Table 3, table 4, and table 5 show the detailed characteristics of all included studies. Most were conducted in the USA or the UK, and together they covered a wide range of clinical topic areas and trial interventions. The PPI interventions were also diverse. Patients and/or members of the public were involved in different activities: eight studies involved patients or lay people in designing recruitment and retention strategies (for example, as community partners, members of a community advisory board, or focus group participants),3441515566697576 12 studies involved patients or lay people in developing patient-facing information (for example, patient information sheets, multimedia and online interventions, recruitment advertisements, and verbal messaging),394143454953585961636678 and 10 studies involved patients or lay people in directly recruiting or retaining participants (for example, hiring lay/community workers or asking existing participants to refer friends/relatives).34364647516065677279 The extent of involvement ranged from one patient advocate acting as a panellist in a one-off educational seminar for recruiting clinicians,59 to more than 80 people helping to develop a patient friendly online trials registry,4142 or community partners initiating and leading their own recruitment strategies.5176 Many intended purposes of involvement also existed, including increasing trust between communities and researchers,344651606579 improving the quality and acceptability of patient-facing information or recruitment messages,39414353616378 accessing potential participants via existing participants,3667 and increasing the cultural competence of the research among minority ethnic communities.364660676972757679 Many of the PPI interventions also included non-PPI components, such as the involvement of other stakeholders or experts,415861656975 or novel modes of information delivery that were not a direct consequence of the PPI.45536067727879
Characteristics of studies included in meta-analyses
We included 19 studies (21 PPI interventions) reporting data from 178 921 participants in our enrolment meta-analyses and five studies (six PPI interventions) reporting data from 6520 participants in our retention meta-analyses. Table 6 shows the aggregate characteristics of these studies, including those used in subgroup and sensitivity analyses.
Six studies could not be included in the enrolment meta-analyses owing to insufficient data, despite our attempts to contact study authors and identify related papers. Three of these studies reported no significant impact of PPI interventions on enrolment,585965 and the other three studies reported an increase in enrolment rates associated with PPI interventions (statistical significance unknown).667578
Risk of bias of studies included in meta-analyses
Of the eight randomised studies, only one was deemed to be at “high” risk of bias owing to missing outcome data,41 two had “some concerns,”4546 and five had “low” risk of bias.3947495361 Of the 12 non-randomised studies, 11 were deemed to be at “serious” risk of bias,3643515560636769727679 and one was deemed to be at “critical” risk of bias owing to potential uncontrolled confounding by patient population, time, or both.34 Often this was because the study was opportunistic—for example, comparing the success of different recruitment strategies—rather than designed specifically to evaluate the impact of PPI versus non-PPI on enrolment or retention.
Impact of PPI interventions on enrolment
Individual study findings
Half (11/21) of the PPI interventions included in our meta-analysis were associated with significantly higher enrolment rates compared with no PPI or non-PPI interventions,3643465155606367697276 nine PPI interventions were not significantly associated with enrolment rate,3539414549515361 and one PPI intervention was associated with significantly lower enrolment (odds ratio 0.41, 95% confidence interval 0.23 to 0.72).79 In this study, lay community members (faith based organisations) attempted to directly recruit African-Americans with diabetes to the trial; however, this yielded a lower enrolment rate than recruitment via the health system (non-PPI). The authors stated that this was not surprising, given “the nature of the provider-patient relationship” and because “African Americans may be less inclined to have their personal health history known by other members of their church congregation, given the stigma associated with chronic illnesses.”79 Contrast this with Vincent et al’s study, which showed the largest PPI effect size in our sample (odds ratio 13.48, 6.07 to 29.95): here, lay community members (Catholic church partners, some of whom shared a high risk of diabetes with the Mexican-American target population) initiated, co-designed, and co-delivered a recruitment strategy that was highly successful compared with strategies initiated by the researchers.77 (Note, however, that both of these outlying studies were non-randomised and judged to be at high risk of bias.)
Main meta-analysis (randomised studies only)
We included seven randomised studies (eight PPI interventions) in our main meta-analysis. These interventions all consisted of patient or lay involvement in the design or delivery of patient information, with Ford et al’s intervention also including recruitment sessions hosted by churches in the target community.46 Pooling the data from seven randomised studies in our main meta-analysis showed that, on average, PPI interventions modestly but significantly increased the odds of a patient enrolling in a clinical trial compared with no PPI (odds ratio 1.16, 1.01 to 1.34; P=0.04). We found low heterogeneity between studies (I2=0.0%), yielding a 95% prediction interval of the odds ratio of 1.01 to 1.34 (fig 2).
Secondary meta-analysis and subgroup analyses (randomised and non-randomised studies combined)
Our secondary meta-analysis, combining 19 randomised and non-randomised studies (21 PPI interventions), also found that, on average, PPI interventions significantly increased the odds of a patient enrolling in a clinical trial compared with no PPI or non-PPI interventions (odds ratio 1.87, 1.25 to 2.80; P=0.004). We found substantial heterogeneity between studies (I2=95.7%), yielding a 95% prediction interval of the odds ratio of 0.36 to 9.86 (fig 3). Exploratory subgroup analyses showed that the overall positive association between PPI interventions and enrolment substantially increased when at least one involved person had lived experience of the health condition under study (odds ratio 3.14, 1.89 to 5.22) and all but disappeared when the involved people had no such lived experience (1.07, 0.74 to 1.53). Meta-regression confirmed that this effect was statistically significant (P=0.02). Subgroup differences between any of the other variables explored (appendix 2), including trial intervention type (simple versus complex), the timing of involvement (designing recruitment and retention strategies versus developing patient-facing information versus direct recruitment or retention of participants), and enrolment rate denominator (before versus after eligibility screening), were not found to be statistically significant using meta-regression (P>0.3). Meta-regression was not able to explain the high between study heterogeneity, but it may be due in part to the diverse range of evaluation methods used and the high risk of bias by confounding in non-randomised studies. It could also be explained by heterogeneity of the PPI interventions: almost all of the PPI interventions in the high quality, randomised studies were aimed at improving patient information, whereas the more complex and more unusual interventions were largely evaluated using poorer quality observational or quasi-experimental methods.
Sensitivity analyses and Peters’ test
The positive overall association between PPI interventions and enrolment remained statistically significant throughout all sensitivity analyses except when we excluded interventions with non-PPI components from the secondary analysis (see appendix 3). Although the estimated effect of PPI interventions actually increased in this analysis (odds ratio=2.70), the exclusion of 15/21 studies yielded a very wide 95% confidence interval (0.83 to 8.84). We could not restrict this particular sensitivity analysis to randomised studies because this subsample included only one “pure” PPI intervention.49 Peters’ test showed no evidence of bias due to small study effects (P=0.92 for main analysis; P=0.59 for secondary analysis).
Cost effectiveness of PPI interventions
Of the two studies reporting the cost per participant enrolled, MacEntee et al reported that a PPI strategy to recruit participants at community centres through a local contact person, although more effective, was more than twice the cost per participant of a non-PPI strategy that used postal invitations ($23 (£18; €20) v $11).60 Chlebowski et al reported that a PPI strategy to recruit trial participants via existing research participants was only one quarter the cost of a non-PPI strategy that used commercial mailing lists to send postal invitations ($59 v $259 per participant enrolled).36
Impact of PPI interventions on retention
Main meta-analysis (randomised studies only)
Pooling the data from three randomised studies (four PPI interventions) in our main meta-analysis did not show a statistically significant effect of PPI interventions on participant retention (odds ratio 1.16, 0.33 to 4.14; P=0.73). Results varied widely across studies, with effect estimates ranging from odds ratios of 0.38 to 2.52 (I2=83.5%; 95% prediction interval 0.06 to 22.37; appendix 4).
Secondary meta-analysis (randomised and non-randomised studies combined)
Our secondary meta-analysis, combining five randomised and non-randomised studies (six PPI interventions), also found no statistically significant effect of PPI interventions on participant retention, compared with no PPI or non-PPI interventions (odds ratio 1.20, 0.52 to 2.77; P=0.59). Again, we found substantial heterogeneity between studies (I2=78.3%), yielding a 95% prediction interval of the odds ratio of 0.20 to 7.18 (forest plot in appendix 5). At the individual study level, only one PPI intervention was significantly associated with retention: this constituted use of lay community health advisers to support participants (the only PPI intervention specifically targeting retention), leading to a significant improvement in retention rates (odds ratio 2.52, 1.82 to 3.50).47 Apart from this example, the PPI interventions primarily targeted enrolment, not retention. We did not do subgroup analyses for retention outcomes because of the small sample size.
Sensitivity analyses and Peters’ test
Sensitivity analyses did not alter the findings (appendix 6), and Peters’ test showed no evidence of bias due to small study effects (P=0.44 for main analysis; P=0.41 for secondary analysis).
This review identified a variety of PPI interventions aimed at improving enrolment and retention of participants in clinical trials. Patients and lay members of the public were involved in designing recruitment and retention strategies and patient-facing information, identifying and approaching potential participants, and troubleshooting when recruitment was poor. We did not identify any studies that assessed the impact on enrolment or retention of PPI in developing the trial question or designing the trial itself.
On average, PPI interventions significantly increased the odds of a patient enrolling in a clinical trial, relative to no PPI or non-PPI recruitment interventions. This remained statistically significant regardless of whether non-randomised studies were excluded or included, as well as in sensitivity analysis that removed studies at highest risk of bias. To illustrate what our main findings could mean in practice: in a hypothetical sample of 1000 patients, of which typically 100 enrol (consistent with the 10% average enrolment rate in our sample of studies), a PPI intervention similar to those included in our meta-analysis of randomised studies would likely lead to between one and 30 (average 14) extra patients being enrolled. As these PPI interventions were mostly restricted to patient or lay involvement in the design or delivery of patient information, the effect size might be even larger for PPI that begins at earlier stages of trial design, as the opportunity to influence patients’ views and experiences would extend beyond just the provision of information.
A key exploratory finding was that the effect size was significantly greater when the people involved had lived experience of the health condition under study, compared with no such lived experience. This is consistent with the view that patients and carers can benefit research through their role as “expert in lived experience,”80 although the precise mechanisms linking such expertise with improvements in enrolment and retention are unclear—something that we are exploring in a complementary realist analysis of the included studies. This finding, along with all other subgroup analysis and meta-regression findings, should be interpreted with caution owing to the potential for study level confounding.
Far fewer studies evaluated the impact of PPI interventions on retention of trial participants. They showed, on average, a modest but non-significant improvement in retention; the very wide 95% confidence intervals mean that we cannot rule out a potentially large increase or decrease in retention associated with PPI. None of the PPI interventions in the retention analysis included people with lived experience of the health condition under study, and most of them primarily targeted enrolment rather than retention.
Strengths and limitations of review
To our knowledge, this is the first attempt to combine data on the impact of PPI on enrolment and retention in health research, providing a quantitative summary and exploring the influence of contextual and intervention factors. Our results are consistent with previous observational studies that suggested an average positive association between PPI and success of recruitment in UK based health studies.1516 Unlike these previous studies, our review encompassed all geographies and clinical areas, and we were able to explore, to some extent, the influence of the characteristics and context of PPI.
Our review has several limitations. Most of the interventions included non-PPI components, and we could not separate out the effects of these from the effects of the PPI components. When interventions including non-PPI components were excluded in a sensitivity analysis of both randomised and non-randomised studies combined, PPI was still associated with improved enrolment but with reduced certainty due to the decrease in sample size.
We were unable to explore the influence of many potentially important factors such as underlying programme theory, the fidelity and sustainability of interventions, the quality of relationships between involved patients and researchers, and the attitude of research leaders towards PPI.2381 We are undertaking a realist analysis of the included papers to shed more light on these complexities.23 The framing of PPI as a complex intervention is itself controversial,82 but we believe that this approach, alongside a range of other perspectives, can usefully contribute to the much broader debate about the impact of PPI in health research.
Our 95% prediction intervals should be interpreted with caution because prediction intervals have been reported to be less reliable in meta-analyses with unbalanced study sizes.83 Also, we were unable to provide a useful summary of the cost effectiveness of PPI, because very few studies included an economic impact assessment; thus an “effective” PPI intervention may not necessarily be cost effective. However, financial modelling of the impact of PPI in a typical oncology trial suggests that PPI interventions that improve enrolment may add considerable financial value.84
Finally, the findings of this study say nothing about the quality or ethical acceptability of PPI in the included studies or patients’ views on the importance of the clinical trials being conducted. PPI may improve enrolment, but this does not rule out negative effects such as an emotional cost to the people involved or patients feeling coerced into enrolling.85 Should patients assume that all trials are conducted for their benefit and automatically endorse every trial? Do (and should) involved patients have the necessary skills to assess the risks involved on behalf of their fellow patients? These are important dilemmas that are beyond the scope of this study.
Implications for clinical trialists and PPI policy makers
Our findings add support to the case for involving patients and carers in the design and conduct of clinical trials. In the UK, funding proposals and protocols for trials are often reviewed by institutional lay panels; our review suggests that, ideally, at least some of these reviewers would be patients and carers with lived experience of the health condition under study.
The apparent failure of some PPI interventions to improve enrolment and retention shows that many factors other than PPI also influence these outcomes. In addition, some PPI interventions in our review were one of several recruitment strategies used by clinical trialists and may not have been sufficient alone; for example, Sanders et al found that although their word of mouth PPI strategy was relatively effective at enrolling those it reached, it contributed only 2.2% of the total participants owing to limited reach (200 people), compared with 70.3% for the targeted mail-out strategy (which reached 21 400 people).67 PPI will not solve all recruitment and retention problems, and clinical trialists would be wise to implement multiple additional strategies to minimise the risk of poor enrolment and retention. Furthermore, involving patients in the early stages of trial development can sometimes lead researchers to abandon the whole idea of the trial,86 suggesting that if the target patients are not convinced that the trial question is worth answering, PPI in later stages of the trial (such as those seen in this review) may be futile.
Unanswered questions and future research
Well planned, high quality evaluations are needed to improve our understanding of the impact of PPI on enrolment and retention in clinical trials. In particular: which types of PPI work best in particular settings and contexts; the mechanisms underlying the impact of PPI on enrolment and retention; the cost effectiveness of PPI interventions (an important part of the drive to improve trial efficiency); the impact of PPI interventions specifically targeting retention (which has received very little attention relative to enrolment); and the impact of PPI at the early stages of trial proposal and design.
What is already known on this topic
Patient and public involvement (PPI) in clinical trials has the potential to improve rates of enrolment and retention of participants
PPI may help by improving trial design, optimising recruitment and retention strategies and patient-facing materials, or directly approaching potential participants
Whether, when, or by how much, PPI affects rates of enrolment and retention of participants is not known
What this study adds
The nature of PPI interventions and the impact of these on trial enrolment and retention vary widely between studies
On average, PPI interventions seem to modestly but significantly increase the odds of participant enrolment
The impact of PPI on retention rates is less clear and requires further primary research evaluating PPI interventions that specifically target retention
We are grateful to Michael Osborne (patient contributor), Shaun Treweek, and Louise Locock (University of Aberdeen) for providing expert advice throughout this study; Ben Feakins (medical statistician, University of Oxford) for providing statistical advice at an early stage of this review; Rebecca Harmston (patient contributor) for contributing to the analysis plan and interpretation of results; and Trish Greenhalgh (NIHR Oxford BRC theme leader, Partnerships for Health Wealth and Innovation) for providing helpful feedback on an early draft of this paper.
Contributors: JCC, AC, SPZ, DE, and SR conceived and designed this review. JCC, IRC, and AP undertook searches, record screening, and data extraction (supervised by JCC). JH wrote the code for and ran the meta-analyses in Stata. All authors contributed to interpretation of the results. JCC wrote the manuscript, and all authors commented on the draft and approved the final version. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted. JCC is the guarantor.
Funding: JC, SPZ, and JH were supported by the National Institute for Health Research (NIHR) Oxford Biomedical Research Centre (BRC). IRC was supported by the University of Oxford Returning Carers Fund. The funders were not involved in the study design; data collection, analysis, or interpretation; or writing the report. All authors had full access to all of the data in the study and can take responsibility for the integrity of the data and accuracy of the data analysis. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR, or the Department of Health and Social Care.
Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare: no support from any organisation for the submitted work other than that described above; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.
Ethical approval: Not needed.
Data sharing: The dataset is available on request from the corresponding author.
Transparency declaration: The lead author (study guarantor) affirms that this manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained.
This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/.