We need a gold standard for randomised control trials studying misinformation and vaccine hesitancy on social media

Vaccine hesitancy and the spread of misinformation on social media have been recognised by the World Health Organization as an urgent threat to public health, with potentially lethal consequences.12 During the covid-19 pandemic, US president Joe Biden concluded that misinformation on social media was “killing people.”3 Research about the negative impacts of health misinformation online is increasingly being published, but how can we effectively counter vaccine misinformation on social media? Correlational, exploratory, and descriptive4 studies are overabundant, but we lack a broader framework for using this research base to inform our efforts. We know a lot already from computational approaches about the unique topics and narratives spread within anti-vaccination echo chambers5 and have documented the types of health misinformation that are prevalent on different social media platforms.67 However, systematic reviews show that real world interventions on social media are few and far between.4 Notably, the field also lacks a gold standard for randomised controlled trials (RCTs) conducted on social media. This lack of standardisation poses fundamental challenges to causal inference, raises major ethical questions, and is riddled with inequities in data access and coverage outside the global north.

A challenge many scholars and practitioners currently face is finding out what “works” when it comes to promoting vaccine acceptance and countering vaccine-related misinformation on social media. One obvious point of difference with traditional clinical trials is that social media researchers do not have full access to the platform, procedural aspects of the design, and the necessary raw data to make valid and complete inferences about the outcome of the trial.8 Causal inference faces many challenges. For example, researchers may not be in control of the randomisation procedures used in social media experiments; the nature of the treatment “exposure” (ie, which users see the “treatment” and for how long), some aspects of which might be set by the companies’ algorithms; and what outcome measures are available.89 Progress has been made in linking social media discourse to actual vaccination uptake,510 but social media companies typically limit access to key behavioural data,8 such as what people watch, share, or click on. Consequently, existing data provide a limited and biased snapshot of the larger population.^6. A gold standard for randomised trials on social media would deliver transparent and reproducible evidence.

Countering misinformation and promoting vaccine acceptance on social media is not without ethical challenges. While it is generally accepted that some content posted online may be considered part of the public domain—and can therefore be (re)used in observational research—actively intervening or even exposing people to misinformation as part of an experimental procedure presents a potential barrier to delivering insights in this nascent field.11 For example, on social media it is not always possible to obtain written and informed consent from individuals because interventions are delivered via ad campaigns or placed directly into a user’s feed so there is no direct contact with users. In one prominent study, researchers “nudged” people directly on Twitter via private messages without informed consent. They argued that obtaining consent would have disrupted the naturalistic social media experience.11 Unlike the Helsinki declaration, No standardised ethics procedures—such as those of the Helsinki declaration—exist for social media research. Some ethics committees may not require ethics approval for observational research, whereas others do. Some won’t allow targeted intervention work on social media, whereas others might if the data are fully anonymised, though substantial variation exists in handling data privacy and compliance with data protection regulations.12 The lack of an overarching international ethics framework creates an unlevel playing field for researchers. One proposal could be to borrow guidelines from established protocols in medicine around the handling of sensitive data.

Important questions surround the role of experts in promoting vaccine acceptance. Although doctors are typically among the most trusted professionals, during the covid-19 pandemic some medical credentials were used to peddle fake cures and outright misinformation about vaccination.13 Although it is important to interrogate how science is communicated,14 the repeated questioning of science and expertise by postmodern scholars has inadvertently played into the hands of anti-vaccination movements and post-truth politics.15 The role of medical professionals in countering misinformation and promoting vaccine acceptance therefore remains an important subject of debate.

Systematic reviews have indicated that nearly all research on the nexus between social media, misinformation, and vaccine hesitancy originates from the global north, or what psychologists often term “western, educated, industrialised, rich, democratic (WEIRD)” populations. Consequently, there is a severe lack of insight from the global south and other parts of the world.1617 Given the diverse social and cultural drivers of misinformation susceptibility and vaccine hesitancy,17 this is an important gap in the literature. How can randomised trials be conducted and replicated in other countries? Social media companies often offer less research support for non-English speaking scholars and practitioners. Moreover, strong asymmetries exist, in which only some institutions are granted access to social media data or platforms. For example, my colleagues and I were able to carry out a large randomised experiment on the YouTube platform to empower people to spot manipulation,9 but this was mainly possible because of an existing collaboration with Google. Most researchers do not have such active collaborations with social media companies; therefore we need to democratise and diversify the process through which researchers can obtain data from, or work with, social media companies to understand how to best counter vaccine hesitancy.18

A recent article14 claimed that scientists’ desire to “cure misinformation” has led to “rushed” interventions. I have not observed any evidence for this claim. On the contrary, scientists have been meticulously studying how to counter misinformation on social media for many years, but are confronted with numerous challenges. What we need now is a gold standard for how to conduct research to counter misinformation and vaccine hesitancy on social media across the world.