Real world evidence on whether facemasks can help in the current pandemic
The current pyramid of evidence puts the randomized trial at the top. But the question of whether facemasks work is a question about whether they work in the real world, worn by real people, in real situations.
This is a pragmatic trials question. We should therefore be focusing the argument of benefit or harm around evidence generated from pragmatic trials – trials which have evaluated masks in real world settings. The recent rapid systematic review looked at the question of what is the evidence facemasks work compared to no facemasks (and included evidence from pragmatic and more explanatory type trials; and did not consider the evidence from pragmatic trials of evidence of different types of facemask). The question we should consider to determine whether facemasks will be of benefit in the current pandemic is: what pragmatic evidence is there that facemasks work when worn by real people in real setting in real situations. Here is my summary of the evidence on this:
Evidence from controlled settings
There is little doubt that masks works in controlled settings – they stop particulates penetrating the air [Leung 2020]. Facemasks also seem to prevent infection spreading when worn by people who are infected [Brainard 2020]. Yet, this doesn’t tell us if they will work in the real world.
Evidence from trials of facemasks vs no mask
Systematic review of facemasks vs no mask [Brainard 2020]
There are three RCTs identified in this review where people wore masks to try to prevent other people becoming infected (primary prevention). The authors of the review interpret the evidence from these three RCTs as a small non-significant effect on influenza like illness. But, this is an incorrect interpretation of the result (RR=0.95, 95% CI: 0.75 to 1.19) as this result is compatible with both benefit and harm. The evidence from these three trials should therefore be interpreted as uninformative (or consistent with either benefit or harm). There are observational studies in this review, but these do not allow us to answer the question of whether the masks provide protection as they will be subject to confounding. The largest of the three RCTs was a pragmatic cluster trial in pilgrims [Alfelali 2020]. This is a well conducted pragmatic cluster randomized trial with low risk of bias, but suffered from low compliance. This found OR 1.35, 95% CI 0.88-2.07 which although non-significant, is more suggestive of harm than benefit.
Conclusion: The largest and most pragmatic trial (which informs on how facemasks will perform in the real world) assessing the benefit of facemasks vs no mask is suggestive of more harm than benefit.
Evidence from trials comparing different sorts of facemasks
(This is not based on a systematic review, so there may be other evidence that I am unaware of)
1. CRT of N95 vs surgical mask in health care workers (Respect Trial) [Radonovich 2019]. Large pragmatic trial in health care works. Low risk of bias. The trial reported a non-significant finding which it interpreted as “no significant difference”. However, despite being non-significant, the confidence interval for the primary outcome rules out anything but a small possibility of any clinically important benefit of the N95 respirator. Again, authors have mis interpreted the statistical finding (OR 1.18 95% CI: 0.95 to 1.45). The evidence from this trial in fact supports more that the N95 respirator might be associated with a small amount of harm. Compliance was high. Therefore, this “surprising finding” of increase risk of the N95 mask (when it is known to perform much better in controlled settings) might be attributable to risk compensation.
2. CRT of surgical vs cloth masks in health care workers in LMIC. [MacLntyre 2015] well conducted low risk of bias. Overwhelmingly suggests harm as opposed to benefit (RR=13.00, 95% CI 1.69 to 100.07). The authors conclude this increased risk is likely to be due to the penetration of cloth marks with disease particulates.
Conclusion: The evidence from pragmatic trials (people wearing masks in everyday settings) suggests wearing of facemasks both induces risk compensation behavior and increased virus spreading from poor mask quality.
References
[Leung 2020] Leung, N.H.L., Chu, D.K.W., Shiu, E.Y.C. et al. Respiratory virus shedding in exhaled breath and efficacy of face masks. Nat Med (2020). https://doi.org/10.1038/s41591-020-0843-2
[Brainard 2020] Brainard JS, Jones N, et al. Facemasks and similar barriers to prevent respiratory illness such as COVID-19: a rapid systematic review. medRxiv 2020.04.01.20049528; doi:10.1101/2020.04.01.20049528. https://www.medrxiv.org/content/10.1101/2020.04.01.20049528v1.full.pdf
[Radonovich 2019] Radonovich LJ Jr, Simberkoff MS, Bessesen MT, Brown AC, Cummings DAT, Gaydos CA, Los JG, Krosche AE, Gibert CL, Gorse GJ, Nyquist AC, Reich NG, Rodriguez-Barradas MC, Price CS, Perl TM; ResPECT investigators. N95 Respirators vs Medical Masks for Preventing Influenza Among Health Care Personnel: A Randomized Clinical Trial. JAMA. 2019 Sep 3;322(9):824-833.
[Alfelali 2020] Alfelali, Mohammad and Haworth, Elizabeth Ann and Barasheed, Osamah and Badahdah, Al-Mamoon and Bokhary, Hamid and Tashani, Mohamed and Azeem, Mohammad Irfan and Kok, Jen and Taylor, Janette and Barnes, Elizabeth Helen and El Bashir, Haitham and Khandaker, Gulam and Holmes, Edward Charles and Dwyer, Dominic Edmund and Heron, Leon and Wilson, Godwin Justus and Booy, Robert and Rashid, Harunor, Facemask versus No Facemask in Preventing Viral Respiratory Infections During Hajj: A Cluster Randomised Open Label Trial (March 8, 2019). Available at SSRN: https://ssrn.com/abstract=3349234 or http://dx.doi.org/10.2139/ssrn.3349234
[Maclntyre 2015] MacIntyre CR, Seale H, Dung TC, et al. A cluster randomised trial of cloth masks compared with medical masks in healthcare workers. BMJ Open2015;5:e006577. doi:10.1136/bmjopen-2014-006577. pmid:25903751
Rapid Response:
Real world evidence on whether facemasks can help in the current pandemic
The current pyramid of evidence puts the randomized trial at the top. But the question of whether facemasks work is a question about whether they work in the real world, worn by real people, in real situations.
This is a pragmatic trials question. We should therefore be focusing the argument of benefit or harm around evidence generated from pragmatic trials – trials which have evaluated masks in real world settings. The recent rapid systematic review looked at the question of what is the evidence facemasks work compared to no facemasks (and included evidence from pragmatic and more explanatory type trials; and did not consider the evidence from pragmatic trials of evidence of different types of facemask). The question we should consider to determine whether facemasks will be of benefit in the current pandemic is: what pragmatic evidence is there that facemasks work when worn by real people in real setting in real situations. Here is my summary of the evidence on this:
Evidence from controlled settings
There is little doubt that masks works in controlled settings – they stop particulates penetrating the air [Leung 2020]. Facemasks also seem to prevent infection spreading when worn by people who are infected [Brainard 2020]. Yet, this doesn’t tell us if they will work in the real world.
Evidence from trials of facemasks vs no mask
Systematic review of facemasks vs no mask [Brainard 2020]
There are three RCTs identified in this review where people wore masks to try to prevent other people becoming infected (primary prevention). The authors of the review interpret the evidence from these three RCTs as a small non-significant effect on influenza like illness. But, this is an incorrect interpretation of the result (RR=0.95, 95% CI: 0.75 to 1.19) as this result is compatible with both benefit and harm. The evidence from these three trials should therefore be interpreted as uninformative (or consistent with either benefit or harm). There are observational studies in this review, but these do not allow us to answer the question of whether the masks provide protection as they will be subject to confounding. The largest of the three RCTs was a pragmatic cluster trial in pilgrims [Alfelali 2020]. This is a well conducted pragmatic cluster randomized trial with low risk of bias, but suffered from low compliance. This found OR 1.35, 95% CI 0.88-2.07 which although non-significant, is more suggestive of harm than benefit.
Conclusion: The largest and most pragmatic trial (which informs on how facemasks will perform in the real world) assessing the benefit of facemasks vs no mask is suggestive of more harm than benefit.
Evidence from trials comparing different sorts of facemasks
(This is not based on a systematic review, so there may be other evidence that I am unaware of)
1. CRT of N95 vs surgical mask in health care workers (Respect Trial) [Radonovich 2019]. Large pragmatic trial in health care works. Low risk of bias. The trial reported a non-significant finding which it interpreted as “no significant difference”. However, despite being non-significant, the confidence interval for the primary outcome rules out anything but a small possibility of any clinically important benefit of the N95 respirator. Again, authors have mis interpreted the statistical finding (OR 1.18 95% CI: 0.95 to 1.45). The evidence from this trial in fact supports more that the N95 respirator might be associated with a small amount of harm. Compliance was high. Therefore, this “surprising finding” of increase risk of the N95 mask (when it is known to perform much better in controlled settings) might be attributable to risk compensation.
2. CRT of surgical vs cloth masks in health care workers in LMIC. [MacLntyre 2015] well conducted low risk of bias. Overwhelmingly suggests harm as opposed to benefit (RR=13.00, 95% CI 1.69 to 100.07). The authors conclude this increased risk is likely to be due to the penetration of cloth marks with disease particulates.
Conclusion: The evidence from pragmatic trials (people wearing masks in everyday settings) suggests wearing of facemasks both induces risk compensation behavior and increased virus spreading from poor mask quality.
References
[Leung 2020] Leung, N.H.L., Chu, D.K.W., Shiu, E.Y.C. et al. Respiratory virus shedding in exhaled breath and efficacy of face masks. Nat Med (2020). https://doi.org/10.1038/s41591-020-0843-2
[Brainard 2020] Brainard JS, Jones N, et al. Facemasks and similar barriers to prevent respiratory illness such as COVID-19: a rapid systematic review. medRxiv 2020.04.01.20049528; doi:10.1101/2020.04.01.20049528. https://www.medrxiv.org/content/10.1101/2020.04.01.20049528v1.full.pdf
[Radonovich 2019] Radonovich LJ Jr, Simberkoff MS, Bessesen MT, Brown AC, Cummings DAT, Gaydos CA, Los JG, Krosche AE, Gibert CL, Gorse GJ, Nyquist AC, Reich NG, Rodriguez-Barradas MC, Price CS, Perl TM; ResPECT investigators. N95 Respirators vs Medical Masks for Preventing Influenza Among Health Care Personnel: A Randomized Clinical Trial. JAMA. 2019 Sep 3;322(9):824-833.
[Alfelali 2020] Alfelali, Mohammad and Haworth, Elizabeth Ann and Barasheed, Osamah and Badahdah, Al-Mamoon and Bokhary, Hamid and Tashani, Mohamed and Azeem, Mohammad Irfan and Kok, Jen and Taylor, Janette and Barnes, Elizabeth Helen and El Bashir, Haitham and Khandaker, Gulam and Holmes, Edward Charles and Dwyer, Dominic Edmund and Heron, Leon and Wilson, Godwin Justus and Booy, Robert and Rashid, Harunor, Facemask versus No Facemask in Preventing Viral Respiratory Infections During Hajj: A Cluster Randomised Open Label Trial (March 8, 2019). Available at SSRN: https://ssrn.com/abstract=3349234 or http://dx.doi.org/10.2139/ssrn.3349234
[Maclntyre 2015] MacIntyre CR, Seale H, Dung TC, et al. A cluster randomised trial of cloth masks compared with medical masks in healthcare workers. BMJ Open2015;5:e006577. doi:10.1136/bmjopen-2014-006577. pmid:25903751
Competing interests: No competing interests