Authors' Reply: Physical distancing interventions and incidence of coronavirus disease 2019: natural experiment in 149 countries
Dear Editor,
In response to our recent article in the BMJ,[1] Fountoulakis et al. noted that our findings of a reduction in the incidence of coronavirus disease 2019 (covid-19) associated with physical distancing interventions were qualitatively in agreement with previously published studies, even though these studies used different effect measures such as reproduction number or transmission potential. The authors also reported that, using similar methodology, their research group found that earlier restrictions on public gatherings was the most effective measure in reducing the covid-19 mortality where all the other physical distancing measures were in place.[2] This finding also aligns with our results that restriction on mass gatherings is a key component of an effective viral containment strategy. These findings strongly support the overall effectiveness of physical distancing interventions despite the differences in study methods an analytic strategies.
Fountoulakis et al. noted that they used covid-19 mortality, as opposed to covid-19 cases, in their study since they considered that “the number of deaths is highly reliable”. We respectfully disagree with this assumption; under-reporting of covid-19 deaths has been reported in many countries including those in the Europe and in the US.[3-7] As we mentioned in our article,[1] covid-19 cases are likely under-reported as are covid-19 deaths.[3-7] However, as noted in the limitations section of our study, we agree with Fountoulakis et al. and Gelman that there are some degrees of heterogeneity across countries in terms of methodology in defining the cases of covid-19, which is inherent to “real-world” data, especially in the context of a pandemic.[8] If we wait for the ‘perfect’ data during a pandemic, we would have to abandon most, if not all, rapid research projects that aim to shed light on the pandemic during these early stages.
In another response, Zadey highlighted two important aspects of the study—the timeline of the ‘lockdown’ in India, and the model fit. This variable 'lockdown' was defined as "a combination of two variables: stay at home regulations and restrictions on movements within a country", as the author noted. The author noted that the India-wide lockdown “in the form of ‘curfew’” was implemented on March 23, 2020, while it was recorded as early as January 26, 2020 in the Oxford covid-19 Government Response Tracker database.[9] We contacted our colleagues of the Oxford covid-19 Government Response Tracker team, and located the specific document that was used for the date for this intervention (more details including the sources of data on these policy interventions are available at https://www.bsg.ox.ac.uk/research/research-projects/coronavirus-governme...). This policy was first recommended by the Government of Kerala in a document published on the 26th of January,[10] which was highly applauded by the World Health Organization for Kerala’s success in handling the pandemic.[11]
As we mentioned in our article, our analysis did not separate out regional vs national measures (largely due to unavailability of systematically collected regional data on covid-19 cases and physical distancing policies across the countries), nor did it examine the adherence to, compliance with, or stringency of the interventions.[1] Also, policy recommendation does not necessarily indicate policy implementations, as we also noted in our article. Therefore, our analysis used the first day of recommendation of the respective policy interventions ('lockdown' in this case), regionally or nationally. However, we acknowledge that regional differences (eg, between states/provinces) can be substantial, especially in larger countries. Since India is one of the largest countries with regional governments, we, therefore, conducted a sensitivity analysis excluding seven largest countries (eg, India), and the main finding remained unchanged.
Zadey and Gelman also raised concerns about the model fit and model parameters. Our interrupted time series model allows for both a change in slope (incidence rate) and a change in level at the time of intervention, the latter of which can therefore look like a "jump" in the fitted line on the incidence graphs (supplementary appendix). We agree with Zadey that change in level may also be relevant in other contexts, but here we chose to focus on change in the slope because we were most interested in the effect over the full post-intervention period examined, and did not anticipate that the intervention would have an immediate effect (i.e. a sudden jump in level) in most countries.
We are of course well aware that the model fits better in some countries than others. A different model may have fit the data better in Canada, for example, but not necessarily in other countries. Using different models in different countries would have precluded the ability to perform a meta-analysis of results across countries.
The graphs shown in the supplementary appendix were provided for additional data on the covid-19 cases for each country up to May 30, 2020. However, our interrupted time series models were restricted up to 30-days post-intervention or May 30, 2020, whichever came first, as specified in our article. The graphs from this analysis are available at https://github.com/shabnam-shbd/COVID-19_Physical_Distancing_Policy/blob..., which shows a considerably better fit in most countries. Again, this model also fits some countries better than others. By presenting graphically the model fit in every country as well as the country-level results, we have been transparent about the approach. The data are available if scientists wish to do more detailed modelling of country-specific data.
We thank all the authors for their interests and scholarly responses to our article.
References
1 Islam N, Sharp SJ, Chowell G, et al. Physical distancing interventions and incidence of coronavirus disease 2019: natural experiment in 149 countries. BMJ 2020;370:m2743. doi:10.1136/bmj.m2743
2 Fountoulakis K, Fountoulakis N, Koupidis S, et al. Factors determining different death rates because of the COVID-19 outbreak among countries. J Public Health 2020;(In Press).
3 Wu J, McCann A, Katz J, et al. 153,000 Missing Deaths: Tracking the True Toll of the Coronavirus Outbreak. N. Y. Times. https://www.nytimes.com/interactive/2020/04/21/world/coronavirus-missing... (accessed 21 Jul 2020).
4 Biswas S. The ‘mystery’ of India’s low Covid-19 death rate. BBC News. 2020. https://www.bbc.com/news/world-asia-india-52435463 (accessed 20 Jul 2020).
5 The Economist. Tourism flows and death rates suggest covid-19 is being under-reported. The Economist 2020. https://www.economist.com/graphic-detail/2020/03/07/tourism-flows-and-de... (accessed 20 Jul 2020).
6 Walsh NP, Krever M. The UK’s ‘coronavirus dashboard’ may be under-reporting deaths significantly. CNN. 2020. https://www.cnn.com/2020/04/07/uk/coronavirus-uk-deaths-intl-gbr/index.html (accessed 21 Jul 2020).
7 Cohen J. Underreporting Of COVID-19 Coronavirus Deaths In The U.S. And Europe (Update). Forbes. https://www.forbes.com/sites/joshuacohen/2020/04/14/underreporting-of-co... (accessed 21 Jul 2020).
8 Zuidgeest MGP, Goetz I, Groenwold RHH, et al. Series: Pragmatic trials and real world evidence: Paper 1. Introduction. J Clin Epidemiol 2017;88:7–13. doi:10.1016/j.jclinepi.2016.12.023
9 Hale T, Petherick A, Phillips T, et al. Variation in government responses to COVID-19. Oxford, UK: : Blavatnik School of Government 2020.
10 Department of Health and Family Welfare, Govt of Kerala. nCorona – Guidelines. 2020. https://web.archive.org/web/20200427215819/http://dhs.kerala.gov.in/wp-c... (accessed 20 Jul 2020).
11 World Health Organization. Responding to COVID-19 - Learnings from Kerala. 2020. https://www.who.int/india/news/feature-stories/detail/responding-to-covi... (accessed 20 Jul 2020).
Competing interests:
No competing interests
21 July 2020
Nazrul Islam
Physician-Epidemiologist
Stephen J Sharp (University of Cambridge, UK); Gerardo Chowell (Georgia State University, USA); Sharmin Shabnam (The Pennsylvania State University, USA); Ichiro Kawachi (Harvard University, USA); Ben Lacey (University of Oxford, UK); Joseph M Massaro (Boston University, USA); Ralph B D’Agostino, Sr (Boston University, USA); Martin White (University of Cambridge, UK)
University of Oxford
Nuffield Department of Population Health, University of Oxford, Oxford, UK
Rapid Response:
Authors' Reply: Physical distancing interventions and incidence of coronavirus disease 2019: natural experiment in 149 countries
Dear Editor,
In response to our recent article in the BMJ,[1] Fountoulakis et al. noted that our findings of a reduction in the incidence of coronavirus disease 2019 (covid-19) associated with physical distancing interventions were qualitatively in agreement with previously published studies, even though these studies used different effect measures such as reproduction number or transmission potential. The authors also reported that, using similar methodology, their research group found that earlier restrictions on public gatherings was the most effective measure in reducing the covid-19 mortality where all the other physical distancing measures were in place.[2] This finding also aligns with our results that restriction on mass gatherings is a key component of an effective viral containment strategy. These findings strongly support the overall effectiveness of physical distancing interventions despite the differences in study methods an analytic strategies.
Fountoulakis et al. noted that they used covid-19 mortality, as opposed to covid-19 cases, in their study since they considered that “the number of deaths is highly reliable”. We respectfully disagree with this assumption; under-reporting of covid-19 deaths has been reported in many countries including those in the Europe and in the US.[3-7] As we mentioned in our article,[1] covid-19 cases are likely under-reported as are covid-19 deaths.[3-7] However, as noted in the limitations section of our study, we agree with Fountoulakis et al. and Gelman that there are some degrees of heterogeneity across countries in terms of methodology in defining the cases of covid-19, which is inherent to “real-world” data, especially in the context of a pandemic.[8] If we wait for the ‘perfect’ data during a pandemic, we would have to abandon most, if not all, rapid research projects that aim to shed light on the pandemic during these early stages.
In another response, Zadey highlighted two important aspects of the study—the timeline of the ‘lockdown’ in India, and the model fit. This variable 'lockdown' was defined as "a combination of two variables: stay at home regulations and restrictions on movements within a country", as the author noted. The author noted that the India-wide lockdown “in the form of ‘curfew’” was implemented on March 23, 2020, while it was recorded as early as January 26, 2020 in the Oxford covid-19 Government Response Tracker database.[9] We contacted our colleagues of the Oxford covid-19 Government Response Tracker team, and located the specific document that was used for the date for this intervention (more details including the sources of data on these policy interventions are available at https://www.bsg.ox.ac.uk/research/research-projects/coronavirus-governme...). This policy was first recommended by the Government of Kerala in a document published on the 26th of January,[10] which was highly applauded by the World Health Organization for Kerala’s success in handling the pandemic.[11]
As we mentioned in our article, our analysis did not separate out regional vs national measures (largely due to unavailability of systematically collected regional data on covid-19 cases and physical distancing policies across the countries), nor did it examine the adherence to, compliance with, or stringency of the interventions.[1] Also, policy recommendation does not necessarily indicate policy implementations, as we also noted in our article. Therefore, our analysis used the first day of recommendation of the respective policy interventions ('lockdown' in this case), regionally or nationally. However, we acknowledge that regional differences (eg, between states/provinces) can be substantial, especially in larger countries. Since India is one of the largest countries with regional governments, we, therefore, conducted a sensitivity analysis excluding seven largest countries (eg, India), and the main finding remained unchanged.
Zadey and Gelman also raised concerns about the model fit and model parameters. Our interrupted time series model allows for both a change in slope (incidence rate) and a change in level at the time of intervention, the latter of which can therefore look like a "jump" in the fitted line on the incidence graphs (supplementary appendix). We agree with Zadey that change in level may also be relevant in other contexts, but here we chose to focus on change in the slope because we were most interested in the effect over the full post-intervention period examined, and did not anticipate that the intervention would have an immediate effect (i.e. a sudden jump in level) in most countries.
We are of course well aware that the model fits better in some countries than others. A different model may have fit the data better in Canada, for example, but not necessarily in other countries. Using different models in different countries would have precluded the ability to perform a meta-analysis of results across countries.
The graphs shown in the supplementary appendix were provided for additional data on the covid-19 cases for each country up to May 30, 2020. However, our interrupted time series models were restricted up to 30-days post-intervention or May 30, 2020, whichever came first, as specified in our article. The graphs from this analysis are available at https://github.com/shabnam-shbd/COVID-19_Physical_Distancing_Policy/blob..., which shows a considerably better fit in most countries. Again, this model also fits some countries better than others. By presenting graphically the model fit in every country as well as the country-level results, we have been transparent about the approach. The data are available if scientists wish to do more detailed modelling of country-specific data.
We thank all the authors for their interests and scholarly responses to our article.
References
1 Islam N, Sharp SJ, Chowell G, et al. Physical distancing interventions and incidence of coronavirus disease 2019: natural experiment in 149 countries. BMJ 2020;370:m2743. doi:10.1136/bmj.m2743
2 Fountoulakis K, Fountoulakis N, Koupidis S, et al. Factors determining different death rates because of the COVID-19 outbreak among countries. J Public Health 2020;(In Press).
3 Wu J, McCann A, Katz J, et al. 153,000 Missing Deaths: Tracking the True Toll of the Coronavirus Outbreak. N. Y. Times. https://www.nytimes.com/interactive/2020/04/21/world/coronavirus-missing... (accessed 21 Jul 2020).
4 Biswas S. The ‘mystery’ of India’s low Covid-19 death rate. BBC News. 2020. https://www.bbc.com/news/world-asia-india-52435463 (accessed 20 Jul 2020).
5 The Economist. Tourism flows and death rates suggest covid-19 is being under-reported. The Economist 2020. https://www.economist.com/graphic-detail/2020/03/07/tourism-flows-and-de... (accessed 20 Jul 2020).
6 Walsh NP, Krever M. The UK’s ‘coronavirus dashboard’ may be under-reporting deaths significantly. CNN. 2020. https://www.cnn.com/2020/04/07/uk/coronavirus-uk-deaths-intl-gbr/index.html (accessed 21 Jul 2020).
7 Cohen J. Underreporting Of COVID-19 Coronavirus Deaths In The U.S. And Europe (Update). Forbes. https://www.forbes.com/sites/joshuacohen/2020/04/14/underreporting-of-co... (accessed 21 Jul 2020).
8 Zuidgeest MGP, Goetz I, Groenwold RHH, et al. Series: Pragmatic trials and real world evidence: Paper 1. Introduction. J Clin Epidemiol 2017;88:7–13. doi:10.1016/j.jclinepi.2016.12.023
9 Hale T, Petherick A, Phillips T, et al. Variation in government responses to COVID-19. Oxford, UK: : Blavatnik School of Government 2020.
10 Department of Health and Family Welfare, Govt of Kerala. nCorona – Guidelines. 2020. https://web.archive.org/web/20200427215819/http://dhs.kerala.gov.in/wp-c... (accessed 20 Jul 2020).
11 World Health Organization. Responding to COVID-19 - Learnings from Kerala. 2020. https://www.who.int/india/news/feature-stories/detail/responding-to-covi... (accessed 20 Jul 2020).
Competing interests: No competing interests