Civilian deaths from weapons used in the Syrian conflict

We have three concerns about this article. First, the article apportions responsibility for casualties to particular perpetrator organizations based on a single snapshot of territorial control that ignores the numerous (and well-documented) changes in this phenomenon over time. Second, combining Syrian Opposition Groups and Islamic State groups into one category (non-state armed groups; NSAG) obscures the fact that these groups are actively fighting each other. Finally, our work with Syrian human rights groups has taught us that their ability to document the conflict varies as a function of which armed groups are in control of each region. Therefore, the very high rate of missing data is likely to result in distorted patterns in the observed data. We will address these concerns in order in this response.

First, the authors present their findings stratified by geographic areas controlled by NSAG and the Syrian Government. This geographic stratification is based on a map of territorial control published by the BBC. The map shows areas controlled by different factions, and where high frequency conflict was reported between October and December 2014. Although they do not state this explicitly, this stratification choice implies an assessment of proportional responsibility. Such assessments will undoubtedly be important in a post-conflict transition and must be able to withstand scrutiny.

Over the course of the conflict, frontlines, battlefields and control of territory have changed rapidly across Syria. For example, analyses conducted by the Syria Conflict Mapping Project at the Carter Center have found that in the past year alone there have been close to 1,000 documented changes in territorial control among the areas they are monitoring. If they are to be accurate, quantitative claims about the geographic dynamics of violence require that analysts understand the detail of which groups controlled which areas, and when those lines of control changed. Occasionally, analysts may also have to recognize that territorial control at a particular time and place is too uncertain to attribute responsibility to a particular group. Instead of adjusting for territorial control over time, this article extrapolates backward three years from the static BBC map and forward six months from the same map. Similarly, temporal comparisons, as made in Figure 2 of the article, have not been adjusted to reflect temporal variation in territorial control. It is vital that any analyses based on territorial control take into account these rapid changes.

Our second concern pertains to the conflation of Islamic State and Syrian Opposition Groups. Since late 2013 these groups have been fighting each other. As a result, geographic regions that are treated in this analysis as being under homogenous territorial control are actually experiencing active armed confrontations between Islamic State and Syrian Opposition Groups. For example, the Washington Post describes community celebrations in February 2014 when the Free Syrian Army regained control over the town of Azaz after a “…five-month reign of terror” by factions of the Islamic State [1].

Lastly, as shared with us directly by Syrian human rights groups and described in “Stolen Futures” by the Oxford Research Group/Every Casualty [2], these groups’ ability to collect data varies dramatically depending on who has control over a territory. Data collection on both killed civilians and combatants is actively impeded by the Syrian government in areas controlled by their forces. In work previously published on estimating the dark figure in Syria [3] we provide evidence for higher levels of underreporting of fatalities in the immediate aftermath of territorial gains by the government.

The authors acknowledge these challenges in Appendix C: 'Conflict deaths in government controlled areas may be more under-reported than those in opposition-controlled areas.' 'Access to bodies and access to the communications resources needed to report such casualties may be easier in urban areas compared to rural areas.' 'In this dataset the numbers of executions of civilians reported in government-controlled areas was extremely low...Underreporting of executions may be even higher in this area than for other deaths from weapons due to the highly controlled and secretive environment of executions...' In other words, reporting rates for weapon types may be correlated with geography generally and territorial control specifically. Curiously, though, the authors present odds ratios in this paper without reflection about how they may be distorted by reporting issues. This is a serious methodological problem.

We agree that more attention should be drawn to the use of indiscriminate weapons and the possibility that already vulnerable populations may be disproportionately affected by such weapons. However, the analyses presented here rest on too many unexamined assumptions, uncontrolled biases, and inadequate data. Analyses such as the ones presented in this paper have the potential to influence the accountability efforts that will likely take place in the future. Without appropriate consideration of their potential limitations, they are more likely to mislead than to inform about important questions regarding culpability, reparation, reconciliation, and ultimately justice.

[1] https://www.washingtonpost.com/world/syria-extremists-retreat-from-key-t...
[2] Salama, Hana, and Hamit Dardagan. "Stolen futures: the hidden toll of child casualties in Syria." (2013).
[3] Price, Megan, Anita Gohdes, and Patrick Ball. "Documents of war: Understanding the Syrian conflict." Significance 12.2 (2015): 14-19.

Megan Price, Human Rights Data Analysis Group
Anita Gohdes , Human Rights Data Analysis Group/Harvard Kennedy School
Jay D. Aronson, Carnegie Mellon University, Center for Human Rights Science
Christopher McNaboe, Carter Center, Syria Conflict
Mapping Project

Why is this article trying to hide the actual numbers and only showing relative risks? By using relative risk given that the subject is already dead, you do not take into account the total number of deaths.

"[We] found that the likelihoods of death were significantly higher for children than for men for air bombardments, shells, ground level explosives, and chemical weapons"

The authors make it sound as if more children die from these causes than men. Your actual data are children vs men for air bombardments 3945 vs 8675, for shells 5598 vs 16747, for explosives 464 vs 1750 and for chemical weapons 128 vs 544 (from Appendix B of the article).

"Women were significantly more likely to die from chemical intoxication (4.72 (3.93 to 5.67)) than men, as were children (2.11, 1.69 to 2.63). Our findings indicate that the use of chemical weapons in NSAG controlled areas was particularly harmful to women and children"

Again, from your own numbers: 128 children, 297 women and 544 men. How is that "particularly harmful to women and children"?

Imagine a hypothetical scenario with 1 child and 10 men dead. If the child and 5 of the men had been stabbed, the other 5 men died from other causes. Is the risk of getting stabbed really twice as high if you are a child?

Martin