Intended for healthcare professionals

Editorials

Lethal autonomous weapons

BMJ 2019; 364 doi: https://doi.org/10.1136/bmj.l1171 (Published 25 March 2019) Cite this as: BMJ 2019;364:l1171
  1. Emilia Javorsky, director1,
  2. Max Tegmark, professor2,
  3. Ira Helfand, co-president3
  1. 1Scientists Against Inhumane Weapons, Watertown, MA, USA
  2. 2Department of Physics & Center for Brains, Minds and Machines, Massachusetts Institute of Technology, Cambridge, MA, USA
  3. 3International Physicians for the Prevention of Nuclear War, Malden, MA, USA
  1. Correspondence to: E Javorsky ejavorsky{at}saiwe.org

It’s not too late to stop this new and potentially catastrophic force

Advances in artificial intelligence are creating the potential to develop fully autonomous lethal weapons.1 These weapons would remove all human control over the use of deadly force. The medical community has a long history of advocacy against the development of lethal weapons, and the World and American Medical Associations both advocate total bans on nuclear, chemical, and biological weapons.2 But while some nations and non-governmental organisations have called for a legally binding ban on these new weapons,34 the medical community has been conspicuously absent from this discourse.

Third revolution in warfare

Several countries are conducting research to develop lethal autonomous weapons. Many commentators have argued that the development of lethal autonomous weapon systems for military use would represent a third revolution in warfare, after the invention of gunpowder and nuclear weapons.5 Although semi-autonomous weapons, such as unmanned drones, are in widespread use, they require human oversight, control, and decision making to ensure, at least in theory, that targets are ethically and legally legitimate. In contrast, lethal autonomous weapon systems are defined as: “any system capable of targeting and initiating the use of potentially lethal force without direct human supervision and direct human involvement in lethal decision making.”6 In other words, they represent the complete automation of lethal harm.

Once developed, such weapons could be produced rapidly, cheaply, and at scale.7 Furthermore, lethality will only increase with use as the machine’s learning algorithms gain access to more data. Without human decision making capability, autonomous weapons have great potential to target civilians in error or malfunction in other ways with no clarity around responsibility and justifiability. These weapons could quickly become ubiquitous on black markets and readily accessible to groups acting outside international laws.

Professional voice

The practical, legal, and ethical ramifications of dehumanising lethal warfare, combined with a high risk of both unintentional and intentional misuse, have amplified calls for an international ban on lethal autonomous weapons and a requirement for meaningful human control of all weapons systems.35 Healthcare professionals must engage in this conversation.

The medical community had a key role in establishing the weapons bans already in place, including, most recently, the Treaty on the Prohibition of Nuclear Weapons.8 International Physicians for the Prevention of Nuclear War, working with the World Medical Association, the World Federation of Public Health Associations, the International Council of Nurses, and the International Committee of the Red Cross, was central in the dialogue about the humanitarian consequences of nuclear war that led to adoption of the treaty. The efficacy of advocacy by international healthcare organisations derives from its moral authority and credibility on the devastating consequences of warfare for human health.

In addition, the medical community has a unique understanding of the risks of automated decision making. The emergence of artificial intelligence applications in medicine has given us insight into the value of human decision makers who take full account of ambiguity, anomalies, data weaknesses, and context to avoid biases that harm patient care.9 To capture this concept, the American Medical Association coined the term “augmented” intelligence in its June 2018 inaugural policy on artificial intelligence.10 If the medical community maintains that humans cannot be replaced by an algorithm in the treatment of patients and prevention of harm, how can it be justifiable to replace human judgment in decisions to kill people, as autonomous weapons are designed to do?

It is not too late to prevent autonomous weapons. Decades of advocacy efforts have shown, however, that once created and in military use, entire classes of weapons are extremely difficult to eliminate: the threat of nuclear war is growing despite the non-proliferation treaty. After they are developed for military use, autonomous weapons could easily spread beyond the battlefield to law enforcement agencies or criminals.

Preventing harm is a key principle of all medical endeavour and an essential area of expertise for all healthcare professionals. The medical community has a history of successful advocacy for weapons bans, is well equipped to detail the humanitarian effects of weapon use, understands the dangers associated with automation, and is experienced in promoting prevention. As we continue to work towards the elimination of nuclear weapons, we must also support efforts to publicise the potentially catastrophic humanitarian consequences of autonomous weapons and help ensure that the full automation of lethal harm is prevented for ever.

Footnotes

  • Competing interests: We have read and understood BMJ policy on declaration of interests and have no relevant interests to declare.

  • Provenance and peer review: Not commissioned; externally peer reviewed.

References

View Abstract