Amid the bewildering array of brutality in Ukraine, military ethicists have been keeping a close eye on whether the war could also become a proving ground for drones that use artificial intelligence, or AI, to decide whom to hurt.
Early in the war, Moscow was rumored to be employing “kamikaze” drones as “hunter-killer robots.” The Russian company that created the weapon boasted of its AI skills – the kind that could potentially enable a machine rather than a human to choose its targets. But the consensus among defense analysts has been that these claims were more marketing hype than credible capability.
Why We Wrote This
At some point, militaries will likely allow artificial intelligence to decide when to pull the trigger – and on whom. Ukraine is showing just how close that moment might be.
Yet there’s no doubt that the demand for AI in drones has been voracious and growing. And if humans for now are pulling the trigger, so to speak, “I don’t think that will last over time,” says Paul Scharre, an expert who formerly worked on autonomous systems policy at the Pentagon.
The question of whether a weapon is ethical is answered in large part, adds security expert Gregory Allen, on whether it’s in the hands of a military “that has any intention of behaving ethically.”
Amid the bewildering array of brutality on and off the battlefields of Ukraine, military ethicists have been keeping a close eye on whether the war could also become a proving ground for drones that use artificial intelligence to decide whom to hurt.
Early in the war, Moscow was rumored to be employing “kamikaze” drones as “hunter-killer robots.” Though the Russian company that created the weapon boasted of its AI skills – the kind that could potentially enable a machine rather than a human to choose its targets – the consensus among defense analysts has been that these claims were more marketing hype than credible capability.
Yet there’s no doubt that the demand for AI in drones has been voracious and growing. The drones on display in Ukraine all have a human pulling the trigger, so to speak – for now. “I don’t think we see any significant evidence that AI or machine learning is being employed in Ukraine in any significant way for the time being,” says Paul Scharre, who previously worked on autonomous systems policy at the Pentagon.
Why We Wrote This
At some point, militaries will likely allow artificial intelligence to decide when to pull the trigger – and on whom. Ukraine is showing just how close that moment might be.
“But I don’t think that will last over time,” he adds.
That’s because before the war, drones were seen as a useful counterterrorism tool against adversaries without air power, but not as particularly effective against big state actors who could easily shoot them down. The current conflict is proving otherwise.
Wars have a way, too, of driving technological leaps. This one could teach combatants – and interested observers – lessons that bring the world closer to AI making “kill” decisions on the battlefield, analysts say. “I think of the Ukrainian war as almost a gateway drug to that,” says Joshua Schwartz, a Grand Strategy, Security, and Statecraft Fellow at Harvard University.
“Scary” – but so is all warIn March 2021, a great fear among AI ethicists was realized: A United Nations panel of experts warned that Turkey had deployed a weapon to Libya – the Kargu-2 quadcopter drone – that could hunt down retreating troops and kill them without “data connectivity” between the human operator and the weapon.
Along with the flood of commentary decrying the use of so-called terminator weapons, a report from the U.S. Military Academy at West Point was more circumspect. It argued that ethical debate surrounding the Kargu-2 should concentrate not on whether it had killed autonomously, but whether in doing so it had complied with laws of armed conflict.
“The focus of humanitarian concerns should be the drone’s ability to distinguish legitimate military targets from protected civilians and to direct its attacks against the former in a discriminate manner,” wrote the paper’s author, Hitoshi Nasu, a professor of law at West Point.
Unlike the first iteration of autonomous weapons – land mines, for example – today’s AI systems are typically designed to avoid civilian casualties. For this reason, people should be more inclined to “embrace them,” Mr. Nasu posited in an interview with the Monitor.
But many critics can’t shake sci-fi-fueled biases, he adds.
“I remember someone writing on this topic, ‘It must be very scary for someone to be targeted by an autonomous system,’” Mr. Nasu says. “Well, warfighting is very scary – whether it’s done by autonomous systems or human beings.”
Critics point out, however, that such a scenario might be especially alarming for soldiers trying in vain to, say, surrender to a machine. This requires “granting quarter” – showing mercy to an enemy laying down arms – and it’s unclear whether the Kargu-2, in the face of “technical challenges,” is capable of it, Mr. Nasu wrote in his paper.
“Abandoning a weapon can be a machine-detectable surrender event,” but precisely what happens when combatants who “express an intention to surrender” are, due to injury, “unable to jettison their weapons,” he acknowledges, is unclear.
Fragments of an explosive device are displayed in Zagreb, Croatia, April 13, 2022. A military drone that apparently flew all the way from the Ukrainian war zone over three European NATO-member states before crashing in an urban zone of the Croatian capital was armed with an explosive device, Croatia’s crash investigators said.
A fast-advancing technologyThe AI that gives these drones the ability to hunt human targets involves “machine learning” technology that has advanced considerably in the past decade.
It relies on an enormous digital library of pictures on which the machine can draw. “If you have images of enemy tanks, but only in certain lighting conditions, or only when they’re not partially obscured by vegetation, or without people crawling on them, then the machine learning the lessons might not recognize the target and make mistakes,” says Mr. Scharre, now vice president of the Center for a New American Security.
Drones with the AI to effectively employ such machine learning, however, are still “5 to 10 years” away, he says. After that, “we’ll see machine learning embedded for target recognition that could open doors to AI making its own targeting decisions.”
The fact that the technology is for now unreliable is not the only reason it is generally unattractive to military professionals.
Good commanders tend to seek the ability to “carefully calibrate the tempo and severity of a conflict,” says Zachary Kallenborn, research affiliate with the National Consortium for the Study of Terrorism and Responses to Terrorism.
This in turn allows them “to achieve an objective and then stand down – like punishing an adversary for taking an action you don’t like but not creating a global war.”
Military leaders are also wary of widespread use of AI in drones because of their potential to change what’s known in Pentagon parlance as “targeting incentives” on the battlefield.
This involves scaring or cowing an enemy enough to get them to do things like surrender – or at least run away.
Instead of pitting robots against other robots, strategists might decide to destroy an area where the humans waging war will actually suffer harm. This includes “cities, or anywhere they can really do damage,” Mr. Kallenborn adds. “I can see warfare escalating very quickly there.”
Officials in China, America’s greatest AI competitor, have expressed anxiety about precisely this, Gregory Allen, former director of strategy and policy at the Pentagon’s Joint Artificial Intelligence Center, wrote in a paper on the topic. “One official told me he was concerned that AI will lower the threshold of military action, because states may be more willing to attack each other with AI military systems due to the lack of casualty risk.”
Influential Chinese think tanks have echoed these warnings but, as in the U.S., it has not stopped governments who don’t want to be left behind in an AI arms race from pursuing the technology.
Risk for civilians: New face of old problemStill, one of the takeaways in the ongoing tragedy of Ukraine is “how much it’s been a 20th-century war, primarily dominated by mud and steel rather than whiz-bang new technology,” Mr. Scharre says, raising the possibility, he argues, that “this fixation on fancy weapons as the ethical problem is mistaken.”
Mr. Allen, now director of the Project on AI Governance at the Center for Strategic and International Studies, concurs. “For a long time the autonomous weapons debate has been heavily focused on whether or not it increases the risk of technical accidents,” including killing civilians, he says in an interview. “But the war in Ukraine is a great reminder that while unintentional harm to civilians is a real tragedy, there is also the unsolved problem of intentional harm to civilians.”
That’s a challenge that spans from old technology to new.
The conflict has made it increasingly clear that the question of whether a weapon is ethical is answered in large part, Mr. Allen adds, on whether it’s in the hands of a military – or a country – “that has any intention of behaving ethically.”
