Technical Risks of (Lethal) Autonomous Weapons Systems

TL;DR

This paper discusses the significant technological risks associated with lethal autonomous weapons systems, highlighting issues of unpredictability, lack of transparency, and potential for uncontrollable behaviors that threaten international security.

Contribution

It identifies and analyzes key systemic risks of (L)AWS, emphasizing their unpredictability and potential to cause unintended, uncontrollable consequences.

Findings

AI decision-making is often a black box, limiting transparency.

Systematic risks include reward hacking and goal misgeneralization.

Unpredictable behaviors can undermine stability and escalate conflicts.

Abstract

The autonomy and adaptability of (Lethal) Autonomous Weapons Systems, (L)AWS in short, promise unprecedented operational capabilities, but they also introduce profound risks that challenge the principles of control, accountability, and stability in international security. This report outlines the key technological risks associated with (L)AWS deployment, emphasizing their unpredictability, lack of transparency, and operational unreliability, which can lead to severe unintended consequences. Key Takeaways: 1. Proposed advantages of (L)AWS can only be achieved through objectification and classification, but a range of systematic risks limit the reliability and predictability of classifying algorithms. 2. These systematic risks include the black-box nature of AI decision-making, susceptibility to reward hacking, goal misgeneralization and potential for emergent behaviors that escape human control. 3. (L)AWS could act in ways that are not just unexpected but also uncontrollable, undermining mission objectives and potentially escalating conflicts. 4. Even rigorously tested systems may behave unpredictably and harmfully in real-world conditions, jeopardizing both strategic stability and humanitarian principles.