Legal Zero-Days: A Novel Risk Vector for Advanced AI Systems

TL;DR

This paper introduces 'Legal Zero-Days' as a new risk vector in AI safety, highlighting legal vulnerabilities that can cause societal disruption and proposing methods to evaluate AI's ability to identify such risks.

Contribution

It defines Legal Zero-Days, develops a risk model and evaluation methodology, and analyzes AI's potential to discover legal vulnerabilities impacting societal stability.

Findings

Current AI models do not reliably find impactful Legal Zero-Days

Legal Zero-Days can cause significant societal disruption

Future AI systems may develop capabilities to discover legal vulnerabilities

Abstract

We introduce the concept of "Legal Zero-Days" as a novel risk vector for advanced AI systems. Legal Zero-Days are previously undiscovered vulnerabilities in legal frameworks that, when exploited, can cause immediate and significant societal disruption without requiring litigation or other processes before impact. We present a risk model for identifying and evaluating these vulnerabilities, demonstrating their potential to bypass safeguards or impede government responses to AI incidents. Using the 2017 Australian dual citizenship crisis as a case study, we illustrate how seemingly minor legal oversights can lead to large-scale governance disruption. We develop a methodology for creating "legal puzzles" as evaluation instruments for assessing AI systems' capabilities to discover such vulnerabilities. Our findings suggest that while current AI models may not reliably find impactful Legal Zero-Days, future systems may develop this capability, presenting both risks and opportunities for improving legal robustness. This work contributes to the broader effort to identify and mitigate previously unrecognized risks from frontier AI systems.