A Structured Approach to Safety Case Construction for AI Systems

TL;DR

This paper presents a systematic, adaptable framework for constructing safety cases for AI systems, addressing their unique challenges like unpredictability and evolving behaviors, to improve safety governance.

Contribution

It introduces comprehensive taxonomies and a reusable safety-case template specifically designed for AI systems, overcoming limitations of traditional safety practices.

Findings

Developed AI-specific claim and argument taxonomies

Created a reusable safety-case template for AI systems

Demonstrated patterns for addressing evaluation and risk in AI safety

Abstract

Safety cases, structured arguments that a system is acceptably safe, are becoming central to the governance of AI systems. Yet, traditional safety-case practices from aviation or nuclear engineering rely on well-specified system boundaries, stable architectures, and known failure modes. Modern AI systems, such as generative and agentic AI, are the opposite. Their capabilities emerge unpredictably from low-level training objectives, their behaviour varies with prompts, and their risk profiles shift through fine-tuning, scaffolding, or deployment context. This study examines how safety cases are currently constructed for AI systems and why classical approaches fail to capture these dynamics. This study introduces comprehensive taxonomies for AI-specific claim types (assertion-based, constraint-based, capability-based), argument types (demonstrative, comparative, causal/explanatory, risk-based, and normative), and evidence families (empirical, mechanistic, comparative, expert-driven, formal methods, operational/field data, and model-based). It then proposes a reusable safety-case template, each of which follows a predefined structure of claims, arguments, and evidence tailored for AI systems. Each template is illustrated by end-to-end patterns that address distinctive challenges, such as evaluation without ground truth, dynamic model updates, and threshold-based risk decisions. The result is a systematic, composable, and reusable approach to constructing and maintaining safety cases that are credible, auditable, and adaptive to the evolving behaviour of generative and frontier AI systems.