Assessing confidence in frontier AI safety cases

TL;DR

This paper explores how to quantify and communicate confidence in safety cases for frontier AI, proposing methods to improve transparency, reproducibility, and decision-making support amidst the challenges of probabilistic assessment.

Contribution

It introduces a novel LLM-based Delphi method for assessing confidence in AI safety arguments and offers strategies to enhance safety case evaluation and communication.

Findings

Numerical confidence quantification is challenging.

The LLM-based Delphi method improves transparency and reproducibility.

Prioritization strategies help focus investigations on argument defeaters.

Abstract

Powerful new frontier AI technologies are bringing many benefits to society but at the same time bring new risks. AI developers and regulators are therefore seeking ways to assure the safety of such systems, and one promising method under consideration is the use of safety cases. A safety case presents a structured argument in support of a top-level claim about a safety property of the system. Such top-level claims are often presented as a binary statement, for example "Deploying the AI system does not pose unacceptable risk". However, in practice, it is often not possible to make such statements unequivocally. This raises the question of what level of confidence should be associated with a top-level claim. We adopt the Assurance 2.0 safety assurance methodology, and we ground our work by specific application of this methodology to a frontier AI inability argument that addresses the harm of cyber misuse. We find that numerical quantification of confidence is challenging, though the processes associated with generating such estimates can lead to improvements in the safety case. We introduce a method for better enabling reproducibility and transparency in probabilistic assessment of confidence in argument leaf nodes through a purely LLM-implemented Delphi method. We propose a method by which AI developers can prioritise, and thereby make their investigation of argument defeaters more efficient. Proposals are also made on how best to communicate confidence information to executive decision-makers.