CoDefeater: Using LLMs To Find Defeaters in Assurance Cases

TL;DR

This paper introduces CoDefeater, a method that uses large language models to automatically identify defeaters in assurance cases, helping safety analysts improve the robustness of safety-critical system arguments.

Contribution

The paper presents a novel automated approach leveraging LLMs to find defeaters in assurance cases, reducing reliance on expert judgment and enhancing safety analysis.

Findings

LLMs can efficiently find known defeaters.

LLMs can identify unforeseen feasible defeaters.

Support for safety analysts in improving assurance case completeness.

Abstract

Constructing assurance cases is a widely used, and sometimes required, process toward demonstrating that safety-critical systems will operate safely in their planned environment. To mitigate the risk of errors and missing edge cases, the concept of defeaters - arguments or evidence that challenge claims in an assurance case - has been introduced. Defeaters can provide timely detection of weaknesses in the arguments, prompting further investigation and timely mitigations. However, capturing defeaters relies on expert judgment, experience, and creativity and must be done iteratively due to evolving requirements and regulations. This paper proposes CoDefeater, an automated process to leverage large language models (LLMs) for finding defeaters. Initial results on two systems show that LLMs can efficiently find known and unforeseen feasible defeaters to support safety analysts in enhancing the completeness and confidence of assurance cases.