AI Incident Monitoring through a Public Health Lens

TL;DR

This paper proposes a public health-inspired framework for monitoring AI incidents, enabling better risk assessment through incident phase analysis and expert collaboration, demonstrated via autonomous vehicles and deepfake case studies.

Contribution

It introduces a novel incident phase framework inspired by public health, facilitating risk measurement and decision-making in AI safety monitoring.

Findings

Autonomous vehicle incident data can be used to determine incident phases.

Expert panels can effectively classify incident emergence stages.

The framework supports future research in AI incident phase determination.

Abstract

Artificial intelligence systems are now deployed at scale across sectors, accompanied by a growing number of real-world incidents ranging from misinformation and cybercrime to autonomous-system failures. Databases of AI incidents index these events, but they cannot measure ``risk'' (i.e., a joint measure of likelihood and severity) without additional data regarding the prevalence of risk-associated systems and their incident reporting rates. As a result, policymakers, companies, and the general public lack a means to weigh the benefits of AI against their in-context risks. Inspired by public-health processes, which presume noisy and incomplete disease surveillance, we identify six phases of incident emergence. We demonstrate the framework through a detailed case study of autonomous vehicles, whose mandatory reporting requirements produces reliable incident-rate ground truth expressed in distance traveled. The case study shows that an informed panel of domain experts (e.g., self-driving experts) can combine their domain expertise, incident data, and a collection of statistical and visualization tools to arrive at incident phase determinations serving public needs. We further demonstrate the approach with a deepfake incident case study and chart a path for future research in incident phase determination.