Designing Incident Reporting Systems for Harms from General-Purpose AI

TL;DR

This paper develops a framework for designing AI incident reporting systems, emphasizing institutional considerations and analyzing case studies to guide policymakers and researchers.

Contribution

It introduces a comprehensive framework with seven dimensions for AI incident reporting system design, informed by case studies and policy analysis.

Findings

Seven key dimensions for incident reporting system design.

Insights from nine case studies across safety-critical industries.

Guidelines for policymakers on appropriate design choices.

Abstract

We introduce a conceptual framework and provide considerations for the institutional design of AI incident reporting systems, i.e., processes for collecting information about safety- and rights-related events caused by general-purpose AI. As general-purpose AI systems are increasingly adopted, they are causing more real-world harms and displaying the potential to cause significantly more dangerous incidents - events that did or could have caused harm to individuals, property, or the environment. Through a literature review, we develop a framework for understanding the institutional design of AI incident reporting systems, which includes seven dimensions: policy goal, actors submitting and receiving reports, type of incidents reported, level of risk materialization, enforcement of reporting, anonymity of reporters, and post-reporting actions. We then examine nine case studies of incident reporting in safety-critical industries to extract design considerations for AI incident reporting in the United States. We discuss, among other factors, differences in systems operated by regulatory vs. non-regulatory government agencies, near miss reporting, the roles of mandatory reporting thresholds and voluntary reporting channels, how to enable safety learning after reporting, sharing incident information, and clarifying legal frameworks for reporting. Our aim is to inform researchers and policymakers about when particular design choices might be more or less appropriate for AI incident reporting.