Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database

TL;DR

The paper introduces the AI Incident Database, a collection of real-world AI failures, aiming to prevent repeated mistakes by providing a shared repository for incident analysis and safety improvements.

Contribution

It presents a structured incident database for AI failures, enabling collective learning and safety enhancements in AI system development.

Findings

Over 1,000 incident reports archived

Supports faceted and full-text search

Facilitates AI safety research and mitigation

Abstract

Mature industrial sectors (e.g., aviation) collect their real world failures in incident databases to inform safety improvements. Intelligent systems currently cause real world harms without a collective memory of their failings. As a result, companies repeatedly make the same mistakes in the design, development, and deployment of intelligent systems. A collection of intelligent system failures experienced in the real world (i.e., incidents) is needed to ensure intelligent systems benefit people and society. The AI Incident Database is an incident collection initiated by an industrial/non-profit cooperative to enable AI incident avoidance and mitigation. The database supports a variety of research and development use cases with faceted and full text search on more than 1,000 incident reports archived to date.