A taxonomic system for failure cause analysis of open source AI incidents

TL;DR

This paper introduces a taxonomic framework for analyzing open source AI incidents, enabling systematic cause analysis despite limited incident transparency, by combining expert knowledge with incident data.

Contribution

It develops a novel taxonomic system and classification workflow for identifying causes of AI incidents using expert insights and community feedback.

Findings

Taxonomy covers incident factors from goals to technical causes

Framework enables structured incident cause analysis in open source AI

Combines expert knowledge with community feedback for annotations

Abstract

While certain industrial sectors (e.g., aviation) have a long history of mandatory incident reporting complete with analytical findings, the practice of artificial intelligence (AI) safety benefits from no such mandate and thus analyses must be performed on publicly known ``open source'' AI incidents. Although the exact causes of AI incidents are seldom known by outsiders, this work demonstrates how to apply expert knowledge on the population of incidents in the AI Incident Database (AIID) to infer the potential and likely technical causative factors that contribute to reported failures and harms. We present early work on a taxonomic system that covers a cascade of interrelated incident factors, from system goals (nearly always known) to methods / technologies (knowable in many cases) and technical failure causes (subject to expert analysis) of the implicated systems. We pair this ontology structure with a comprehensive classification workflow that leverages expert knowledge and community feedback, resulting in taxonomic annotations grounded by incident data and human expertise.