Coordinated Flaw Disclosure for AI: Beyond Security Vulnerabilities

TL;DR

This paper proposes a structured Coordinated Flaw Disclosure framework for AI, inspired by cybersecurity practices, to improve transparency, accountability, and trust in AI systems by addressing algorithmic flaws systematically.

Contribution

It introduces a novel CFD framework with features like extended model cards, dynamic scope, adjudication, and automation, tailored for AI and ML transparency challenges.

Findings

Review of ML disclosure evolution and practices

Proposed CFD framework with innovative features

Outline of a real-world CFD pilot implementation

Abstract

Harm reporting in Artificial Intelligence (AI) currently lacks a structured process for disclosing and addressing algorithmic flaws, relying largely on an ad-hoc approach. This contrasts sharply with the well-established Coordinated Vulnerability Disclosure (CVD) ecosystem in software security. While global efforts to establish frameworks for AI transparency and collaboration are underway, the unique challenges presented by machine learning (ML) models demand a specialized approach. To address this gap, we propose implementing a Coordinated Flaw Disclosure (CFD) framework tailored to the complexities of ML and AI issues. This paper reviews the evolution of ML disclosure practices, from ad hoc reporting to emerging participatory auditing methods, and compares them with cybersecurity norms. Our framework introduces innovations such as extended model cards, dynamic scope expansion, an independent adjudication panel, and an automated verification process. We also outline a forthcoming real-world pilot of CFD. We argue that CFD could significantly enhance public trust in AI systems. By balancing organizational and community interests, CFD aims to improve AI accountability in a rapidly evolving technological landscape.