AuditMAI: Towards An Infrastructure for Continuous AI Auditing

TL;DR

AuditMAI proposes a blueprint infrastructure to enable continuous AI auditing, addressing current gaps in integration and automation for responsible AI system management.

Contribution

It introduces AuditMAI, a comprehensive blueprint infrastructure designed to facilitate continuous AI auditing based on literature and industrial use cases.

Findings

Defined AI auditability based on literature

Derived requirements from industrial use cases

Developed the AuditMAI blueprint

Abstract

Artificial Intelligence (AI) Auditability is a core requirement for achieving responsible AI system design. However, it is not yet a prominent design feature in current applications. Existing AI auditing tools typically lack integration features and remain as isolated approaches. This results in manual, high-effort, and mostly one-off AI audits, necessitating alternative methods. Inspired by other domains such as finance, continuous AI auditing is a promising direction to conduct regular assessments of AI systems. The issue remains, however, since the methods for continuous AI auditing are not mature yet at the moment. To address this gap, we propose the Auditability Method for AI (AuditMAI), which is intended as a blueprint for an infrastructure towards continuous AI auditing. For this purpose, we first clarified the definition of AI auditability based on literature. Secondly, we derived requirements from two industrial use cases for continuous AI auditing tool support. Finally, we developed AuditMAI and discussed its elements as a blueprint for a continuous AI auditability infrastructure.