Making AI Compliance Evidence Machine-Readable

TL;DR

This paper introduces OSCAL as a standardized, machine-readable format for AI compliance evidence, enabling automated assurance aligned with various governance frameworks.

Contribution

It proposes extending OSCAL for AI governance, defines a three-layer architecture, and provides an open-source SDK for generating compliance evidence during model training.

Findings

Validated approach on credit scoring and medical imaging systems.

Open-source SDK produces compliant, machine-readable assurance evidence.

Extended OSCAL covers lifecycle, enforcement, and risk traceability.

Abstract

AI Assurance -- producing the machine-readable evidence required to demonstrate compliance with AI governance frameworks -- has mature policy scaffolding but lacks the infrastructure to operationalize it. Organizations building high-risk AI systems under the EU AI Act face a gap: frameworks such as the EU AI Act, ISO/IEC 42001, and NIST AI RMF specify what to assure but provide no executable format for how. This paper proposes OSCAL -- the NIST standard adopted for FedRAMP cybersecurity compliance -- as a candidate interchange format for AI governance, complementing rather than replacing the emerging JTC21 standards stack. We define 16 property extensions covering lifecycle phases, enforcement semantics, risk traceability, and risk-acceptance justification, and present a three-layer Compliance-as-Code architecture (policy, evidence, enforcement) that generates assurance evidence as a byproduct of model training. The SDK produces native OSCAL Assessment Results validated against the NIST JSON schema. We test the approach on two Annex III high-risk systems: a credit scoring model and a medical imaging segmentation system. The architecture and reference implementation are open-source under Apache 2.0.