Attestable Audits: Verifiable AI Safety Benchmarks Using Trusted Execution Environments

TL;DR

This paper introduces Attestable Audits, a method using Trusted Execution Environments to provide verifiable and confidential AI safety benchmarks, addressing trust and privacy issues in AI model evaluation.

Contribution

It presents a novel approach to verifiable AI safety benchmarks that preserves confidentiality and trust using Trusted Execution Environments.

Findings

Prototype demonstrates feasibility on Llama-3.1 benchmarks.

Ensures verifiable interaction with AI models in untrusted settings.

Protects sensitive data during audits.

Abstract

Benchmarks are important measures to evaluate safety and compliance of AI models at scale. However, they typically do not offer verifiable results and lack confidentiality for model IP and benchmark datasets. We propose Attestable Audits, which run inside Trusted Execution Environments and enable users to verify interaction with a compliant AI model. Our work protects sensitive data even when model provider and auditor do not trust each other. This addresses verification challenges raised in recent AI governance frameworks. We build a prototype demonstrating feasibility on typical audit benchmarks against Llama-3.1.