Bench-2-CoP: Can We Trust Benchmarking for EU AI Compliance?

TL;DR

This paper introduces Bench-2-CoP, a framework that systematically assesses how well current AI benchmarks cover the capabilities relevant to EU AI regulation, revealing significant gaps in risk-related evaluation areas.

Contribution

It provides the first quantitative analysis of benchmark coverage against EU AI Act capabilities, highlighting critical gaps and proposing a systematic evaluation approach.

Findings

Benchmarks focus mainly on hallucination and reliability, neglecting key functional capabilities.

Capabilities related to loss-of-control scenarios are entirely absent from benchmarks.

Current benchmarks are insufficient for comprehensive regulatory risk assessment.

Abstract

The rapid advancement of General Purpose AI (GPAI) models necessitates robust evaluation frameworks, especially with emerging regulations like the EU AI Act and its associated Code of Practice (CoP). Current AI evaluation practices depend heavily on established benchmarks, but these tools were not designed to measure the systemic risks that are the focus of the new regulatory landscape. This research addresses the urgent need to quantify this "benchmark-regulation gap." We introduce Bench-2-CoP, a novel, systematic framework that uses validated LLM-as-judge analysis to map the coverage of 194,955 questions from widely-used benchmarks against the EU AI Act's taxonomy of model capabilities and propensities. Our findings reveal a profound misalignment: the evaluation ecosystem dedicates the vast majority of its focus to a narrow set of behavioral propensities. On average, benchmarks devote 61.6% of their regulatory-relevant questions to "Tendency to hallucinate" and 31.2% to "Lack of performance reliability", while critical functional capabilities are dangerously neglected. Crucially, capabilities central to loss-of-control scenarios, including evading human oversight, self-replication, and autonomous AI development, receive zero coverage in the entire benchmark corpus. This study provides the first comprehensive, quantitative analysis of this gap, demonstrating that current public benchmarks are insufficient, on their own, for providing the evidence of comprehensive risk assessment required for regulatory compliance and offering critical insights for the development of next-generation evaluation tools.