Lost in Vagueness: Towards Context-Sensitive Standards for Robustness Assessment under the EU AI Act

TL;DR

This paper emphasizes the importance of context-sensitive standards for assessing AI robustness under the EU AI Act, proposing a multi-layered framework to improve clarity, adaptability, and operational relevance.

Contribution

It introduces a novel multi-layered standardisation framework that incorporates context-specific factors and dynamic repositories to enhance robustness assessment practices.

Findings

Robustness depends on use case, data, and model context.

Current standards lack detailed guidance for context-specific robustness evaluation.

A proposed framework enables adaptable and meaningful robustness assessments.

Abstract

Robustness is a key requirement for high-risk AI systems under the EU Artificial Intelligence Act (AI Act). However, both its definition and assessment methods remain underspecified, leaving providers with little concrete direction on how to demonstrate compliance. This stems from the Act's horizontal approach, which establishes general obligations applicable across all AI systems, but leaves the task of providing technical guidance to harmonised standards. This paper investigates what it means for AI systems to be robust and illustrates the need for context-sensitive standardisation. We argue that robustness is not a fixed property of a system, but depends on which aspects of performance are expected to remain stable ("robustness of what"), the perturbations the system must withstand ("robustness to what") and the operational environment. We identify three contextual drivers--use case, data and model--that shape the relevant perturbations and influence the choice of tests, metrics and benchmarks used to evaluate robustness. The need to provide at least a range of technical options that providers can assess and implement in light of the system's purpose is explicitly recognised by the standardisation request for the AI Act, but planned standards, still focused on horizontal coverage, do not yet offer this level of detail. Building on this, we propose a context-sensitive multi-layered standardisation framework where horizontal standards set common principles and terminology, while domain-specific ones identify risks across the AI lifecycle and guide appropriate practices, organised in a dynamic repository where providers can propose new informative methods and share lessons learned. Such a system reduces the interpretative burden, mitigates arbitrariness and addresses the obsolescence of static standards, ensuring that robustness assessment is both adaptable and operationally meaningful.