Comparison requires valid measurement: Rethinking attack success rate comparisons in AI red teaming

TL;DR

This paper critically examines how attack success rates in AI red teaming are often misinterpreted due to invalid measurement practices, emphasizing the importance of valid comparisons grounded in measurement theory.

Contribution

It introduces a conceptual framework based on social science measurement theory to determine when attack success rates are meaningfully comparable in AI security evaluations.

Findings

Identifies common pitfalls in ASR comparisons

Provides conditions for valid attack success rate comparisons

Highlights measurement validity challenges in AI red teaming

Abstract

We argue that conclusions drawn about relative system safety or attack method efficacy via AI red teaming are often not supported by evidence provided by attack success rate (ASR) comparisons. We show, through conceptual, theoretical, and empirical contributions, that many conclusions are founded on apples-to-oranges comparisons or low-validity measurements. Our arguments are grounded in asking a simple question: When can attack success rates be meaningfully compared? To answer this question, we draw on ideas from social science measurement theory and inferential statistics, which, taken together, provide a conceptual grounding for understanding when numerical values obtained through the quantification of system attributes can be meaningfully compared. Through this lens, we articulate conditions under which ASRs can and cannot be meaningfully compared. Using jailbreaking as a running example, we provide examples and extensive discussion of apples-to-oranges ASR comparisons and measurement validity challenges.