GF-Score: Certified Class-Conditional Robustness Evaluation with Fairness Guarantees

TL;DR

GF-Score provides a detailed, class-wise evaluation of neural network robustness with fairness guarantees, eliminating the need for adversarial attacks and revealing class-specific vulnerabilities.

Contribution

The paper introduces GF-Score, a novel attack-free framework for decomposing and quantifying class-level robustness disparities in neural networks.

Findings

Per-class robustness scores reveal consistent vulnerability patterns.

More robust models tend to have greater class disparity.

The framework is exact and applicable to models on CIFAR-10 and ImageNet.

Abstract

Adversarial robustness is essential for deploying neural networks in safety-critical applications, yet standard evaluation methods either require expensive adversarial attacks or report only a single aggregate score that obscures how robustness is distributed across classes. We introduce the \emph{GF-Score} (GREAT-Fairness Score), a framework that decomposes the certified GREAT Score into per-class robustness profiles and quantifies their disparity through four metrics grounded in welfare economics: the Robustness Disparity Index (RDI), the Normalized Robustness Gini Coefficient (NRGC), Worst-Case Class Robustness (WCR), and a Fairness-Penalized GREAT Score (FP-GREAT). The framework further eliminates the original method's dependence on adversarial attacks through a self-calibration procedure that tunes the temperature parameter using only clean accuracy correlations. Evaluating 22 models from RobustBench across CIFAR-10 and ImageNet, we find that the decomposition is exact, that per-class scores reveal consistent vulnerability patterns (e.g., ``cat'' is the weakest class in 76\% of CIFAR-10 models), and that more robust models tend to exhibit greater class-level disparity. These results establish a practical, attack-free auditing pipeline for diagnosing where certified robustness guarantees fail to protect all classes equally. We release our code on \href{https://github.com/aryashah2k/gf-score}{GitHub}.