Towards Fair Class-wise Robustness: Class Optimal Distribution Adversarial Training

TL;DR

This paper introduces CODAT, a novel adversarial training framework that optimizes class-wise robustness fairly by leveraging distributionally robust optimization with theoretical guarantees, improving robustness and fairness.

Contribution

It proposes a min-max training framework with a closed-form solution for class-wise weight optimization, ensuring consistent joint optimization of weights and model parameters.

Findings

Improves robust fairness across classes in neural networks.

Outperforms state-of-the-art methods in robustness and fairness metrics.

Provides theoretical guarantees for class weight optimization.

Abstract

Adversarial training has proven to be a highly effective method for improving the robustness of deep neural networks against adversarial attacks. Nonetheless, it has been observed to exhibit a limitation in terms of robust fairness, characterized by a significant disparity in robustness across different classes. Recent efforts to mitigate this problem have turned to class-wise reweighted methods. However, these methods suffer from a lack of rigorous theoretical analysis and are limited in their exploration of the weight space, as they mainly rely on existing heuristic algorithms or intuition to compute weights. In addition, these methods fail to guarantee the consistency of the optimization direction due to the decoupled optimization of weights and the model parameters. They potentially lead to suboptimal weight assignments and consequently, a suboptimal model. To address these problems, this paper proposes a novel min-max training framework, Class Optimal Distribution Adversarial Training (CODAT), which employs distributionally robust optimization to fully explore the class-wise weight space, thus enabling the identification of the optimal weight with theoretical guarantees. Furthermore, we derive a closed-form optimal solution to the internal maximization and then get a deterministic equivalent objective function, which provides a theoretical basis for the joint optimization of weights and model parameters. Meanwhile, we propose a fairness elasticity coefficient for the evaluation of the algorithm with regard to both robustness and robust fairness. Experimental results on various datasets show that the proposed method can effectively improve the robust fairness of the model and outperform the state-of-the-art approaches.