AdAUC: End-to-end Adversarial AUC Optimization Against Long-tail Problems

TL;DR

This paper introduces AdAUC, an end-to-end adversarial training method that optimizes AUC for long-tail datasets, addressing class imbalance and coupling issues with a saddle point reformulation and convergence guarantees.

Contribution

It proposes a novel adversarial AUC optimization framework for long-tail data, reformulating the problem as a saddle point and providing an end-to-end training algorithm with convergence analysis.

Findings

Improves AUC performance on long-tail datasets

Enhances robustness against adversarial examples in imbalanced settings

Demonstrates effectiveness through extensive experiments

Abstract

It is well-known that deep learning models are vulnerable to adversarial examples. Existing studies of adversarial training have made great progress against this challenge. As a typical trait, they often assume that the class distribution is overall balanced. However, long-tail datasets are ubiquitous in a wide spectrum of applications, where the amount of head class instances is larger than the tail classes. Under such a scenario, AUC is a much more reasonable metric than accuracy since it is insensitive toward class distribution. Motivated by this, we present an early trial to explore adversarial training methods to optimize AUC. The main challenge lies in that the positive and negative examples are tightly coupled in the objective function. As a direct result, one cannot generate adversarial examples without a full scan of the dataset. To address this issue, based on a concavity regularization scheme, we reformulate the AUC optimization problem as a saddle point problem, where the objective becomes an instance-wise function. This leads to an end-to-end training protocol. Furthermore, we provide a convergence guarantee of the proposed algorithm. Our analysis differs from the existing studies since the algorithm is asked to generate adversarial examples by calculating the gradient of a min-max problem. Finally, the extensive experimental results show the performance and robustness of our algorithm in three long-tail datasets.