Learning Representations Robust to Group Shifts and Adversarial Examples

TL;DR

This paper introduces a novel algorithm combining adversarial training and group distribution robust optimization to enhance the robustness of deep neural networks against input perturbations and distribution shifts, achieving superior results on benchmark datasets.

Contribution

It proposes a new method that integrates adversarial training with group distribution robust optimization for improved robustness in neural networks.

Findings

Outperforms existing methods on robust metrics

Maintains high standard accuracy

Effective across multiple image benchmark datasets

Abstract

Despite the high performance achieved by deep neural networks on various tasks, extensive studies have demonstrated that small tweaks in the input could fail the model predictions. This issue of deep neural networks has led to a number of methods to improve model robustness, including adversarial training and distributionally robust optimization. Though both of these two methods are geared towards learning robust models, they have essentially different motivations: adversarial training attempts to train deep neural networks against perturbations, while distributional robust optimization aims at improving model performance on the most difficult "uncertain distributions". In this work, we propose an algorithm that combines adversarial training and group distribution robust optimization to improve robust representation learning. Experiments on three image benchmark datasets illustrate that the proposed method achieves superior results on robust metrics without sacrificing much of the standard measures.