High-dimensional (Group) Adversarial Training in Linear Regression

TL;DR

This paper analyzes the effectiveness of adversarial training in high-dimensional linear regression, demonstrating near-optimal convergence rates and improved bounds for group adversarial training under sparsity assumptions.

Contribution

It provides the first non-asymptotic analysis of adversarial training in high-dimensional linear regression, including both standard and group adversarial methods.

Findings

Convergence rate matches the minimax rate up to a logarithmic factor.

Group adversarial training yields better prediction bounds under group sparsity.

Theoretical guarantees are established for high-dimensional settings.

Abstract

Adversarial training can achieve robustness against adversarial perturbations and has been widely used in machine learning models. This paper delivers a non-asymptotic consistency analysis of the adversarial training procedure under $\ell_\infty$-perturbation in high-dimensional linear regression. It will be shown that the associated convergence rate of prediction error can achieve the minimax rate up to a logarithmic factor in the high-dimensional linear regression on the class of sparse parameters. Additionally, the group adversarial training procedure is analyzed. Compared with classic adversarial training, it will be proved that the group adversarial training procedure enjoys a better prediction error upper bound under certain group-sparsity patterns.