Efficient Optimization Algorithms for Linear Adversarial Training

TL;DR

This paper introduces specialized optimization algorithms for linear adversarial training that significantly improve efficiency and scalability in large-scale regression and classification tasks.

Contribution

It develops tailored algorithms based on extended variable reformulations, outperforming generic solvers for large-scale linear adversarial training.

Findings

Algorithms demonstrate faster convergence in numerical experiments.

Methods enable scalable adversarial training for large datasets.

Reformulations improve computational efficiency over standard approaches.

Abstract

Adversarial training can be used to learn models that are robust against perturbations. For linear models, it can be formulated as a convex optimization problem. Compared to methods proposed in the context of deep learning, leveraging the optimization structure allows significantly faster convergence rates. Still, the use of generic convex solvers can be inefficient for large-scale problems. Here, we propose tailored optimization algorithms for the adversarial training of linear models, which render large-scale regression and classification problems more tractable. For regression problems, we propose a family of solvers based on iterative ridge regression and, for classification, a family of solvers based on projected gradient descent. The methods are based on extended variable reformulations of the original problem. We illustrate their efficiency in numerical examples.