Distributionally and Adversarially Robust Logistic Regression via Intersecting Wasserstein Balls

TL;DR

This paper introduces a new distributionally robust logistic regression method using intersecting Wasserstein balls, which improves adversarial robustness and reduces overfitting by leveraging auxiliary datasets.

Contribution

It proposes a novel intersecting Wasserstein ambiguity set for distributionally robust logistic regression, with tractable optimization and improved performance over benchmarks.

Findings

Outperforms benchmark methods on standard datasets.

Provides a tractable convex reformulation for the Wasserstein DR problem.

Demonstrates effectiveness of intersecting ambiguity sets in robustness enhancement.

Abstract

Adversarially robust optimization (ARO) has emerged as the *de facto* standard for training models that hedge against adversarial attacks in the test stage. While these models are robust against adversarial attacks, they tend to suffer severely from overfitting. To address this issue, some successful methods replace the empirical distribution in the training stage with alternatives including *(i)* a worst-case distribution residing in an ambiguity set, resulting in a distributionally robust (DR) counterpart of ARO; *(ii)* a mixture of the empirical distribution with a distribution induced by an auxiliary (*e.g.*, synthetic, external, out-of-domain) dataset. Inspired by the former, we study the Wasserstein DR counterpart of ARO for logistic regression and show it admits a tractable convex optimization reformulation. Adopting the latter setting, we revise the DR approach by intersecting its ambiguity set with another ambiguity set built using the auxiliary dataset, which offers a significant improvement whenever the Wasserstein distance between the data generating and auxiliary distributions can be estimated. We study the underlying optimization problem, develop efficient solution algorithms, and demonstrate that the proposed method outperforms benchmark approaches on standard datasets.