Tractable and Near-Optimal Adversarial Algorithms for Robust Estimation in Contaminated Gaussian Models

TL;DR

This paper introduces tractable adversarial algorithms with simple discriminators for robust estimation in contaminated Gaussian models, achieving near-optimal error rates and outperforming traditional methods.

Contribution

It develops computationally feasible adversarial algorithms with spline discriminators that attain minimax optimal rates in robust Gaussian estimation.

Findings

Algorithms achieve minimax or near-minimax rates depending on divergence and penalty.

Proposed methods outperform classic robust estimators and neural network-based GANs.

First demonstration of near-optimal error rates for linear discriminators under contamination.

Abstract

Consider the problem of simultaneous estimation of location and variance matrix under Huber's contaminated Gaussian model. First, we study minimum $f$-divergence estimation at the population level, corresponding to a generative adversarial method with a nonparametric discriminator and establish conditions on $f$-divergences which lead to robust estimation, similarly to robustness of minimum distance estimation. More importantly, we develop tractable adversarial algorithms with simple spline discriminators, which can be implemented via nested optimization such that the discriminator parameters can be fully updated by maximizing a concave objective function given the current generator. The proposed methods are shown to achieve minimax optimal rates or near-optimal rates depending on the $f$-divergence and the penalty used. This is the first time such near-optimal error rates are established for adversarial algorithms with linear discriminators under Huber's contamination model. We present simulation studies to demonstrate advantages of the proposed methods over classic robust estimators, pairwise methods, and a generative adversarial method with neural network discriminators.