Trimmed Maximum Likelihood Estimation for Robust Learning in Generalized Linear Models

TL;DR

This paper demonstrates that a simple iterative trimmed maximum likelihood estimator is robust and near-optimal for learning generalized linear models under both label and covariate corruptions, extending its effectiveness to challenging adversarial settings.

Contribution

The paper proves the near-optimal robustness of the iterative trimmed MLE for generalized linear models under adversarial corruptions, including label and covariate corruptions.

Findings

Achieves minimax near-optimal risk under label corruptions

Extends robustness to label and covariate corruptions

Demonstrates effectiveness across Gaussian, Poisson, and Binomial regressions

Abstract

We study the problem of learning generalized linear models under adversarial corruptions. We analyze a classical heuristic called the iterative trimmed maximum likelihood estimator which is known to be effective against label corruptions in practice. Under label corruptions, we prove that this simple estimator achieves minimax near-optimal risk on a wide range of generalized linear models, including Gaussian regression, Poisson regression and Binomial regression. Finally, we extend the estimator to the more challenging setting of label and covariate corruptions and demonstrate its robustness and optimality in that setting as well.