Robust and Sparse Regression in GLM by Stochastic Optimization

TL;DR

This paper introduces a stochastic optimization method for robust and sparse generalized linear models using $ extgamma$-divergence, improving efficiency and robustness in high-dimensional data with outliers.

Contribution

It extends robust sparse linear regression to generalized linear models with a stochastic gradient approach, ensuring convergence and efficiency for large-scale problems.

Findings

The proposed method outperformed existing methods in numerical experiments.

It effectively handles heavy contamination and outliers in high-dimensional data.

The approach is applicable to linear, logistic, and Poisson regression with $L_1$ regularization.

Abstract

The generalized linear model (GLM) plays a key role in regression analyses. In high-dimensional data, the sparse GLM has been used but it is not robust against outliers. Recently, the robust methods have been proposed for the specific example of the sparse GLM. Among them, we focus on the robust and sparse linear regression based on the $\gamma$-divergence. The estimator of the $\gamma$-divergence has strong robustness under heavy contamination. In this paper, we extend the robust and sparse linear regression based on the $\gamma$-divergence to the robust and sparse GLM based on the $\gamma$-divergence with a stochastic optimization approach in order to obtain the estimate. We adopt the randomized stochastic projected gradient descent as a stochastic optimization approach and extend the established convergence property to the classical first-order necessary condition. By virtue of the stochastic optimization approach, we can efficiently estimate parameters for very large problems. Particularly, we show the linear regression, logistic regression and Poisson regression with $L_1$ regularization in detail as specific examples of robust and sparse GLM. In numerical experiments and real data analysis, the proposed method outperformed comparative methods.