A Random Block-Coordinate Douglas-Rachford Splitting Method with Low Computational Complexity for Binary Logistic Regression

TL;DR

This paper introduces a stochastic block-coordinate Douglas-Rachford splitting algorithm for sparse binary logistic regression, utilizing the Lambert W function for proximity operations, and demonstrates its efficiency over stochastic gradient methods.

Contribution

The paper presents a novel stochastic block-coordinate Douglas-Rachford splitting method tailored for sparse logistic regression, incorporating the Lambert W function for proximity calculations.

Findings

Efficient convergence demonstrated on standard datasets.

Outperforms stochastic gradient-like methods in experiments.

Utilizes the Lambert W function for logistic loss proximity operator.

Abstract

In this paper, we propose a new optimization algorithm for sparse logistic regression based on a stochastic version of the Douglas-Rachford splitting method. Our algorithm sweeps the training set by randomly selecting a mini-batch of data at each iteration, and it allows us to update the variables in a block coordinate manner. Our approach leverages the proximity operator of the logistic loss, which is expressed with the generalized Lambert W function. Experiments carried out on standard datasets demonstrate the efficiency of our approach w.r.t. stochastic gradient-like methods.