Parallel Coordinate Descent for L1-Regularized Loss Minimization

TL;DR

This paper introduces Shotgun, a parallel coordinate descent algorithm for L1-regularized loss minimization, demonstrating its theoretical convergence and superior scalability in empirical tests for large-scale sparse models.

Contribution

The paper presents a novel parallel coordinate descent algorithm, Shotgun, with proven convergence bounds and empirical validation showing improved scalability over existing solvers.

Findings

Shotgun achieves near-linear speedups in practice.

Theoretical convergence bounds match empirical performance.

Outperforms existing solvers on large-scale L1-regularized problems.

Abstract

We propose Shotgun, a parallel coordinate descent algorithm for minimizing L1-regularized losses. Though coordinate descent seems inherently sequential, we prove convergence bounds for Shotgun which predict linear speedups, up to a problem-dependent limit. We present a comprehensive empirical study of Shotgun for Lasso and sparse logistic regression. Our theoretical predictions on the potential for parallelism closely match behavior on real data. Shotgun outperforms other published solvers on a range of large problems, proving to be one of the most scalable algorithms for L1.