A fast algorithm for solving the lasso problem exactly without homotopy using differential inclusions

TL;DR

This paper introduces a novel, exact, and fast algorithm for solving the lasso problem using differential inclusions, eliminating the need for homotopy methods and outperforming existing algorithms in speed and accuracy.

Contribution

The work develops a new differential inclusions-based approach that transforms the dual lasso problem into an integrable projected dynamical system, enabling exact solutions without homotopy.

Findings

Algorithm computes solutions up to machine precision

Outperforms state-of-the-art in efficiency and accuracy

Provides a rigorous homotopy continuation method

Abstract

We prove in this work that the well-known lasso problem can be solved exactly without homotopy using novel differential inclusions techniques. Specifically, we show that a selection principle from the theory of differential inclusions transforms the dual lasso problem into the problem of calculating the trajectory of a projected dynamical system that we prove is integrable. Our analysis yields an exact algorithm for the lasso problem, numerically up to machine precision, that is amenable to computing regularization paths and is very fast. Moreover, we show the continuation of solutions to the integrable projected dynamical system in terms of the hyperparameter naturally yields a rigorous homotopy algorithm. Numerical experiments confirm that our algorithm outperforms the state-of-the-art algorithms in both efficiency and accuracy. Beyond this work, we expect our results and analysis can be adapted to compute exact or approximate solutions to a broader class of polyhedral-constrained optimization problems.