A Fast and Scalable Polyatomic Frank-Wolfe Algorithm for the LASSO

TL;DR

This paper introduces a novel Polyatomic Frank-Wolfe algorithm tailored for high-dimensional LASSO problems, enhancing efficiency and scalability over traditional methods by using generalized greedy steps and adaptive re-optimization.

Contribution

The paper presents a new scalable Frank-Wolfe variant that incorporates polyatomic updates and adaptive re-optimization, improving convergence and runtime in high-dimensional LASSO tasks.

Findings

Outperforms state-of-the-art methods in runtime

Provides convergence guarantees for the proposed algorithm

Effective in simulated compressed sensing scenarios

Abstract

We propose a fast and scalable Polyatomic Frank-Wolfe (P-FW) algorithm for the resolution of high-dimensional LASSO regression problems. The latter improves upon traditional Frank-Wolfe methods by considering generalized greedy steps with polyatomic (i.e. linear combinations of multiple atoms) update directions, hence allowing for a more efficient exploration of the search space. To preserve sparsity of the intermediate iterates, we re-optimize the LASSO problem over the set of selected atoms at each iteration. For efficiency reasons, the accuracy of this re-optimization step is relatively low for early iterations and gradually increases with the iteration count. We provide convergence guarantees for our algorithm and validate it in simulated compressed sensing setups. Our experiments reveal that P-FW outperforms state-of-the-art methods in terms of runtime, both for FW methods and optimal first-order proximal gradient methods such as the Fast Iterative Soft-Thresholding Algorithm (FISTA).