Accelerated Block Coordinate Proximal Gradients with Applications in High Dimensional Statistics

TL;DR

This paper introduces a novel accelerated block coordinate proximal gradient method with adaptive momentum for nonconvex optimization, demonstrating improved performance and local linear convergence in high-dimensional statistical problems.

Contribution

It proposes a new variant of the accelerated proximal gradient method that combines adaptive momentum and block coordinate updates, enhancing convergence in nonconvex settings.

Findings

Achieves local linear convergence in sparse linear regression problems.

Demonstrates improved empirical performance over existing methods.

Applicable to various regularizations like Lasso and SCAP.

Abstract

Nonconvex optimization problems arise in different research fields and arouse lots of attention in signal processing, statistics and machine learning. In this work, we explore the accelerated proximal gradient method and some of its variants which have been shown to converge under nonconvex context recently. We show that a novel variant proposed here, which exploits adaptive momentum and block coordinate update with specific update rules, further improves the performance of a broad class of nonconvex problems. In applications to sparse linear regression with regularizations like Lasso, grouped Lasso, capped $\ell_1$ and SCAP, the proposed scheme enjoys provable local linear convergence, with experimental justification.