Converting ADMM to a Proximal Gradient for Efficient Sparse Estimation

TL;DR

This paper introduces a method to convert ADMM solutions into proximal gradient methods for sparse estimation, significantly improving computational efficiency in problems like convex clustering and trend filtering.

Contribution

It proposes a general conversion technique from ADMM to proximal gradient methods under Lipschitz continuity assumptions, enhancing efficiency in sparse estimation tasks.

Findings

Significant speed-up in sparse estimation algorithms.

Successful application to convex clustering and trend filtering.

Numerical experiments demonstrate improved efficiency.

Abstract

In sparse estimation, such as fused lasso and convex clustering, we apply either the proximal gradient method or the alternating direction method of multipliers (ADMM) to solve the problem. It takes time to include matrix division in the former case, while an efficient method such as FISTA (fast iterative shrinkage-thresholding algorithm) has been developed in the latter case. This paper proposes a general method for converting the ADMM solution to the proximal gradient method, assuming that assumption that the derivative of the objective function is Lipschitz continuous. Then, we apply it to sparse estimation problems, such as sparse convex clustering and trend filtering, and we show by numerical experiments that we can obtain a significant improvement in terms of efficiency.