Learning Model-Based Sparsity via Projected Gradient Descent

TL;DR

This paper investigates a projected gradient descent method for estimating structured-sparse parameters within a non-convex model, providing theoretical guarantees under certain conditions and demonstrating its application to Generalized Linear Models.

Contribution

It introduces a novel application of projected gradient descent to non-convex structured-sparse models with theoretical analysis and practical relevance.

Findings

Algorithm produces accurate approximations under Stable Model-Restricted Hessian

Applicable to Generalized Linear Models

Avoids regularization parameter tuning

Abstract

Several convex formulation methods have been proposed previously for statistical estimation with structured sparsity as the prior. These methods often require a carefully tuned regularization parameter, often a cumbersome or heuristic exercise. Furthermore, the estimate that these methods produce might not belong to the desired sparsity model, albeit accurately approximating the true parameter. Therefore, greedy-type algorithms could often be more desirable in estimating structured-sparse parameters. So far, these greedy methods have mostly focused on linear statistical models. In this paper we study the projected gradient descent with non-convex structured-sparse parameter model as the constraint set. Should the cost function have a Stable Model-Restricted Hessian the algorithm produces an approximation for the desired minimizer. As an example we elaborate on application of the main results to estimation in Generalized Linear Model.