Generalized Information Criteria for Structured Sparse Models

TL;DR

This paper introduces a new Generalized Information Criteria (GIC) for structured sparse models, providing non-asymptotic bounds and conditions for consistent model selection, applicable to high-dimensional regularized estimation.

Contribution

It proposes the GIC framework that considers sparsity patterns, enabling effective model selection and regularization parameter tuning in high-dimensional settings.

Findings

GIC achieves model selection consistency under certain conditions.

Non-asymptotic bounds for model selection are established.

Applicable to group LASSO and low-rank matrix regression scenarios.

Abstract

Regularized m-estimators are widely used due to their ability of recovering a low-dimensional model in high-dimensional scenarios. Some recent efforts on this subject focused on creating a unified framework for establishing oracle bounds, and deriving conditions for support recovery. Under this same framework, we propose a new Generalized Information Criteria (GIC) that takes into consideration the sparsity pattern one wishes to recover. We obtain non-asymptotic model selection bounds and sufficient conditions for model selection consistency of the GIC. Furthermore, we show that the GIC can also be used for selecting the regularization parameter within a regularized $m$-estimation framework, which allows practical use of the GIC for model selection in high-dimensional scenarios. We provide examples of group LASSO in the context of generalized linear regression and low rank matrix regression.