Non-asymptotic model selection for linear non least-squares estimation in regression models and inverse problems

TL;DR

This paper introduces a non-asymptotic, data-driven model selection method for linear estimators in regression and inverse problems, extending existing results to non-least-squares estimators with broad applications.

Contribution

It generalizes non-asymptotic model selection to any linear estimators, not just least-squares, under general or mild assumptions, with sharp theoretical guarantees.

Findings

Provides a unified model selection criterion for regression and inverse problems.

Achieves sharp non-asymptotic bounds similar to Birgé and Massart's results.

Applicable to various fields like image processing, signal processing, and applied statistics.

Abstract

We propose to address the common problem of linear estimation in linear statistical models by using a model selection approach via penalization. Depending then on the framework in which the linear statistical model is considered namely the regression framework or the inverse problem framework, a data-driven model selection criterion is obtained either under general assumptions, or under the mild assumption of model identifiability respectively. The proposed approach was stimulated by the important recent non-asymptotic model selection results due to Birg\'e and Massart mainly (Birge and Massart 2007), and our results in this paper, like theirs, are non-asymptotic and turn to be sharp. Our main contribution in this paper resides in the fact that these linear estimators are not necessarily least-squares estimators but can be any linear estimators. The proposed approach finds therefore potential applications in countless fields of engineering and applied science (image science, signal processing,applied statistics, coding, to name a few) in which one is interested in recovering some unknown vector quantity of interest as the one, for example, which achieves the best trade-off between a term of fidelity to data, and a term of regularity or/and parsimony of the solution. The proposed approach provides then such applications with an interesting model selection framework that allows them to achieve such a goal.