The Brouwer Lecture 2005: Statistical estimation with model selection

TL;DR

This paper discusses the significance of model selection in statistical estimation, illustrating how choosing appropriate finite-dimensional models like histograms and variable selection impacts estimation accuracy and risk bounds.

Contribution

It introduces a general framework for finite-dimensional models in statistics and analyzes the performance of model selection procedures with concrete examples.

Findings

Model selection procedures can achieve favorable risk bounds.

Histogram partitioning and variable selection are effective in statistical estimation.

Performance depends on the choice of models and selection criteria.

Abstract

The purpose of this paper is to explain the interest and importance of (approximate) models and model selection in Statistics. Starting from the very elementary example of histograms we present a general notion of finite dimensional model for statistical estimation and we explain what type of risk bounds can be expected from the use of one such model. We then give the performance of suitable model selection procedures from a family of such models. We illustrate our point of view by two main examples: the choice of a partition for designing a histogram from an n-sample and the problem of variable selection in the context of Gaussian regression.