Rejoinder on: Minimal penalties and the slope heuristics: a survey

TL;DR

This paper provides new theoretical insights into the slope heuristics for model selection, demonstrating its effectiveness even with biased models and dependent Gaussian noise, thereby extending its applicability.

Contribution

It introduces two new results: the validity of slope heuristics with biased models and with Gaussian noise having dependence, expanding previous understanding.

Findings

Slope heuristics works with significantly biased models.

Validity established for Gaussian noise with dependence.

Provides expectations of key quantities under new noise assumptions.

Abstract

This text is the rejoinder following the discussion of a survey paper about minimal penalties and the slope heuristics (Arlot, 2019. Minimal penalties and the slope heuristics: a survey. Journal de la SFDS). While commenting on the remarks made by the discussants, it provides two new results about the slope heuristics for model selection among a collection of projection estimators in least-squares fixed-design regression. First, we prove that the slope heuristics works even when all models are significantly biased. Second, when the noise is Gaussian with a general dependence structure, we compute expectations of key quantities, showing that the slope heuristics certainly is valid in this setting also.