Linear Models are Most Favorable among Generalized Linear Models

TL;DR

This paper proves that among a broad class of generalized linear models, the canonical linear model has the lowest possible maximum risk, making it the most favorable choice in terms of minimax risk.

Contribution

It establishes a nonasymptotic lower bound on the minimax risk for generalized linear models and shows the canonical linear model attains this bound up to a constant.

Findings

Canonical linear model matches the minimax risk lower bound.

Provides a nonasymptotic risk lower bound for generalized linear models.

Uses an information-theoretic Bayesian Cramér-Rao bound in the proof.

Abstract

We establish a nonasymptotic lower bound on the $L_2$ minimax risk for a class of generalized linear models. It is further shown that the minimax risk for the canonical linear model matches this lower bound up to a universal constant. Therefore, the canonical linear model may be regarded as most favorable among the considered class of generalized linear models (in terms of minimax risk). The proof makes use of an information-theoretic Bayesian Cram\'er-Rao bound for log-concave priors, established by Aras et al. (2019).