Kullback-Leibler aggregation and misspecified generalized linear models

TL;DR

This paper extends aggregation methods to exponential family distributions in regression, providing sharp, optimal oracle inequalities for penalized likelihood estimators without requiring model correctness or parameter identifiability.

Contribution

It introduces a generalized aggregation framework for exponential family distributions, extending beyond Gaussian models, with theoretical guarantees and optimal bounds.

Findings

Derived sharp oracle inequalities for the proposed estimators

Established minimax optimality of the bounds

Extended aggregation techniques to a broader class of distributions

Abstract

In a regression setup with deterministic design, we study the pure aggregation problem and introduce a natural extension from the Gaussian distribution to distributions in the exponential family. While this extension bears strong connections with generalized linear models, it does not require identifiability of the parameter or even that the model on the systematic component is true. It is shown that this problem can be solved by constrained and/or penalized likelihood maximization and we derive sharp oracle inequalities that hold both in expectation and with high probability. Finally all the bounds are proved to be optimal in a minimax sense.