An inverse Sanov theorem for exponential families

TL;DR

This paper establishes large deviation principles for posterior distributions and maximum likelihood estimators within exponential families, including misspecified models, and explores the relationship between their rate functions.

Contribution

It proves an inverse Sanov theorem for exponential families, extending large deviation results to misspecified models and analyzing the connection between posterior and MLE rate functions.

Findings

LDP for posterior distributions in exponential families

LDP for maximum likelihood estimators under misspecification

Rate functions may differ from Kullback-Leibler divergences with exchanged arguments

Abstract

We prove the large deviation principle (LDP) for posterior distributions arising from subfamilies of full exponential families, allowing misspecification of the model. Moreover, motivated by the so-called inverse Sanov Theorem (see e.g. Ganesh and O'Connell 1999 and 2000), we prove the LDP for the corresponding maximum likelihood estimator, and we study the relationship between rate functions. In our setting, even in the non misspecified case, it is not true in general that the rate functions for posterior distributions and for maximum likelihood estimators are Kullback-Leibler divergences with exchanged arguments.