Minimum divergence estimators, maximum likelihood and exponential families

TL;DR

This paper establishes a dual representation formula for divergence between distributions in parametric models, showing that all differentiable divergences lead to the MLE within exponential families, without requiring grouping or smoothing.

Contribution

It introduces a dual representation formula for divergence and demonstrates that all differentiable divergences produce the MLE in exponential families, without smoothing or grouping.

Findings

All differentiable divergences induce the same estimator as MLE in exponential families.

The estimators for divergence and parameters do not require grouping or smoothing.

The dual representation formula simplifies divergence estimation in parametric models.

Abstract

In this note we prove the dual representation formula of the divergence between two distributions in a parametric model. Resulting estimators for the divergence as for the parameter are derived. These estimators do not make use of any grouping nor smoothing. It is proved that all differentiable divergences induce the same estimator of the parameter on any regular exponential family, which is nothing else but the MLE.