The extended Bregman divergence and parametric estimation

TL;DR

This paper introduces an extended Bregman divergence that incorporates an exponent of the density, enabling the creation of new divergence families for robust statistical inference, demonstrated through theoretical development and empirical evaluation.

Contribution

It extends the Bregman divergence framework by considering density exponents, allowing new divergence families like the GSB divergence for robust estimation.

Findings

New divergence families can be developed within the extended framework

The minimum GSB divergence estimator performs well in simulations

Empirical results support the theoretical advantages of the extended divergence

Abstract

Minimization of suitable statistical distances~(between the data and model densities) has proved to be a very useful technique in the field of robust inference. Apart from the class of $\phi$-divergences of \cite{a} and \cite{b}, the Bregman divergence (\cite{c}) has been extensively used for this purpose. However, since the data density must have a linear presence in the cross product term of the Bregman divergence involving both the data and model densities, several useful divergences cannot be captured by the usual Bregman form. In this respect, we provide an extension of the ordinary Bregman divergence by considering an exponent of the density function as the argument rather than the density function itself. We demonstrate that many useful divergence families, which are not ordinarily Bregman divergences, can be accommodated within this extended description. Using this formulation, one can develop many new families of divergences which may be useful in robust inference. In particular, through an application of this extension, we propose the new class of the GSB divergence family. We explore the applicability of the minimum GSB divergence estimator in discrete parametric models. Simulation studies as well as conforming real data examples are given to demonstrate the performance of the estimator and to substantiate the theory developed.