Bregman-divergence-guided Legendre exponential dispersion model with finite cumulants (K-LED)

TL;DR

This paper introduces a flexible class of exponential dispersion models guided by Bregman divergence, relaxing traditional conditions and enabling easier computation of mean parameters, with applications to distributions like Tweedie, Bernoulli, and Poisson.

Contribution

It proposes the K-LED framework, a generalized exponential dispersion model with convex Legendre-type cumulant functions, facilitating easier mean parameter estimation and encompassing various distributions.

Findings

Extended normal distribution based on Tweedie distribution.

2-LED models satisfy mean-variance relations of quasi-likelihood functions.

New K-LED model with convex logistic loss includes Bernoulli and Poisson distributions.

Abstract

Exponential dispersion model is a useful framework in machine learning and statistics. Primarily, thanks to the additive structure of the model, it can be achieved without difficulty to estimate parameters including mean. However, tight conditions on cumulant function, such as analyticity, strict convexity, and steepness, reduce the class of exponential dispersion model. In this work, we present relaxed exponential dispersion model K-LED (Legendre exponential dispersion model with K cumulants). The cumulant function of the proposed model is a convex function of Legendre type having continuous partial derivatives of K-th order on the interior of a convex domain. Most of the K-LED models are developed via Bregman-divergence-guided log-concave density function with coercivity shape constraints. The main advantage of the proposed model is that the first cumulant (or the mean parameter space) of the 1-LED model is easily computed through the extended global optimum property of Bregman divergence. An extended normal distribution is introduced as an example of 1-LED based on Tweedie distribution. On top of that, we present 2-LED satisfying mean-variance relation of quasi-likelihood function. There is an equivalence between a subclass of quasi-likelihood function and a regular 2-LED model, of which the canonical parameter space is open. A typical example is a regular 2-LED model with power variance function, i.e., a variance is in proportion to the power of the mean of observations. This model is equivalent to a subclass of beta-divergence (or a subclass of quasi-likelihood function with power variance function). Furthermore, a new parameterized K-LED model, the cumulant function of which is the convex extended logistic loss function, is proposed. This model includes Bernoulli distribution and Poisson distribution.