A New Information Theoretical Concept: Information-Weighted Heavy-tailed Distributions

TL;DR

This paper introduces a novel class of heavy-tailed distributions derived from information-weighting of standard distributions, with properties suitable for applications sensitive to tail behavior.

Contribution

It defines a new information-theoretic method to generate heavy-tailed distributions from arbitrary continuous densities, including properties and asymptotic behavior analysis.

Findings

All proposed densities are heavy-tailed.

Information-weighted distributions preserve regular variation index.

Additive property for joint distributions of independent variables.

Abstract

Given an arbitrary continuous probability density function, it is introduced a conjugated probability density, which is defined through the Shannon information associated with its cumulative distribution function. These new densities are computed from a number of standard distributions, including uniform, normal, exponential, Pareto, logistic, Kumaraswamy, Rayleigh, Cauchy, Weibull, and Maxwell-Boltzmann. The case of joint information-weighted probability distribution is assessed. An additive property is derived in the case of independent variables. One-sided and two-sided information-weighting are considered. The asymptotic behavior of the tail of the new distributions is examined. It is proved that all probability densities proposed here define heavy-tailed distributions. It is shown that the weighting of distributions regularly varying with extreme-value index $\alpha > 0$ still results in a regular variation distribution with the same index. This approach can be particularly valuable in applications where the tails of the distribution play a major role.