On R\'enyi and Tsallis entropies and divergences for exponential families
Frank Nielsen, Richard Nock

TL;DR
This paper derives closed-form formulas for Rényi and Tsallis divergences and entropies within exponential families, enabling easier computation for common distributions like Gaussian and Gamma.
Contribution
It provides a unified framework with closed-form expressions for Rényi and Tsallis divergences and entropies for exponential family distributions.
Findings
Closed-form expressions for Rényi and Tsallis divergences within exponential families.
Closed-form formulas for Rényi and Tsallis entropies for specific sub-families.
Applicable to distributions like Gaussian, Gamma, and Beta.
Abstract
Many common probability distributions in statistics like the Gaussian, multinomial, Beta or Gamma distributions can be studied under the unified framework of exponential families. In this paper, we prove that both R\'enyi and Tsallis divergences of distributions belonging to the same exponential family admit a generic closed form expression. Furthermore, we show that R\'enyi and Tsallis entropies can also be calculated in closed-form for sub-families including the Gaussian or exponential distributions, among others.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsStatistical Mechanics and Entropy · Statistical Distribution Estimation and Applications · Bayesian Methods and Mixture Models
