Analytical Results for Two Exponential Family Distributions in Hierarchical Dirichlet Processes

TL;DR

This paper derives explicit analytical results for Poisson and normal distributions within the Hierarchical Dirichlet Process framework, expanding its applicability beyond the common Dirichlet-multinomial models.

Contribution

It provides closed-form conjugate prior-likelihood pairs for Poisson and normal distributions in HDP, enhancing analytical tractability for hierarchical Bayesian models.

Findings

Explicit Gamma-Poisson conjugacy expressions derived

Normal-Gamma-Normal conjugacy expressions derived

Mathematical proofs clarify conjugacy in hierarchical models

Abstract

The Hierarchical Dirichlet Process (HDP) provides a flexible Bayesian nonparametric framework for modeling grouped data with a shared yet unbounded collection of mixture components. While existing applications of the HDP predominantly focus on the Dirichlet-multinomial conjugate structure, the framework itself is considerably more general and, in principle, accommodates a broad class of conjugate prior-likelihood pairs. In particular, exponential family distributions offer a unified and analytically tractable modeling paradigm that encompasses many commonly used distributions. In this paper, we investigate analytic results for two important members of the exponential family within the HDP framework: the Poisson distribution and the normal distribution. We derive explicit closed-form expressions for the corresponding Gamma-Poisson and Normal-Gamma-Normal conjugate pairs under the hierarchical Dirichlet process construction. Detailed derivations and proofs are provided to clarify the underlying mathematical structure and to demonstrate how conjugacy can be systematically exploited in hierarchical nonparametric models. Our work extends the applicability of the HDP beyond the Dirichlet-multinomial setting and furnishes practical analytic results for researchers employing hierarchical Bayesian nonparametrics.