Optimal Gaussian approximations to the posterior for log-linear models with Diaconis-Ylvisaker priors

TL;DR

This paper develops an optimal Gaussian approximation to the Bayesian posterior for log-linear models with Diaconis-Ylvisaker priors, enabling scalable and accurate inference in sparse contingency table analysis.

Contribution

It derives the first finite-sample bounds and convergence rates for the Gaussian approximation to the posterior in this setting.

Findings

The approximation is highly accurate in simulations.

Finite-sample bounds are established for the approximation.

Empirical results demonstrate practical effectiveness.

Abstract

In contingency table analysis, sparse data is frequently encountered for even modest numbers of variables, resulting in non-existence of maximum likelihood estimates. A common solution is to obtain regularized estimates of the parameters of a log-linear model. Bayesian methods provide a coherent approach to regularization, but are often computationally intensive. Conjugate priors ease computational demands, but the conjugate Diaconis-Ylvisaker priors for the parameters of log-linear models do not give rise to closed form credible regions, complicating posterior inference. Here we derive the optimal Gaussian approximation to the posterior for log-linear models with Diaconis-Ylvisaker priors, and provide convergence rate and finite-sample bounds for the Kullback-Leibler divergence between the exact posterior and the optimal Gaussian approximation. We demonstrate empirically in simulations and a real data application that the approximation is highly accurate, even in relatively small samples. The proposed approximation provides a computationally scalable and principled approach to regularized estimation and approximate Bayesian inference for log-linear models.