Lower-dimensional posterior density and cluster summaries for overparameterized Bayesian models

TL;DR

This paper introduces a method to project complex, overparameterized Bayesian models onto lower-dimensional parametric summaries, enhancing interpretability while maintaining most of the original model's fit and providing uncertainty quantification.

Contribution

The novel approach combines flexible Bayesian modeling with lower-dimensional parametric summaries through a decision-theoretic projection, improving interpretability and uncertainty quantification.

Findings

Effective summaries for density and cluster estimation.

Preserves most of the original model's fit.

Applicable to synthetic and real datasets.

Abstract

The usefulness of Bayesian models for density and cluster estimation is well established across multiple literatures. However, there is still a known tension between the use of simpler, more interpretable models and more flexible, complex ones. In this paper, we propose a novel method that integrates these two approaches by projecting the fit of a flexible, overparameterized model onto a lower-dimensional parametric surrogate, which serves as a summary. This process increases interpretability while preserving most of the fit of the original model. Our approach involves three main steps. First, we fit the data using nonparametric or overparameterized models. Second, we project the posterior predictive distribution of the original model onto a sequence of parametric summary point estimates with varying dimensions using a decision-theoretic approach. Finally, given the parametric summary estimate, obtained in the second step, that best approximates the original model, we construct uncertainty quantification for this summary by projecting the original posterior distribution. We demonstrate the effectiveness of our method for generating summaries for both nonparametric and overparameterized models, delivering both point estimates and uncertainty quantification for density and cluster summaries across synthetic and real datasets.