Design-Conditional Prior Elicitation for Dirichlet Process Mixtures: A Unified Framework for Cluster Counts and Weight Control

TL;DR

This paper introduces a unified framework called Design-Conditional Elicitation (DCE) for calibrating hyperparameters in Dirichlet process mixture models, improving cluster count control and weight distribution while reducing computational complexity and bias.

Contribution

The paper presents DCE, a novel method for translating practitioner beliefs into hyperpriors, with a new calibration algorithm and diagnostic tools, implemented in an open-source R package.

Findings

DCE reduces bias in cluster recovery compared to default priors.

Simulation shows DCE improves weight distribution control and reduces posterior collapse.

DCE enhances model interpretability and calibration in educational and behavioral research.

Abstract

Dirichlet process mixture (DPM) models are widely used for semiparametric Bayesian analysis in educational and behavioral research, yet specifying the concentration parameter remains a critical barrier. Default hyperpriors often impose strong, unintended assumptions about clustering, while existing calibration methods based on cluster counts suffer from computational inefficiency and fail to control the distribution of mixture weights. This article introduces Design-Conditional Elicitation (DCE), a unified framework that translates practitioner beliefs about cluster structure into coherent Gamma hyperpriors for a fixed design size J. DCE makes three contributions. First, it solves the computational bottleneck using Two-Stage Moment Matching (TSMM), which couples a closed-form approximation with an exact Newton refinement to calibrate hyperparameters without grid search. Second, addressing the "unintended prior" phenomenon, DCE incorporates a Dual-Anchor protocol to diagnose and optionally constrain the risk of weight dominance while transparently reporting the resulting trade-off against cluster-count fidelity. Third, the complete workflow is implemented in the open-source DPprior R package with reproducible diagnostics and a reporting checklist. Simulation studies demonstrate that common defaults such as Gamma(1, 1) induce posterior collapse rates exceeding 60% regardless of the true cluster structure, while DCE-calibrated priors substantially reduce bias and improve recovery across varying levels of data informativeness.