A survey of non-exchangeable priors for Bayesian nonparametric models

TL;DR

This survey reviews non-exchangeable priors in Bayesian nonparametric models, focusing on dependent processes that vary with covariates, highlighting their similarities, applications, and inference methods.

Contribution

It provides a comprehensive overview of dependent nonparametric processes, clarifying their relationships and aiding in model selection and development.

Findings

Many models share underlying structures

Understanding similarities aids in model selection

Facilitates development of new models and inference techniques

Abstract

Dependent nonparametric processes extend distributions over measures, such as the Dirichlet process and the beta process, to give distributions over collections of measures, typically indexed by values in some covariate space. Such models are appropriate priors when exchangeability assumptions do not hold, and instead we want our model to vary fluidly with some set of covariates. Since the concept of dependent nonparametric processes was formalized by MacEachern [1], there have been a number of models proposed and used in the statistics and machine learning literatures. Many of these models exhibit underlying similarities, an understanding of which, we hope, will help in selecting an appropriate prior, developing new models, and leveraging inference techniques.