Random Function Priors for Correlation Modeling

TL;DR

This paper introduces a novel Bayesian nonparametric approach called PRME, using random function priors to model correlations among hidden features in high-dimensional data, learned efficiently with neural networks.

Contribution

It proposes the population random measure embedding (PRME), a new nonparametric model for correlation modeling using random functions, and demonstrates efficient neural network-based inference.

Findings

Effective modeling of feature correlations in high-dimensional data.

Efficient neural network-based inference for the proposed model.

Theoretical foundation via a representation theorem on exchangeable measures.

Abstract

The likelihood model of high dimensional data $X_n$ can often be expressed as $p(X_n|Z_n,\theta)$, where $\theta\mathrel{\mathop:}=(\theta_k)_{k\in[K]}$ is a collection of hidden features shared across objects, indexed by $n$, and $Z_n$ is a non-negative factor loading vector with $K$ entries where $Z_{nk}$ indicates the strength of $\theta_k$ used to express $X_n$. In this paper, we introduce random function priors for $Z_n$ for modeling correlations among its $K$ dimensions $Z_{n1}$ through $Z_{nK}$, which we call \textit{population random measure embedding} (PRME). Our model can be viewed as a generalized paintbox model~\cite{Broderick13} using random functions, and can be learned efficiently with neural networks via amortized variational inference. We derive our Bayesian nonparametric method by applying a representation theorem on separately exchangeable discrete random measures.