Exact Non-Parametric Bayesian Inference on Infinite Trees

TL;DR

This paper introduces an exact Bayesian inference method for infinite tree models that adaptively estimates probability densities from i.i.d. data, providing fast computations and theoretical guarantees.

Contribution

It develops a novel, exact inference algorithm for a prior over infinite trees, enabling adaptive density estimation with proven convergence and consistency.

Findings

Efficient inference algorithm for infinite tree priors

Proven asymptotic convergence and consistency

Illustrated behavior on prototypical functions

Abstract

Given i.i.d. data from an unknown distribution, we consider the problem of predicting future items. An adaptive way to estimate the probability density is to recursively subdivide the domain to an appropriate data-dependent granularity. A Bayesian would assign a data-independent prior probability to "subdivide", which leads to a prior over infinite(ly many) trees. We derive an exact, fast, and simple inference algorithm for such a prior, for the data evidence, the predictive distribution, the effective model dimension, moments, and other quantities. We prove asymptotic convergence and consistency results, and illustrate the behavior of our model on some prototypical functions.