A recursive procedure for density estimation on the binary hypercube

TL;DR

This paper introduces a recursive, computationally efficient method for estimating high-dimensional binary densities that adapts to sparsity, achieving near-optimal accuracy with manageable complexity.

Contribution

It presents a novel recursive estimation procedure for binary densities that is computationally feasible in high dimensions and adapts to unknown sparsity levels.

Findings

Runs in probabilistic polynomial time for sparse densities

Achieves near-minimax mean-squared error with moderate samples

Complexity decreases as density sparsity increases

Abstract

This paper describes a recursive estimation procedure for multivariate binary densities (probability distributions of vectors of Bernoulli random variables) using orthogonal expansions. For $d$ covariates, there are $2^d$ basis coefficients to estimate, which renders conventional approaches computationally prohibitive when $d$ is large. However, for a wide class of densities that satisfy a certain sparsity condition, our estimator runs in probabilistic polynomial time and adapts to the unknown sparsity of the underlying density in two key ways: (1) it attains near-minimax mean-squared error for moderate sample sizes, and (2) the computational complexity is lower for sparser densities. Our method also allows for flexible control of the trade-off between mean-squared error and computational complexity.