Fast Semiparametric Density Regression with Weight-localized Predictive Recursion

TL;DR

The paper introduces PRx, a fast recursive algorithm for covariate-dependent density regression that scales efficiently and produces competitive estimates with minimal computational effort.

Contribution

It extends predictive recursion to regression settings using kernel-based localization, enabling fast, scalable, and consistent covariate-dependent density estimation.

Findings

PRx scales linearly with sample size and covariate dimension.

PRx produces density estimates comparable to Bayesian methods.

PRx significantly reduces computational time compared to MCMC-based methods.

Abstract

Predictive recursion (PR) is a fast algorithm for nonparametric estimation of a mixing density, with connections to sequential Bayesian updating under a Dirichlet process prior and rigorous frequentist consistency guarantees. Extending PR to the regression setting, where one seeks to estimate how a mixing density varies with covariate, is nontrivial: dependent Dirichlet process priors, the natural Bayesian generalization, gives no simple recursive updating formula. We introduce PRx, which overcomes this challenge through combining kernel-based weight localization with the recursive scheme of the original PR algorithm. The algorithm scales linearly in sample size and covariate dimension, completing in seconds to minutes where MCMC-based competitors require hours. Exactly as with ordinary PR, the algorithm produces as a byproduct a likelihood score, the PRMLx, whose maximizer is shown to be a consistent estimator for unmixed parameters. In simulations and case studies PRx produces conditional density estimates competitive with established Bayesian procedures at a fraction of the computational cost, and can also be adapted for a wide range of statistical applications including Bayesian model comparison and covariate-dependent multiple testing.