Predictive Inference via Kernel Density Estimates

TL;DR

This paper investigates the convergence properties of two kernel-based prediction rules, revealing their different limiting behaviors and potential Bayesian interpretations.

Contribution

It demonstrates the almost sure weak convergence of both classic and recursive kernel prediction processes, offering new insights into Bayesian interpretations of kernel density estimation.

Findings

Classic kernel density estimator converges to a compactly supported measure.

Recursive kernel prediction converges to a non-compactly supported measure.

Both processes exhibit almost sure weak convergence.

Abstract

Kernel density estimation is a widely used nonparametric approach to estimate an unknown distribution. Recent work in Bayesian predictive inference has considered stochastic processes formed by specifying the predictive distribution for the next data point given all observed data such that the resulting predictive distributions converge weakly almost surely. We study two kernel based prediction rules: the classic kernel density estimator, and a recursive version previously introduced for online problems. We show that both processes converge weakly almost surely, which opens the door for new Bayesian interpretations of kernel density estimation. Surprisingly, the process based on the classic kernel density estimates converges to a compactly supported measure, while the recursive version converges to a non-compactly supported measure.