Constrained recursive kernel density/regression estimation by stochastic quasi-gradient methods

TL;DR

This paper introduces a stochastic optimization framework for nonparametric kernel density and regression estimation, providing recursive constrained estimators with proven convergence rates and optimal parameters, including for moving densities.

Contribution

It develops a novel stochastic quasi-gradient approach for constrained kernel estimation, extending classical methods and analyzing moving density/regression scenarios.

Findings

Established convergence rates for the proposed estimators

Derived asymptotically optimal estimation parameters

Extended analysis to moving density/regression cases

Abstract

The paper considers nonparametric kernel density/regression estimation from a stochastic optimization point of view. The estimation problem is represented through a family of stochastic optimization problems. Recursive constrained estimators are obtained by application of stochastic (quasi)gradient methods to these problems, classical kernel estimates are derived as particular cases. Accuracy and rate of convergence of the obtained estimates are established, and asymptotically optimal estimation procedure parameters are found. The case of moving density/regression is particularly studied.