Generalization Bounds for Contextual Stochastic Optimization using Kernel Regression

TL;DR

This paper provides finite-sample generalization bounds and sample complexity analysis for Nadaraya-Watson kernel regression in contextual stochastic optimization, addressing performance guarantees beyond asymptotic behavior.

Contribution

It derives finite-sample generalization bounds and optimal kernel bandwidth for Nadaraya-Watson regression in contextual optimization, filling a gap in performance guarantees.

Findings

Finite-sample generalization bounds established.

Suboptimality bounds for Nadaraya-Watson solutions derived.

Optimal kernel bandwidth and sample complexity analyzed.

Abstract

In this paper, we consider contextual stochastic optimization using Nadaraya-Watson kernel regression, which is one of the most common approaches in nonparametric regression. Recent studies have explored the asymptotic convergence behavior of using Nadaraya-Watson kernel regression in contextual stochastic optimization; however, the performance guarantee under finite samples remains an open question. This paper derives a finite-sample generalization bound of the Nadaraya-Watson estimator with a spherical kernel under a generic loss function. Based on the generalization bound, we further establish a suboptimality bound for the solution of the Nadaraya-Watson approximation problem relative to the optimal solution. Finally, we derive the optimal kernel bandwidth and provide a sample complexity analysis of the Nadaraya-Watson approximation problem.