Bayesian Kernel and Mutual $k$-Nearest Neighbor Regression

TL;DR

This paper introduces Bayesian extensions for kernel and mutual k-nearest neighbor regression methods, providing probabilistic estimates and hyperparameter selection, with demonstrated convergence and improved performance on datasets.

Contribution

The paper presents novel Bayesian versions of kernel and mutual k-NN regression that include hyperparameter selection and theoretical convergence guarantees.

Findings

Bayesian methods converge asymptotically to classical regression methods.

Proposed methods outperform or match existing methods on datasets.

Effective hyperparameter selection demonstrated through simulations.

Abstract

We propose Bayesian extensions of two nonparametric regression methods which are kernel and mutual $k$-nearest neighbor regression methods. Derived based on Gaussian process models for regression, the extensions provide distributions for target value estimates and the framework to select the hyperparameters. It is shown that both the proposed methods asymptotically converge to kernel and mutual $k$-nearest neighbor regression methods, respectively. The simulation results show that the proposed methods can select proper hyperparameters and are better than or comparable to the former methods for an artificial data set and a real world data set.