Kernel Regression by Mode Calculation of the Conditional Probability Distribution

TL;DR

This paper introduces a kernel regression method based on mode calculation of the conditional distribution, offering an alternative to expectation-based regression like Nadaraya-Watson, with experimental comparisons highlighting its advantages and limitations.

Contribution

It proposes a novel kernel regression approach that directly finds the mode of the conditional distribution, addressing the challenge of global optimization in joint distribution estimation.

Findings

The mode-based regression can outperform expectation-based methods in certain scenarios.

Experimental results demonstrate the method's advantages over Nadaraya-Watson regression.

The approach has specific shortcomings identified through comparative experiments.

Abstract

The most direct way to express arbitrary dependencies in datasets is to estimate the joint distribution and to apply afterwards the argmax-function to obtain the mode of the corresponding conditional distribution. This method is in practice difficult, because it requires a global optimization of a complicated function, the joint distribution by fixed input variables. This article proposes a method for finding global maxima if the joint distribution is modeled by a kernel density estimation. Some experiments show advantages and shortcomings of the resulting regression method in comparison to the standard Nadaraya-Watson regression technique, which approximates the optimum by the expectation value.