Overlapping Cover Local Regression Machines

TL;DR

This paper introduces the Overlapping Domain Cover (ODC) for kernel machines, enhancing local regression speed and accuracy by using spatially cohesive overlapping data subsets, applicable to various models including TGP and GPR.

Contribution

The paper proposes a novel ODC framework that reduces regression complexity and improves local prediction accuracy, with theoretical justification and broad applicability.

Findings

Reduced TGP regression complexity from cubic to quadratic

Improved prediction accuracy on human pose datasets

Applicable to multiple kernel regression methods

Abstract

We present the Overlapping Domain Cover (ODC) notion for kernel machines, as a set of overlapping subsets of the data that covers the entire training set and optimized to be spatially cohesive as possible. We show how this notion benefit the speed of local kernel machines for regression in terms of both speed while achieving while minimizing the prediction error. We propose an efficient ODC framework, which is applicable to various regression models and in particular reduces the complexity of Twin Gaussian Processes (TGP) regression from cubic to quadratic. Our notion is also applicable to several kernel methods (e.g., Gaussian Process Regression(GPR) and IWTGP regression, as shown in our experiments). We also theoretically justified the idea behind our method to improve local prediction by the overlapping cover. We validated and analyzed our method on three benchmark human pose estimation datasets and interesting findings are discussed.