The Reconstruction Approach: From Interpolation to Regression

TL;DR

This paper presents the reconstruction approach, an interpolation-based method for nonparametric regression that offers computational efficiency and new insights, especially beneficial for large datasets.

Contribution

It introduces the reconstruction approach, unifying existing methods and creating new, efficient estimation techniques with reduced computational complexity.

Findings

Kernel ridge regression as a special case

Effective surrogates for complex models

Suitable for large datasets with reduced computational burden

Abstract

This paper introduces an interpolation-based method, called the reconstruction approach, for nonparametric regression. Based on the fact that interpolation usually has negligible errors compared to statistical estimation, the reconstruction approach uses an interpolator to parameterize the regression function with its values at finite knots, and then estimates these values by (regularized) least squares. Some popular methods including kernel ridge regression can be viewed as its special cases. It is shown that, the reconstruction idea not only provides different angles to look into existing methods, but also produces new effective experimental design and estimation methods for nonparametric models. In particular, for some methods of complexity O(n3), where n is the sample size, this approach provides effective surrogates with much less computational burden. This point makes it very suitable for large datasets.