Hybrid least squares for learning functions from highly noisy data

TL;DR

This paper introduces a hybrid least squares method combining Christoffel sampling and optimal experimental design to improve function approximation from highly noisy data, with theoretical guarantees and practical validation.

Contribution

It presents a novel hybrid approach that enhances estimation accuracy and efficiency in noisy environments, extending to convex constraints and adaptive subspace techniques.

Findings

Improved sample complexity over existing methods

Theoretical optimality properties demonstrated

Numerical validation on synthetic and finance data

Abstract

Motivated by the need for efficient estimation of conditional expectations, we consider a least-squares function approximation problem with heavily polluted data. Existing methods that are powerful in the small noise regime are suboptimal when large noise is present. We propose a hybrid approach that combines Christoffel sampling with certain types of optimal experimental design to address this issue. We show that the proposed algorithm enjoys appropriate optimality properties for both sample point generation and noise mollification, leading to improved computational efficiency and sample complexity compared to existing methods. We also extend the algorithm to convex-constrained settings with similar theoretical guarantees. When the target function is defined as the expectation of a random field, we extend our approach to leverage adaptive random subspaces and establish results on the approximation capacity of the adaptive procedure. Our theoretical findings are supported by numerical studies on both synthetic data and on a more challenging stochastic simulation problem in computational finance.