Least squares approximations of measures via geometric condition numbers

TL;DR

This paper introduces a geometric approach to approximate the d-dimensional least squares error of a probability measure in a Hilbert space using volume-based functions, generalizing variance concepts.

Contribution

It provides a new comparison between volume-based averages and least squares errors, extending elementary variance ideas to higher dimensions with geometric properties.

Findings

Averages of volume-scaled functions approximate the squared least squares error.

The comparison depends on a simple geometric property of the measure.

Connections to algorithms for clustering and volume sampling are established.

Abstract

For a probability measure on a real separable Hilbert space, we are interested in "volume-based" approximations of the d-dimensional least squares error of it, i.e., least squares error with respect to a best fit d-dimensional affine subspace. Such approximations are given by averaging real-valued multivariate functions which are typically scalings of squared (d+1)-volumes of (d+1)-simplices. Specifically, we show that such averages are comparable to the square of the d-dimensional least squares error of that measure, where the comparison depends on a simple quantitative geometric property of it. This result is a higher dimensional generalization of the elementary fact that the double integral of the squared distances between points is proportional to the variance of measure. We relate our work to two recent algorithms, one for clustering affine subspaces and the other for Monte-Carlo SVD based on volume sampling.