Linear Regression on Manifold Structured Data: the Impact of Extrinsic Geometry on Solutions

TL;DR

This paper investigates how the extrinsic geometry of data manifolds embedded in Euclidean space influences the uniqueness and stability of linear regression solutions, highlighting the role of curvature and nonlinearity.

Contribution

It provides a theoretical analysis of the impact of manifold curvature on the existence and uniqueness of linear regression solutions on manifold-structured data.

Findings

Linear regression lacks uniqueness on flat submanifolds.

Curvature affects the stability of solutions in normal directions.

Manifold geometry influences out-of-distribution inference stability.

Abstract

In this paper, we study linear regression applied to data structured on a manifold. We assume that the data manifold is smooth and is embedded in a Euclidean space, and our objective is to reveal the impact of the data manifold's extrinsic geometry on the regression. Specifically, we analyze the impact of the manifold's curvatures (or higher order nonlinearity in the parameterization when the curvatures are locally zero) on the uniqueness of the regression solution. Our findings suggest that the corresponding linear regression does not have a unique solution when the embedded submanifold is flat in some dimensions. Otherwise, the manifold's curvature (or higher order nonlinearity in the embedding) may contribute significantly, particularly in the solution associated with the normal directions of the manifold. Our findings thus reveal the role of data manifold geometry in ensuring the stability of regression models for out-of-distribution inferences.