On Manifold Learning in Plato's Cave: Remarks on Manifold Learning and Physical Phenomena

TL;DR

This paper discusses how manifold learning techniques can misrepresent the true geometry of physical phenomena due to measurement distortions, highlighting potential pitfalls in dimensionality reduction methods.

Contribution

It provides a mathematical analysis of metric distortions in manifold learning and demonstrates how common data processing can lead to incorrect conclusions.

Findings

Measurement distortions affect manifold learning accuracy

Common data processing can cause misleading results

The issues are broadly applicable to dimensionality reduction

Abstract

Many techniques in machine learning attempt explicitly or implicitly to infer a low-dimensional manifold structure of an underlying physical phenomenon from measurements without an explicit model of the phenomenon or the measurement apparatus. This paper presents a cautionary tale regarding the discrepancy between the geometry of measurements and the geometry of the underlying phenomenon in a benign setting. The deformation in the metric illustrated in this paper is mathematically straightforward and unavoidable in the general case, and it is only one of several similar effects. While this is not always problematic, we provide an example of an arguably standard and harmless data processing procedure where this effect leads to an incorrect answer to a seemingly simple question. Although we focus on manifold learning, these issues apply broadly to dimensionality reduction and unsupervised learning.