Joint Characterization of Multiscale Information in High Dimensional Data

TL;DR

This paper introduces a multiscale joint characterization method combining PCA and t-SNE to analyze high-dimensional data, capturing both global and local variance structures for improved signal detection.

Contribution

It presents a novel approach that integrates PCA and t-SNE for multiscale analysis, enhancing the detection of signals not visible with either method alone.

Findings

Joint characterization detects signals missed by individual methods.

PCA provides interpretable global structure.

t-SNE reveals local clusters effectively.

Abstract

High dimensional data can contain multiple scales of variance. Analysis tools that preferentially operate at one scale can be ineffective at capturing all the information present in this cross-scale complexity. We propose a multiscale joint characterization approach designed to exploit synergies between global and local approaches to dimensionality reduction. We illustrate this approach using Principal Components Analysis (PCA) to characterize global variance structure and t-stochastic neighbor embedding (t-sne) to characterize local variance structure. Using both synthetic images and real-world imaging spectroscopy data, we show that joint characterization is capable of detecting and isolating signals which are not evident from either PCA or t-sne alone. Broadly, t-sne is effective at rendering a randomly oriented low-dimensional map of local clusters, and PCA renders this map interpretable by providing global, physically meaningful structure. This approach is illustrated using imaging spectroscopy data, and may prove particularly useful for other geospatial data given robust local variance structure due to spatial autocorrelation and physical interpretability of global variance structure due to spectral properties of Earth surface materials. However, the fundamental premise could easily be extended to other high dimensional datasets, including image time series and non-image data.