Intrinsic Dimension Correlation: uncovering nonlinear connections in multimodal representations

TL;DR

This paper introduces a new metric to detect complex nonlinear correlations in high-dimensional data, validated on synthetic and neural network multimodal representations, revealing hidden relationships between visual and textual embeddings.

Contribution

It proposes a novel correlation metric based on intrinsic dimensionality, capable of uncovering nonlinear relationships in high-dimensional manifolds, especially in neural network representations.

Findings

Successfully detects nonlinear correlations in synthetic data.

Reveals strong nonlinear correlations between visual and textual embeddings.

Outperforms existing methods in identifying similarities in multimodal representations.

Abstract

To gain insight into the mechanisms behind machine learning methods, it is crucial to establish connections among the features describing data points. However, these correlations often exhibit a high-dimensional and strongly nonlinear nature, which makes them challenging to detect using standard methods. This paper exploits the entanglement between intrinsic dimensionality and correlation to propose a metric that quantifies the (potentially nonlinear) correlation between high-dimensional manifolds. We first validate our method on synthetic data in controlled environments, showcasing its advantages and drawbacks compared to existing techniques. Subsequently, we extend our analysis to large-scale applications in neural network representations. Specifically, we focus on latent representations of multimodal data, uncovering clear correlations between paired visual and textual embeddings, whereas existing methods struggle significantly in detecting similarity. Our results indicate the presence of highly nonlinear correlation patterns between latent manifolds.