NOMAD Projection

TL;DR

NOMAD Projection is a scalable, GPU-compatible nonlinear dimensionality reduction method for visualizing large unstructured datasets, improving upon existing techniques in speed and performance, and enabling comprehensive data mapping.

Contribution

It introduces NOMAD Projection, a novel scalable visualization method for unstructured data that can run on multiple GPUs and provides theoretical and empirical advantages over existing methods.

Findings

Demonstrates superior speed and performance compared to state-of-the-art methods

Enables complete data mapping of large datasets like Multilingual Wikipedia

Provides theoretical bounds relating to InfoNC-t-SNE

Abstract

The rapid adoption of generative AI has driven an explosion in the size of datasets consumed and produced by AI models. Traditional methods for unstructured data visualization, such as t-SNE and UMAP, have not kept up with the pace of dataset scaling. This presents a significant challenge for AI explainability, which relies on methods such as t-SNE and UMAP for exploratory data analysis. In this paper, we introduce Negative Or Mean Affinity Discrimination (NOMAD) Projection, the first method for unstructured data visualization via nonlinear dimensionality reduction that can run on multiple GPUs at train time. We provide theory that situates NOMAD Projection as an approximate upper bound on the InfoNC-t-SNE loss, and empirical results that demonstrate NOMAD Projection's superior performance and speed profile compared to existing state-of-the-art methods. We demonstrate the scalability of NOMAD Projection by computing the first complete data map of Multilingual Wikipedia.