Order-Preserving Dimension Reduction for Multimodal Semantic Embedding

TL;DR

This paper introduces Order-Preserving Dimension Reduction (OPDR), a method that reduces embedding dimensions in multimodal data while maintaining KNN ranking, thereby improving retrieval efficiency without sacrificing accuracy.

Contribution

The paper proposes a novel OPDR technique that preserves KNN order during dimension reduction and introduces a new global KNN quality measure for multimodal embeddings.

Findings

OPDR retains high recall accuracy in multimodal retrieval tasks.

Significant reduction in computational costs achieved with OPDR.

Effective integration of OPDR with existing embedding and reduction techniques.

Abstract

Searching for the $k$-nearest neighbors (KNN) in multimodal data retrieval is computationally expensive, particularly due to the inherent difficulty in comparing similarity measures across different modalities. Recent advances in multimodal machine learning address this issue by mapping data into a shared embedding space; however, the high dimensionality of these embeddings (hundreds to thousands of dimensions) presents a challenge for time-sensitive vision applications. This work proposes Order-Preserving Dimension Reduction (OPDR), aiming to reduce the dimensionality of embeddings while preserving the ranking of KNN in the lower-dimensional space. One notable component of OPDR is a new measure function to quantify KNN quality as a global metric, based on which we derive a closed-form map between target dimensionality and key contextual parameters. We have integrated OPDR with multiple state-of-the-art dimension-reduction techniques, distance functions, and embedding models; experiments on a variety of multimodal datasets demonstrate that OPDR effectively retains recall high accuracy while significantly reducing computational costs.