Factorized Transport Alignment for Multimodal and Multiview E-commerce Representation Learning

TL;DR

This paper introduces Factorized Transport, a scalable method for aligning multiple views and modalities in e-commerce representations, improving retrieval performance without increasing online computation.

Contribution

It proposes a scalable, efficient framework that unifies multimodal and multi-view learning using a lightweight optimal transport approximation, enhancing large-scale e-commerce retrieval.

Findings

Achieves up to +7.9% Recall@500 over baselines.

Reduces training complexity from quadratic to constant per item.

Maintains retrieval efficiency at inference with fused embeddings.

Abstract

The rapid growth of e-commerce requires robust multimodal representations that capture diverse signals from user-generated listings. Existing vision-language models (VLMs) typically align titles with primary images, i.e., single-view, but overlook non-primary images and auxiliary textual views that provide critical semantics in open marketplaces such as Etsy or Poshmark. To this end, we propose a framework that unifies multimodal and multi-view learning through Factorized Transport, a lightweight approximation of optimal transport, designed for scalability and deployment efficiency. During training, the method emphasizes primary views while stochastically sampling auxiliary ones, reducing training cost from quadratic in the number of views to constant per item. At inference, all views are fused into a single cached embedding, preserving the efficiency of two-tower retrieval with no additional online overhead. On an industrial dataset of 1M product listings and 0.3M interactions, our approach delivers consistent improvements in cross-view and query-to-item retrieval, achieving up to +7.9% Recall@500 over strong multimodal baselines. Overall, our framework bridges scalability with optimal transport-based learning, making multi-view pretraining practical for large-scale e-commerce search.