Unified Vision-Language Representation Modeling for E-Commerce Same-Style Products Retrieval

TL;DR

This paper introduces a unified vision-language model for e-commerce product retrieval that combines visual and textual features, improving accuracy especially in image-less industries, and is deployed on Alibaba.

Contribution

It proposes a novel joint embedding approach with a sampling strategy and contrastive loss for cross-modal product retrieval in e-commerce.

Findings

Superior retrieval performance demonstrated offline

Increased user clicks and conversions online

Deployed successfully on Alibaba platform

Abstract

Same-style products retrieval plays an important role in e-commerce platforms, aiming to identify the same products which may have different text descriptions or images. It can be used for similar products retrieval from different suppliers or duplicate products detection of one supplier. Common methods use the image as the detected object, but they only consider the visual features and overlook the attribute information contained in the textual descriptions, and perform weakly for products in image less important industries like machinery, hardware tools and electronic component, even if an additional text matching module is added. In this paper, we propose a unified vision-language modeling method for e-commerce same-style products retrieval, which is designed to represent one product with its textual descriptions and visual contents. It contains one sampling skill to collect positive pairs from user click log with category and relevance constrained, and a novel contrastive loss unit to model the image, text, and image+text representations into one joint embedding space. It is capable of cross-modal product-to-product retrieval, as well as style transfer and user-interactive search. Offline evaluations on annotated data demonstrate its superior retrieval performance, and online testings show it can attract more clicks and conversions. Moreover, this model has already been deployed online for similar products retrieval in alibaba.com, the largest B2B e-commerce platform in the world.