Cross-Domain Product Representation Learning for Rich-Content E-Commerce

TL;DR

This paper introduces a large-scale dataset and a multimodal framework for unified product recognition across diverse rich-content e-commerce media, enhancing cross-domain product search and recommendation.

Contribution

It presents the first dataset covering product pages, short videos, and live streams, and proposes COPE, a multimodal learning framework for unified product representation.

Findings

COPE effectively learns a joint feature space for multiple media domains.

The ROPE dataset enables comprehensive cross-domain product recognition research.

Experiments show improved accuracy in downstream recognition tasks.

Abstract

The proliferation of short video and live-streaming platforms has revolutionized how consumers engage in online shopping. Instead of browsing product pages, consumers are now turning to rich-content e-commerce, where they can purchase products through dynamic and interactive media like short videos and live streams. This emerging form of online shopping has introduced technical challenges, as products may be presented differently across various media domains. Therefore, a unified product representation is essential for achieving cross-domain product recognition to ensure an optimal user search experience and effective product recommendations. Despite the urgent industrial need for a unified cross-domain product representation, previous studies have predominantly focused only on product pages without taking into account short videos and live streams. To fill the gap in the rich-content e-commerce area, in this paper, we introduce a large-scale cRoss-dOmain Product Ecognition dataset, called ROPE. ROPE covers a wide range of product categories and contains over 180,000 products, corresponding to millions of short videos and live streams. It is the first dataset to cover product pages, short videos, and live streams simultaneously, providing the basis for establishing a unified product representation across different media domains. Furthermore, we propose a Cross-dOmain Product rEpresentation framework, namely COPE, which unifies product representations in different domains through multimodal learning including text and vision. Extensive experiments on downstream tasks demonstrate the effectiveness of COPE in learning a joint feature space for all product domains.