GUIM -- General User and Item Embedding with Mixture of Representation in E-commerce

TL;DR

This paper introduces GUIM, a novel embedding model inspired by BERT, that creates general, multi-modal representations for users and products in Alibaba's e-commerce ecosystem, improving various downstream tasks.

Contribution

We propose GUIM, a new model using mixture of representations and contrastive learning to generate comprehensive user and item embeddings across large-scale e-commerce data.

Findings

Embeddings from GUIM outperform baselines in downstream tasks.

The mixture of representation effectively captures diverse user interests.

Contrastive learning reduces computational costs significantly.

Abstract

Our goal is to build general representation (embedding) for each user and each product item across Alibaba's businesses, including Taobao and Tmall which are among the world's biggest e-commerce websites. The representation of users and items has been playing a critical role in various downstream applications, including recommendation system, search, marketing, demand forecasting and so on. Inspired from the BERT model in natural language processing (NLP) domain, we propose a GUIM (General User Item embedding with Mixture of representation) model to achieve the goal with massive, structured, multi-modal data including the interactions among hundreds of millions of users and items. We utilize mixture of representation (MoR) as a novel representation form to model the diverse interests of each user. In addition, we use the InfoNCE from contrastive learning to avoid intractable computational costs due to the numerous size of item (token) vocabulary. Finally, we propose a set of representative downstream tasks to serve as a standard benchmark to evaluate the quality of the learned user and/or item embeddings, analogous to the GLUE benchmark in NLP domain. Our experimental results in these downstream tasks clearly show the comparative value of embeddings learned from our GUIM model.