Theoretical Understandings of Product Embedding for E-commerce Machine Learning

TL;DR

This paper provides a theoretical analysis of product embeddings in e-commerce, explaining their effectiveness and how their quality impacts downstream tasks, supported by exploratory experiments.

Contribution

It offers the first comprehensive theoretical framework for product embeddings in e-commerce, linking training algorithms to dimension reduction and task performance.

Findings

Skip-gram embeddings perform dimension reduction for product relatedness.

Embedding quality influences downstream task generalization.

Experimental results support the theoretical insights.

Abstract

Product embeddings have been heavily investigated in the past few years, serving as the cornerstone for a broad range of machine learning applications in e-commerce. Despite the empirical success of product embeddings, little is known on how and why they work from the theoretical standpoint. Analogous results from the natural language processing (NLP) often rely on domain-specific properties that are not transferable to the e-commerce setting, and the downstream tasks often focus on different aspects of the embeddings. We take an e-commerce-oriented view of the product embeddings and reveal a complete theoretical view from both the representation learning and the learning theory perspective. We prove that product embeddings trained by the widely-adopted skip-gram negative sampling algorithm and its variants are sufficient dimension reduction regarding a critical product relatedness measure. The generalization performance in the downstream machine learning task is controlled by the alignment between the embeddings and the product relatedness measure. Following the theoretical discoveries, we conduct exploratory experiments that supports our theoretical insights for the product embeddings.