Towards Lifelong Learning Embeddings: An Algorithmic Approach to Dynamically Extend Embeddings

TL;DR

This paper presents a modular algorithm that dynamically extends embeddings in e-commerce, enabling continuous learning and better handling of new products without retraining from scratch.

Contribution

It introduces a novel algorithm for extending embedding sizes while retaining learned knowledge, addressing the challenge of dynamic product catalogs in e-commerce.

Findings

Outperforms traditional fixed-size embeddings in experiments

Mitigates cold start problem for new products

Enables dynamic embedding extension without retraining

Abstract

The rapid evolution of technology has transformed business operations and customer interactions worldwide, with personalization emerging as a key opportunity for e-commerce companies to engage customers more effectively. The application of machine learning, particularly that of deep learning models, has gained significant traction due to its ability to rapidly recognize patterns in large datasets, thereby offering numerous possibilities for personalization. These models use embeddings to map discrete information, such as product IDs, into a latent vector space, a method increasingly popular in recent years. However, e-commerce's dynamic nature, characterized by frequent new product introductions, poses challenges for these embeddings, which typically require fixed dimensions and inputs, leading to the need for periodic retraining from scratch. This paper introduces a modular algorithm that extends embedding input size while preserving learned knowledge, addressing the challenges posed by e-commerce's dynamism. The proposed algorithm also incorporates strategies to mitigate the cold start problem associated with new products. The results of initial experiments suggest that this method outperforms traditional embeddings.