Efficient Mixed Dimension Embeddings for Matrix Factorization

TL;DR

This paper introduces two matrix factorization models with mixed dimension embeddings optimized via alternating least squares, enabling scalable and efficient large-scale recommender systems that handle popularity skew effectively.

Contribution

It proposes novel mixed dimension embedding models for matrix factorization that are optimized with alternating least squares for large-scale recommender systems.

Findings

Models can be trained in a massively parallel manner.

Effective handling of popularity skew in large datasets.

Reduces parameters and overfitting on rare users/items.

Abstract

Despite the prominence of neural network approaches in the field of recommender systems, simple methods such as matrix factorization with quadratic loss are still used in industry for several reasons. These models can be trained with alternating least squares, which makes them easy to implement in a massively parallel manner, thus making it possible to utilize billions of events from real-world datasets. Large-scale recommender systems need to account for severe popularity skew in the distributions of users and items, so a lot of research is focused on implementing sparse, mixed dimension or shared embeddings to reduce both the number of parameters and overfitting on rare users and items. In this paper we propose two matrix factorization models with mixed dimension embeddings, which can be optimized in a massively parallel fashion using the alternating least squares approach.