On the Consistency of Average Embeddings for Item Recommendation

TL;DR

This paper examines the effectiveness of averaging item embeddings in recommender systems, introducing a metric to measure their consistency and analyzing both theoretical and empirical behaviors.

Contribution

It proposes an expected precision score to evaluate average embedding consistency and analyzes its theoretical and empirical properties in recommendation contexts.

Findings

Real-world averages are less consistent for recommendation tasks.

Theoretical analysis provides insights into the conditions affecting average embedding consistency.

Empirical data from music streaming shows discrepancies with theoretical assumptions.

Abstract

A prevalent practice in recommender systems consists in averaging item embeddings to represent users or higher-level concepts in the same embedding space. This paper investigates the relevance of such a practice. For this purpose, we propose an expected precision score, designed to measure the consistency of an average embedding relative to the items used for its construction. We subsequently analyze the mathematical expression of this score in a theoretical setting with specific assumptions, as well as its empirical behavior on real-world data from music streaming services. Our results emphasize that real-world averages are less consistent for recommendation, which paves the way for future research to better align real-world embeddings with assumptions from our theoretical setting.