EncodeRec: An Embedding Backbone for Recommendation Systems

TL;DR

EncodeRec is a novel method that adapts pre-trained language model embeddings for recommendation systems by aligning them with recommendation objectives, resulting in more domain-specific and discriminative item representations without retraining the language model.

Contribution

The paper introduces EncodeRec, a technique that learns compact, informative embeddings from item descriptions while keeping the language model frozen, improving recommendation performance.

Findings

Significant improvements over PLM-based baselines in recommendation benchmarks.

Effective as both a backbone for sequential recommendation and semantic ID tokenization.

Maintains computational efficiency by freezing language model parameters.

Abstract

Recent recommender systems increasingly leverage embeddings from large pre-trained language models (PLMs). However, such embeddings exhibit two key limitations: (1) PLMs are not explicitly optimized to produce structured and discriminative embedding spaces, and (2) their representations remain overly generic, often failing to capture the domain-specific semantics crucial for recommendation tasks. We present EncodeRec, an approach designed to align textual representations with recommendation objectives while learning compact, informative embeddings directly from item descriptions. EncodeRec keeps the language model parameters frozen during recommender system training, making it computationally efficient without sacrificing semantic fidelity. Experiments across core recommendation benchmarks demonstrate its effectiveness both as a backbone for sequential recommendation models and for semantic ID tokenization, showing substantial gains over PLM-based and embedding model baselines. These results underscore the pivotal role of embedding adaptation in bridging the gap between general-purpose language models and practical recommender systems.