Efficient and Effective Query Context-Aware Learning-to-Rank Model for Sequential Recommendation

TL;DR

This paper introduces a novel method for integrating query context into transformer-based sequential recommender systems, improving ranking relevance and diversity while addressing challenges of context alignment and privacy constraints.

Contribution

It proposes a new attention-based fusion technique for query context in transformers, enhancing recommendation quality in real-world scenarios.

Findings

Improved ranking relevance and diversity in offline experiments.

Enhanced user engagement in online platform tests.

Effective integration of query context despite privacy and feature limitations.

Abstract

Modern sequential recommender systems commonly use transformer-based models for next-item prediction. While these models demonstrate a strong balance between efficiency and quality, integrating interleaving features - such as the query context (e.g., browse category) under which next-item interactions occur - poses challenges. Effectively capturing query context is crucial for refining ranking relevance and enhancing user engagement, as it provides valuable signals about user intent within a session. Unlike item features, historical query context is typically not aligned with item sequences and may be unavailable at inference due to privacy constraints or feature store limitations - making its integration into transformers both challenging and error-prone. This paper analyzes different strategies for incorporating query context into transformers trained with a causal language modeling procedure as a case study. We propose a new method that effectively fuses the item sequence with query context within the attention mechanism. Through extensive offline and online experiments on a large-scale online platform and open datasets, we present evidence that our proposed method is an effective approach for integrating query context to improve model ranking quality in terms of relevance and diversity.