Context-Aware Learning to Rank with Self-Attention

TL;DR

This paper introduces a self-attention based neural model for learning to rank that considers item interactions during both training and inference, leading to improved performance on e-commerce search and benchmark datasets.

Contribution

It proposes a novel context-aware neural network with self-attention for learning to rank, enhancing interaction modeling during inference, which was previously limited to training.

Findings

Significant performance improvements over MLP baselines.

Consistent gains across pointwise, pairwise, and listwise losses.

State-of-the-art results on MSLR-WEB30K benchmark.

Abstract

Learning to rank is a key component of many e-commerce search engines. In learning to rank, one is interested in optimising the global ordering of a list of items according to their utility for users.Popular approaches learn a scoring function that scores items individually (i.e. without the context of other items in the list) by optimising a pointwise, pairwise or listwise loss. The list is then sorted in the descending order of the scores. Possible interactions between items present in the same list are taken into account in the training phase at the loss level. However, during inference, items are scored individually, and possible interactions between them are not considered. In this paper, we propose a context-aware neural network model that learns item scores by applying a self-attention mechanism. The relevance of a given item is thus determined in the context of all other items present in the list, both in training and in inference. We empirically demonstrate significant performance gains of self-attention based neural architecture over Multi-LayerPerceptron baselines, in particular on a dataset coming from search logs of a large scale e-commerce marketplace, Allegro.pl. This effect is consistent across popular pointwise, pairwise and listwise losses.Finally, we report new state-of-the-art results on MSLR-WEB30K, the learning to rank benchmark.