Enhancing User Behavior Sequence Modeling by Generative Tasks for Session Search

TL;DR

This paper introduces a novel encoder-decoder framework with generative tasks to improve session search ranking by better modeling user behavior sequences and reducing noise from irrelevant actions.

Contribution

It proposes three new generative tasks integrated with ranking models to enhance understanding of user intent in session search, demonstrating improved performance over existing methods.

Findings

Outperforms existing baseline models on public search logs.

Generative tasks improve the accuracy of user intent inference.

Applicable to various Transformer-based encoder-decoder models.

Abstract

Users' search tasks have become increasingly complicated, requiring multiple queries and interactions with the results. Recent studies have demonstrated that modeling the historical user behaviors in a session can help understand the current search intent. Existing context-aware ranking models primarily encode the current session sequence (from the first behavior to the current query) and compute the ranking score using the high-level representations. However, there is usually some noise in the current session sequence (useless behaviors for inferring the search intent) that may affect the quality of the encoded representations. To help the encoding of the current user behavior sequence, we propose to use a decoder and the information of future sequences and a supplemental query. Specifically, we design three generative tasks that can help the encoder to infer the actual search intent: (1) predicting future queries, (2) predicting future clicked documents, and (3) predicting a supplemental query. We jointly learn the ranking task with these generative tasks using an encoder-decoder structured approach. Extensive experiments on two public search logs demonstrate that our model outperforms all existing baselines, and the designed generative tasks can actually help the ranking task. Besides, additional experiments also show that our approach can be easily applied to various Transformer-based encoder-decoder models and improve their performance.