Improving Sequential Query Recommendation with Immediate User Feedback

TL;DR

This paper introduces a transformer-based query recommendation algorithm that incorporates immediate user feedback through a multi-armed bandit framework, significantly improving recommendation performance in interactive data exploration.

Contribution

It extends transformer models with a multi-armed bandit approach to adapt to real-time user feedback, addressing limitations of previous sequence-to-sequence methods.

Findings

Significant reduction in per-round regret compared to state-of-the-art models.

Effective adaptation to immediate user feedback in query recommendations.

Validated on large-scale logs from an online literature discovery service.

Abstract

We propose an algorithm for next query recommendation in interactive data exploration settings, like knowledge discovery for information gathering. The state-of-the-art query recommendation algorithms are based on sequence-to-sequence learning approaches that exploit historical interaction data. Due to the supervision involved in the learning process, such approaches fail to adapt to immediate user feedback. We propose to augment the transformer-based causal language models for query recommendations to adapt to the immediate user feedback using multi-armed bandit (MAB) framework. We conduct a large-scale experimental study using log files from a popular online literature discovery service and demonstrate that our algorithm improves the per-round regret substantially, with respect to the state-of-the-art transformer-based query recommendation models, which do not make use of immediate user feedback. Our data model and source code are available at https://github.com/shampp/exp3_ss