An Adversarial Imitation Click Model for Information Retrieval

TL;DR

This paper introduces AICM, an adversarial imitation learning framework for click models in information retrieval, which improves modeling of user behavior, reduces exposure bias, and outperforms existing methods on web search data.

Contribution

The paper proposes a novel adversarial imitation learning approach for click models, explicitly learning user utility, modeling interactions dynamically, and reducing exposure bias.

Findings

AICM outperforms state-of-the-art models in traditional click metrics.

AICM reduces exposure bias from quadratic to linear in sequence length.

AICM effectively recovers underlying click sequence patterns.

Abstract

Modern information retrieval systems, including web search, ads placement, and recommender systems, typically rely on learning from user feedback. Click models, which study how users interact with a ranked list of items, provide a useful understanding of user feedback for learning ranking models. Constructing "right" dependencies is the key of any successful click model. However, probabilistic graphical models (PGMs) have to rely on manually assigned dependencies, and oversimplify user behaviors. Existing neural network based methods promote PGMs by enhancing the expressive ability and allowing flexible dependencies, but still suffer from exposure bias and inferior estimation. In this paper, we propose a novel framework, Adversarial Imitation Click Model (AICM), based on imitation learning. Firstly, we explicitly learn the reward function that recovers users' intrinsic utility and underlying intentions. Secondly, we model user interactions with a ranked list as a dynamic system instead of one-step click prediction, alleviating the exposure bias problem. Finally, we minimize the JS divergence through adversarial training and learn a stable distribution of click sequences, which makes AICM generalize well across different distributions of ranked lists. A theoretical analysis has indicated that AICM reduces the exposure bias from $O(T^2)$ to $O(T)$. Our studies on a public web search dataset show that AICM not only outperforms state-of-the-art models in traditional click metrics but also achieves superior performance in addressing the exposure bias and recovering the underlying patterns of click sequences.