MA4DIV: Multi-Agent Reinforcement Learning for Search Result Diversification

TL;DR

This paper introduces MA4DIV, a multi-agent reinforcement learning approach for search result diversification that models documents as cooperative agents, directly optimizing diversity metrics and outperforming existing methods in effectiveness and efficiency.

Contribution

The paper proposes a novel MARL-based framework for SRD, enabling direct optimization of diversity metrics and improved training efficiency over traditional greedy approaches.

Findings

MA4DIV outperforms baseline methods on TREC datasets.

It achieves higher diversity and relevance in search results.

The approach is especially effective in industrial-scale settings.

Abstract

Search result diversification (SRD), which aims to ensure that documents in a ranking list cover a broad range of subtopics, is a significant and widely studied problem in Information Retrieval and Web Search. Existing methods primarily utilize a paradigm of "greedy selection", i.e., selecting one document with the highest diversity score at a time or optimize an approximation of the objective function. These approaches tend to be inefficient and are easily trapped in a suboptimal state. To address these challenges, we introduce Multi-Agent reinforcement learning (MARL) for search result DIVersity, which called MA4DIV. In this approach, each document is an agent and the search result diversification is modeled as a cooperative task among multiple agents. By modeling the SRD ranking problem as a cooperative MARL problem, this approach allows for directly optimizing the diversity metrics, such as $\alpha$-NDCG, while achieving high training efficiency. We conducted experiments on public TREC datasets and a larger scale dataset in the industrial setting. The experiemnts show that MA4DIV achieves substantial improvements in both effectiveness and efficiency than existing baselines, especially on the industrial dataset. The code of MA4DIV can be seen on https://github.com/chenyiqun/MA4DIV.