Novel Approaches to Accelerating the Convergence Rate of Markov Decision Process for Search Result Diversification

TL;DR

This paper introduces two novel methods, MDP-DIV-kNN and MDP-DIV-NTN, that significantly accelerate the convergence of Markov Decision Process-based search result diversification with minimal accuracy loss.

Contribution

The paper proposes two innovative techniques to speed up MDP-DIV convergence, addressing large action space and data scarcity issues in search result diversification.

Findings

Convergence rate improved by 3 times with the new methods.

Minimal degradation or improvement in ranking accuracy.

Effective reduction of search space in diversification process.

Abstract

Recently, some studies have utilized the Markov Decision Process for diversifying (MDP-DIV) the search results in information retrieval. Though promising performances can be delivered, MDP-DIV suffers from a very slow convergence, which hinders its usability in real applications. In this paper, we aim to promote the performance of MDP-DIV by speeding up the convergence rate without much accuracy sacrifice. The slow convergence is incurred by two main reasons: the large action space and data scarcity. On the one hand, the sequential decision making at each position needs to evaluate the query-document relevance for all the candidate set, which results in a huge searching space for MDP; on the other hand, due to the data scarcity, the agent has to proceed more "trial and error" interactions with the environment. To tackle this problem, we propose MDP-DIV-kNN and MDP-DIV-NTN methods. The MDP-DIV-kNN method adopts a $k$ nearest neighbor strategy, i.e., discarding the $k$ nearest neighbors of the recently-selected action (document), to reduce the diversification searching space. The MDP-DIV-NTN employs a pre-trained diversification neural tensor network (NTN-DIV) as the evaluation model, and combines the results with MDP to produce the final ranking solution. The experiment results demonstrate that the two proposed methods indeed accelerate the convergence rate of the MDP-DIV, which is 3x faster, while the accuracies produced barely degrade, or even are better.