Re-ranking Based Diversification: A Unifying View

TL;DR

This paper unifies various re-ranking algorithms for diversification by analyzing their mathematical foundations, focusing on submodular and modular functions, and explores how hyperparameter tuning affects relevance-diversity trade-offs.

Contribution

It provides a unifying theoretical framework for re-ranking algorithms based on submodular and modular functions, and links hyperparameter tuning to total curvature adjustment.

Findings

Most algorithms are based on maximizing submodular or modular functions.

Adjusting hyperparameters effectively tunes the total curvature for relevance-diversity trade-off.

The study offers insights into the optimality of diversification algorithms.

Abstract

We analyze different re-ranking algorithms for diversification and show that majority of them are based on maximizing submodular/modular functions from the class of parameterized concave/linear over modular functions. We study the optimality of such algorithms in terms of the `total curvature'. We also show that by adjusting the hyperparameter of the concave/linear composition to trade-off relevance and diversity, if any, one is in fact tuning the `total curvature' of the function for relevance-diversity trade-off.