On the Mathematical Relationship between Expected n-call@k and the Relevance vs. Diversity Trade-off

TL;DR

This paper derives a mathematical relationship between expected n-call@k and the relevance-diversity trade-off, showing that diversification increases as n approaches 1, independent of set size k.

Contribution

It provides a formal quantification of how optimizing n-call@k influences result set diversification, revealing a simple relationship dependent on n.

Findings

Diversification increases as n approaches 1.

The trade-off is independent of the result set size k.

A mathematical formula links expected n-call@k to relevance-diversity balance.

Abstract

It has been previously noted that optimization of the n-call@k relevance objective (i.e., a set-based objective that is 1 if at least n documents in a set of k are relevant, otherwise 0) encourages more result set diversification for smaller n, but this statement has never been formally quantified. In this work, we explicitly derive the mathematical relationship between expected n-call@k and the relevance vs. diversity trade-off --- through fortuitous cancellations in the resulting combinatorial optimization, we show the trade-off is a simple and intuitive function of n (notably independent of the result set size k e n), where diversification increases as n approaches 1.