Ranking with Multiple Objectives

TL;DR

This paper introduces a framework for ranking that balances multiple objectives by maximizing a concave function of weighted score aggregates, providing an approximation algorithm with resource augmentation guarantees, validated on synthetic and real data.

Contribution

It formulates a general multi-objective ranking problem and proposes an approximation algorithm with resource augmentation guarantees for balanced outcomes.

Findings

The algorithm achieves near-optimal results with slight resource augmentation.

Simulations demonstrate effective balancing of objectives on synthetic and real data.

The approach generalizes to various aggregation functions and ranking scenarios.

Abstract

In search and advertisement ranking, it is often required to simultaneously maximize multiple objectives. For example, the objectives can correspond to multiple intents of a search query, or in the context of advertising, they can be relevance and revenue. It is important to efficiently find rankings which strike a good balance between such objectives. Motivated by such applications, we formulate a general class of problems where - each result gets a different score corresponding to each objective, - the results of a ranking are aggregated by taking, for each objective, a weighted sum of the scores in the order of the ranking, and - an arbitrary concave function of the aggregates is maximized. Combining the aggregates using a concave function will naturally lead to more balanced outcomes. We give an approximation algorithm in a bicriteria/resource augmentation setting: the algorithm with a slight advantage does as well as the optimum. In particular, if the aggregation step is just the sum of the top k results, then the algorithm outputs k + 1 results which do as well the as the optimal top k results. We show how this approach helps with balancing different objectives via simulations on synthetic data as well as on real data from LinkedIn.