Active Bipartite Ranking with Smooth Posterior Distributions

TL;DR

This paper introduces a novel active bipartite ranking algorithm called smooth-rank that handles continuous, H"older smooth distributions, providing theoretical guarantees and demonstrating superior empirical performance over existing methods.

Contribution

The paper develops a new active ranking algorithm for continuous distributions, extending beyond previous discrete-only approaches, with PAC guarantees and bounds on sampling time.

Findings

smooth-rank is PAC(ε,δ) with theoretical guarantees

The algorithm outperforms existing methods empirically

Provides bounds on sampling time for the proposed method

Abstract

In this article, bipartite ranking, a statistical learning problem involved in many applications and widely studied in the passive context, is approached in a much more general \textit{active setting} than the discrete one previously considered in the literature. While the latter assumes that the conditional distribution is piece wise constant, the framework we develop permits in contrast to deal with continuous conditional distributions, provided that they fulfill a H\"older smoothness constraint. We first show that a naive approach based on discretisation at a uniform level, fixed \textit{a priori} and consisting in applying next the active strategy designed for the discrete setting generally fails. Instead, we propose a novel algorithm, referred to as smooth-rank and designed for the continuous setting, which aims to minimise the distance between the ROC curve of the estimated ranking rule and the optimal one w.r.t. the $\sup$ norm. We show that, for a fixed confidence level $\epsilon>0$ and probability $\delta\in (0,1)$, smooth-rank is PAC$(\epsilon,\delta)$. In addition, we provide a problem dependent upper bound on the expected sampling time of smooth-rank and establish a problem dependent lower bound on the expected sampling time of any PAC$(\epsilon,\delta)$ algorithm. Beyond the theoretical analysis carried out, numerical results are presented, providing solid empirical evidence of the performance of the algorithm proposed, which compares favorably with alternative approaches.