Top-K Pairwise Ranking: Bridging the Gap Among Ranking-Based Measures for Multi-Label Classification

TL;DR

This paper introduces Top-K Pairwise Ranking (TKPR), a new measure for multi-label ranking that aligns with existing measures, supported by a surrogate risk minimization framework with theoretical guarantees and validated by experiments.

Contribution

It proposes TKPR, a novel measure for multi-label ranking, and develops a convex surrogate risk minimization framework with theoretical and empirical validation.

Findings

TKPR is compatible with existing ranking measures.

The framework has Fisher consistency and a sharp generalization bound.

Empirical results demonstrate the effectiveness of the proposed approach.

Abstract

Multi-label ranking, which returns multiple top-ranked labels for each instance, has a wide range of applications for visual tasks. Due to its complicated setting, prior arts have proposed various measures to evaluate model performances. However, both theoretical analysis and empirical observations show that a model might perform inconsistently on different measures. To bridge this gap, this paper proposes a novel measure named Top-K Pairwise Ranking (TKPR), and a series of analyses show that TKPR is compatible with existing ranking-based measures. In light of this, we further establish an empirical surrogate risk minimization framework for TKPR. On one hand, the proposed framework enjoys convex surrogate losses with the theoretical support of Fisher consistency. On the other hand, we establish a sharp generalization bound for the proposed framework based on a novel technique named data-dependent contraction. Finally, empirical results on benchmark datasets validate the effectiveness of the proposed framework.