Noise-Tolerant Interactive Learning from Pairwise Comparisons

TL;DR

This paper investigates how noisy pairwise comparison oracles can improve the efficiency of learning binary classifiers, providing algorithms and bounds that nearly optimize query complexity under various noise conditions.

Contribution

It introduces a method leveraging comparison oracles to reduce learning complexity and characterizes its effectiveness with theoretical bounds under noise.

Findings

Comparison oracle reduces learning to threshold function estimation.

Proposed algorithm interacts with label and comparison oracles efficiently.

Lower bounds show near-optimal query complexity.

Abstract

We study the problem of interactively learning a binary classifier using noisy labeling and pairwise comparison oracles, where the comparison oracle answers which one in the given two instances is more likely to be positive. Learning from such oracles has multiple applications where obtaining direct labels is harder but pairwise comparisons are easier, and the algorithm can leverage both types of oracles. In this paper, we attempt to characterize how the access to an easier comparison oracle helps in improving the label and total query complexity. We show that the comparison oracle reduces the learning problem to that of learning a threshold function. We then present an algorithm that interactively queries the label and comparison oracles and we characterize its query complexity under Tsybakov and adversarial noise conditions for the comparison and labeling oracles. Our lower bounds show that our label and total query complexity is almost optimal.