Selective Prediction from Agreement: A Lipschitz-Consistent Version Space Approach

TL;DR

This paper introduces a Lipschitz-based version space method for selective classification with abstention, ensuring predictions only when all consistent models agree, and proposes an efficient querying strategy.

Contribution

It presents a novel Lipschitz margin-based framework for selective prediction with abstention and a greedy algorithm for budgeted label querying.

Findings

The method guarantees predictions only when all Lipschitz-consistent models agree.

A submodular proxy enables efficient budgeted querying with approximation guarantees.

The approach effectively balances prediction confidence and abstention in fixed-pool settings.

Abstract

We consider selective classification with abstention in the fixed-pool (or transductive) setting, where the unlabeled pool is given beforehand and only a subset of points can be queried for labels. Our main insight is to view selective prediction through agreement: given queried labels and Lipschitz margin constraints in an embedding space, the version space of Lipschitz-consistent classification heads is well defined. We obtain upper and lower Lipschitz margin bounds that define, for each pool point, a set of certified valid labels containing the prediction of every head in the version space. The model therefore predicts only when the label is forced (i.e., all consistent heads agree), and abstains otherwise. We also propose a monotone submodular geometric proxy for budgeted querying, and show that a greedy algorithm retains the standard approximation factor.