Learning to Partially Defer for Sequences

TL;DR

This paper introduces a sequence-specific deferment framework in Learning to Defer, allowing models to selectively defer parts of sequence predictions to experts, improving cost-accuracy tradeoffs across various tasks.

Contribution

It proposes a novel sequence-level deferment setting and two post-hoc rejector models for granular deferrals, enhancing existing L2D methods.

Findings

Granular deferrals outperform whole deferrals in experiments.

Token-level rejectors enable precise deferral of specific outputs.

Improved cost-accuracy tradeoffs demonstrated on multiple tasks.

Abstract

In the Learning to Defer (L2D) framework, a prediction model can either make a prediction or defer it to an expert, as determined by a rejector. Current L2D methods train the rejector to decide whether to reject the {\em entire prediction}, which is not desirable when the model predicts long sequences. We present an L2D setting for sequence outputs where the system can defer \textit{specific outputs} of the whole model prediction to an expert in an effort to interleave the expert and machine throughout the prediction. We propose two types of model-based post-hoc rejectors for pre-trained predictors: a token-level rejector, which defers specific token predictions to experts with next token prediction capabilities, and a one-time rejector for experts without such abilities, which defers the remaining sequence from a specific point onward. In the experiments, we also empirically demonstrate that such granular deferrals achieve better cost-accuracy tradeoffs than whole deferrals on Traveling salesman solvers, News summarization, and Weather prediction.