Improving Learning-to-Defer Algorithms Through Fine-Tuning

TL;DR

This paper enhances learning-to-defer algorithms by introducing fine-tuning methods that adapt AI task partitioning to specific individuals, tested on synthetic and image datasets, revealing strengths and limitations in capturing human skill nuances.

Contribution

The paper proposes two fine-tuning algorithms to improve learning-to-defer models for personalized AI-human collaboration, with empirical evaluation on synthetic and image datasets.

Findings

Fine-tuning captures simple human skill patterns.

Struggles with nuanced human skills.

Future work needed with semi-supervised approaches.

Abstract

The ubiquity of AI leads to situations where humans and AI work together, creating the need for learning-to-defer algorithms that determine how to partition tasks between AI and humans. We work to improve learning-to-defer algorithms when paired with specific individuals by incorporating two fine-tuning algorithms and testing their efficacy using both synthetic and image datasets. We find that fine-tuning can pick up on simple human skill patterns, but struggles with nuance, and we suggest future work that uses robust semi-supervised to improve learning.