Adaptor: Advancing Assistive Teleoperation with Few-Shot Learning and Cross-Operator Generalization

TL;DR

Adaptor is a few-shot learning framework that improves cross-operator intent recognition in assistive teleoperation, handling variability and enhancing success rates with robust generalization.

Contribution

It introduces a novel two-stage approach combining intent modeling and policy encoding to achieve state-of-the-art performance in diverse operator scenarios.

Findings

Achieves state-of-the-art success rates on real-world and simulated benchmarks.

Demonstrates low variance and robust generalization across operators with different expertise.

Improves efficiency in assistive teleoperation tasks.

Abstract

Assistive teleoperation enhances efficiency via shared control, yet inter-operator variability, stemming from diverse habits and expertise, induces highly heterogeneous trajectory distributions that undermine intent recognition stability. We present Adaptor, a few-shot framework for robust cross-operator intent recognition. The Adaptor bridges the domain gap through two stages: (i) preprocessing, which models intent uncertainty by synthesizing trajectory perturbations via noise injection and performs geometry-aware keyframe extraction; and (ii) policy learning, which encodes the processed trajectories with an Intention Expert and fuses them with the pre-trained vision-language model context to condition an Action Expert for action generation. Experiments on real-world and simulated benchmarks demonstrate that Adaptor achieves state-of-the-art performance, improving success rates and efficiency over baselines. Moreover, the method exhibits low variance across operators with varying expertise, demonstrating robust cross-operator generalization.