ROSER: Few-Shot Robotic Sequence Retrieval for Scalable Robot Learning

TL;DR

ROSER is a few-shot retrieval framework that extracts task-specific segments from unlabeled robotic logs, significantly improving data utilization for robot learning with minimal demonstrations.

Contribution

The paper introduces ROSER, a novel few-shot retrieval method that requires no task-specific training and outperforms existing approaches in accuracy and efficiency.

Findings

ROSER achieves sub-millisecond inference per match.

It outperforms classical and learned retrieval methods across multiple datasets.

ROSER enables effective reuse of large-scale robotic logs for task learning.

Abstract

A critical bottleneck in robot learning is the scarcity of task-labeled, segmented training data, despite the abundance of large-scale robotic datasets recorded as long, continuous interaction logs. Existing datasets contain vast amounts of diverse behaviors, yet remain structurally incompatible with modern learning frameworks that require cleanly segmented, task-specific trajectories. We address this data utilization crisis by formalizing robotic sequence retrieval: the task of extracting reusable, task-centric segments from unlabeled logs using only a few reference examples. We introduce ROSER, a lightweight few-shot retrieval framework that learns task-agnostic metric spaces over temporal windows, enabling accurate retrieval with as few as 3-5 demonstrations, without any task-specific training required. To validate our approach, we establish comprehensive evaluation protocols and benchmark ROSER against classical alignment methods, learned embeddings, and language model baselines across three large-scale datasets (e.g., LIBERO, DROID, and nuScenes). Our experiments demonstrate that ROSER consistently outperforms all prior methods in both accuracy and efficiency, achieving sub-millisecond per-match inference while maintaining superior distributional alignment. By reframing data curation as few-shot retrieval, ROSER provides a practical pathway to unlock underutilized robotic datasets, fundamentally improving data availability for robot learning.