ManiLong-Shot: Interaction-Aware One-Shot Imitation Learning for Long-Horizon Manipulation

TL;DR

ManiLong-Shot introduces an interaction-aware framework for one-shot imitation learning that effectively handles long-horizon manipulation tasks by decomposing them into primitives, enabling generalization and robust real-world robot performance.

Contribution

This work presents a novel primitive-based approach for long-horizon OSIL, leveraging interaction-aware primitives and high-level reasoning to improve generalization and practical applicability.

Findings

Achieves 22.8% relative improvement over SOTA in simulation.

Generalizes from 10 short-horizon to 20 unseen long-horizon tasks.

Successfully executes long-horizon tasks on real robots.

Abstract

One-shot imitation learning (OSIL) offers a promising way to teach robots new skills without large-scale data collection. However, current OSIL methods are primarily limited to short-horizon tasks, thus limiting their applicability to complex, long-horizon manipulations. To address this limitation, we propose ManiLong-Shot, a novel framework that enables effective OSIL for long-horizon prehensile manipulation tasks. ManiLong-Shot structures long-horizon tasks around physical interaction events, reframing the problem as sequencing interaction-aware primitives instead of directly imitating continuous trajectories. This primitive decomposition can be driven by high-level reasoning from a vision-language model (VLM) or by rule-based heuristics derived from robot state changes. For each primitive, ManiLong-Shot predicts invariant regions critical to the interaction, establishes correspondences between the demonstration and the current observation, and computes the target end-effector pose, enabling effective task execution. Extensive simulation experiments show that ManiLong-Shot, trained on only 10 short-horizon tasks, generalizes to 20 unseen long-horizon tasks across three difficulty levels via one-shot imitation, achieving a 22.8% relative improvement over the SOTA. Additionally, real-robot experiments validate ManiLong-Shot's ability to robustly execute three long-horizon manipulation tasks via OSIL, confirming its practical applicability.