TL;DR
This paper introduces BiSAIL, a framework for zero-shot online adaptation of bimanual robot skills using interactive language feedback, enabling flexible, human-in-the-loop customization in real-world tasks.
Contribution
BiSAIL is a novel hierarchical approach that infers adaptation goals from multimodal feedback and modulates bimanual motions via diffusion, advancing robot adaptability and generalization.
Findings
BiSAIL outperforms existing methods in adaptability and generalization.
Demonstrated on six bimanual tasks across two platforms.
Enables non-expert users to customize robots through verbal corrections.
Abstract
Developing general-purpose robots capable of autonomously operating in human living environments requires the ability to adapt to continuously evolving task conditions. However, adapting high-dimensional coordinated bimanual skills to novel task variations at deployment remains a fundamental challenge. In this work, we present BiSAIL (Bimanual Skill Adaptation via Interactive Language), a novel framework that enables zero-shot online adaptation of offline-learned bimanual skills through interactive language feedback. The key idea of BiSAIL is to adopt a hierarchical reason-then-modulate paradigm, which first infers generalized adaptation objectives from multimodal task variations, and then adapts bimanual motions via diffusion modulation to achieve the inferred objectives. Extensive real-robot experiments across six bimanual tasks and two dual-arm platforms demonstrate that BiSAIL…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
