Hybrid-Diffusion Models: Combining Open-loop Routines with Visuomotor Diffusion Policies

TL;DR

This paper introduces Hybrid-Diffusion models that integrate open-loop routines with visuomotor policies, enhancing manipulation task performance by triggering predefined routines during inference.

Contribution

It proposes a novel hybrid approach combining open-loop routines with diffusion policies, and develops TAPs for seamless routine execution during demonstrations.

Findings

Effective in real-world tasks like vial aspiration and liquid transfer

Improves accuracy and speed over pure visuomotor policies

Demonstrates seamless routine triggering during manipulation

Abstract

Despite the fact that visuomotor-based policies obtained via imitation learning demonstrate good performances in complex manipulation tasks, they usually struggle to achieve the same accuracy and speed as traditional control based methods. In this work, we introduce Hybrid-Diffusion models that combine open-loop routines with visuomotor diffusion policies. We develop Teleoperation Augmentation Primitives (TAPs) that allow the operator to perform predefined routines, such as locking specific axes, moving to perching waypoints, or triggering task-specific routines seamlessly during demonstrations. Our Hybrid-Diffusion method learns to trigger such TAPs during inference. We validate the method on challenging real-world tasks: Vial Aspiration, Open-Container Liquid Transfer, and container unscrewing. All experimental videos are available on the project's website: https://hybriddiffusion.github.io/