Gentle Manipulation Policy Learning via Demonstrations from VLM Planned Atomic Skills

TL;DR

This paper introduces a hierarchical framework combining visual language models, reinforcement learning, and knowledge distillation to enable scalable, long-horizon contact-rich manipulation without extensive real-world data.

Contribution

It presents a novel integration of VLMs, RL, and knowledge distillation for atomic skill learning, reducing reliance on human demonstrations and enhancing task generalization.

Findings

Effective policy learning in simulation for complex tasks

VLM-guided demonstrations improve generalization

System achieves successful real-world deployment

Abstract

Autonomous execution of long-horizon, contact-rich manipulation tasks traditionally requires extensive real-world data and expert engineering, posing significant cost and scalability challenges. This paper proposes a novel framework integrating hierarchical semantic decomposition, reinforcement learning (RL), visual language models (VLMs), and knowledge distillation to overcome these limitations. Complex tasks are decomposed into atomic skills, with RL-trained policies for each primitive exclusively in simulation. Crucially, our RL formulation incorporates explicit force constraints to prevent object damage during delicate interactions. VLMs perform high-level task decomposition and skill planning, generating diverse expert demonstrations. These are distilled into a unified policy via Visual-Tactile Diffusion Policy for end-to-end execution. We conduct comprehensive ablation studies exploring different VLM-based task planners to identify optimal demonstration generation pipelines, and systematically compare imitation learning algorithms for skill distillation. Extensive simulation experiments and physical deployment validate that our approach achieves policy learning for long-horizon manipulation without costly human demonstrations, while the VLM-guided atomic skill framework enables scalable generalization to diverse tasks.