ESPADA: Execution Speedup via Semantics Aware Demonstration Data Downsampling for Imitation Learning

TL;DR

ESPADA is a semantic-aware demonstration downsampling framework that accelerates imitation learning by selectively reducing demonstration data in non-critical segments, achieving about 2x speed-up without sacrificing success.

Contribution

It introduces a novel semantic and spatially aware segmentation method using VLM-LLM and DTW, enabling effective data downsampling without extra data or retraining.

Findings

ESPADA achieves approximately 2x speed-up in imitation learning tasks.

It maintains success rates comparable to full demonstration data.

The method works across simulation and real-world experiments.

Abstract

Behavior-cloning based visuomotor policies enable precise manipulation but often inherit the slow, cautious tempo of human demonstrations, limiting practical deployment. However, prior studies on acceleration methods mainly rely on statistical or heuristic cues that ignore task semantics and can fail across diverse manipulation settings. We present ESPADA, a semantic and spatially aware framework that segments demonstrations using a VLM-LLM pipeline with 3D gripper-object relations, enabling aggressive downsampling only in non-critical segments while preserving precision-critical phases, without requiring extra data or architectural modifications, or any form of retraining. To scale from a single annotated episode to the full dataset, ESPADA propagates segment labels via Dynamic Time Warping (DTW) on dynamics-only features. Across both simulation and real-world experiments with ACT and DP baselines, ESPADA achieves approximately a 2x speed-up while maintaining success rates, narrowing the gap between human demonstrations and efficient robot control.