ConceptACT: Episode-Level Concepts for Sample-Efficient Robotic Imitation Learning

TL;DR

ConceptACT introduces episode-level semantic concepts into robotic imitation learning using transformers, significantly improving sample efficiency and convergence speed by leveraging human-provided annotations during training.

Contribution

It presents a novel transformer-based architecture that integrates semantic concepts during training, enhancing imitation learning without requiring semantic input during deployment.

Findings

Faster convergence and higher sample efficiency compared to standard ACT.

Semantic supervision with concepts outperforms naive auxiliary or language-conditioned methods.

Architectural integration via attention mechanisms is crucial for performance gains.

Abstract

Imitation learning enables robots to acquire complex manipulation skills from human demonstrations, but current methods rely solely on low-level sensorimotor data while ignoring the rich semantic knowledge humans naturally possess about tasks. We present ConceptACT, an extension of Action Chunking with Transformers that leverages episode-level semantic concept annotations during training to improve learning efficiency. Unlike language-conditioned approaches that require semantic input at deployment, ConceptACT uses human-provided concepts (object properties, spatial relationships, task constraints) exclusively during demonstration collection, adding minimal annotation burden. We integrate concepts using a modified transformer architecture in which the final encoder layer implements concept-aware cross-attention, supervised to align with human annotations. Through experiments on two robotic manipulation tasks with logical constraints, we demonstrate that ConceptACT converges faster and achieves superior sample efficiency compared to standard ACT. Crucially, we show that architectural integration through attention mechanisms significantly outperforms naive auxiliary prediction losses or language-conditioned models. These results demonstrate that properly integrated semantic supervision provides powerful inductive biases for more efficient robot learning.