LACY: A Vision-Language Model-based Language-Action Cycle for Self-Improving Robotic Manipulation

TL;DR

LACY introduces a bidirectional vision-language framework for robotic manipulation that enhances generalization and self-improvement by learning to both act and explain actions, leading to significant performance gains.

Contribution

It presents a unified model that jointly learns language-to-action and action-to-language mappings, enabling self-supervised refinement without extra human labels.

Findings

Improves task success rates by 56.46% on average.

Enhances robustness of language-action grounding.

Operates effectively in both simulation and real-world settings.

Abstract

Learning generalizable policies for robotic manipulation increasingly relies on large-scale models that map language instructions to actions (L2A). However, this one-way paradigm often produces policies that execute tasks without deeper contextual understanding, limiting their ability to generalize or explain their behavior. We argue that the complementary skill of mapping actions back to language (A2L) is essential for developing more holistic grounding. An agent capable of both acting and explaining its actions can form richer internal representations and unlock new paradigms for self-supervised learning. We introduce LACY (Language-Action Cycle), a unified framework that learns such bidirectional mappings within a single vision-language model. LACY is jointly trained on three synergistic tasks: generating parameterized actions from language (L2A), explaining observed actions in language (A2L), and verifying semantic consistency between two language descriptions (L2C). This enables a self-improving cycle that autonomously generates and filters new training data through an active augmentation strategy targeting low-confidence cases, thereby improving the model without additional human labels. Experiments on pick-and-place tasks in both simulation and the real world show that LACY improves task success rates by 56.46% on average and yields more robust language-action grounding for robotic manipulation. Project page: https://vla2026.github.io/LACY/