Mind to Hand: Purposeful Robotic Control via Embodied Reasoning

TL;DR

Lumo-1 is a unified vision-language-action model that advances embodied reasoning and robotic control by integrating pre-trained multimodal reasoning, structured training stages, and reinforcement learning, enabling robots to perform complex, generalist tasks with human-like understanding.

Contribution

The paper introduces Lumo-1, a novel three-stage training pipeline that combines vision-language pre-training, cross-embodiment data, and reinforcement learning to enhance robot reasoning and action alignment.

Findings

Lumo-1 outperforms baselines in embodied reasoning tasks.

It demonstrates strong generalization to new objects and environments.

Excels in long-horizon and natural language instruction tasks.

Abstract

Humans act with context and intention, with reasoning playing a central role. While internet-scale data has enabled broad reasoning capabilities in AI systems, grounding these abilities in physical action remains a major challenge. We introduce Lumo-1, a generalist vision-language-action (VLA) model that unifies robot reasoning ("mind") with robot action ("hand"). Our approach builds upon the general multi-modal reasoning capabilities of pre-trained vision-language models (VLMs), progressively extending them to embodied reasoning and action prediction, and ultimately towards structured reasoning and reasoning-action alignment. This results in a three-stage pre-training pipeline: (1) Continued VLM pre-training on curated vision-language data to enhance embodied reasoning skills such as planning, spatial understanding, and trajectory prediction; (2) Co-training on cross-embodiment robot data alongside vision-language data; and (3) Action training with reasoning process on trajectories collected on Astribot S1, a bimanual mobile manipulator with human-like dexterity and agility. Finally, we integrate reinforcement learning to further refine reasoning-action consistency and close the loop between semantic inference and motor control. Extensive experiments demonstrate that Lumo-1 achieves significant performance improvements in embodied vision-language reasoning, a critical component for generalist robotic control. Real-world evaluations further show that Lumo-1 surpasses strong baselines across a wide range of challenging robotic tasks, with strong generalization to novel objects and environments, excelling particularly in long-horizon tasks and responding to human-natural instructions that require reasoning over strategy, concepts and space.