Incentivizing Multimodal Reasoning in Large Models for Direct Robot Manipulation

TL;DR

This paper introduces ReasonManip, a large multimodal model that leverages advanced reasoning to improve robotic manipulation, demonstrating high generalizability, transferability, and interpretability through a novel task formulation and reinforcement learning.

Contribution

The paper presents a new approach that enables large multimodal models to directly infer robotic actions via reasoning, using a novel spatial representation and fine-tuning with dialogue-based datasets.

Findings

ReasonManip shows strong generalization to new environments and objects.

The model achieves effective sim-to-real transfer.

It provides transparent reasoning linking high-level decisions to low-level control.

Abstract

Recent Large Multimodal Models have demonstrated remarkable reasoning capabilities, especially in solving complex mathematical problems and realizing accurate spatial perception. Our key insight is that these emerging abilities can naturally extend to robotic manipulation by enabling LMMs to directly infer the next goal in language via reasoning, rather than relying on a separate action head. However, this paradigm meets two main challenges: i) How to make LMMs understand the spatial action space, and ii) How to fully exploit the reasoning capacity of LMMs in solving these tasks. To tackle the former challenge, we propose a novel task formulation, which inputs the current states of object parts and the gripper, and reformulates rotation by a new axis representation instead of traditional Euler angles. This representation is more compatible with spatial reasoning and easier to interpret within a unified language space. For the latter challenge, we design a pipeline to utilize cutting-edge LMMs to generate a small but high-quality reasoning dataset of multi-round dialogues that successfully solve manipulation tasks for supervised fine-tuning. Then, we perform reinforcement learning by trial-and-error interactions in simulation to further enhance the model's reasoning abilities for robotic manipulation. Our resulting reasoning model built upon a 7B backbone, named ReasonManip, demonstrates three notable advantages driven by its system-2 level reasoning capabilities: i) exceptional generalizability to out-of-distribution environments, objects, and tasks; ii) inherent sim-to-real transfer ability enabled by the unified language representation shared across domains; iii) transparent interpretability connecting high-level reasoning and low-level control. Extensive experiments demonstrate the effectiveness of the proposed paradigm and its potential to advance LMM-driven robotic manipulation.