Reinforced Reasoning for Embodied Planning

TL;DR

This paper introduces a reinforcement fine-tuning framework that enhances embodied planning by integrating reasoning capabilities, leading to significant performance improvements on interactive environment benchmarks.

Contribution

It presents a novel reinforcement fine-tuning approach that incorporates reasoning into embodied planning models, improving multi-step decision-making in dynamic environments.

Findings

Outperforms similar or larger models on Embench benchmark

Shows strong generalization to unseen environments

Demonstrates the effectiveness of reinforcement-driven reasoning in embodied AI

Abstract

Embodied planning requires agents to make coherent multi-step decisions based on dynamic visual observations and natural language goals. While recent vision-language models (VLMs) excel at static perception tasks, they struggle with the temporal reasoning, spatial understanding, and commonsense grounding needed for planning in interactive environments. In this work, we introduce a reinforcement fine-tuning framework that brings R1-style reasoning enhancement into embodied planning. We first distill a high-quality dataset from a powerful closed-source model and perform supervised fine-tuning (SFT) to equip the model with structured decision-making priors. We then design a rule-based reward function tailored to multi-step action quality and optimize the policy via Generalized Reinforced Preference Optimization (GRPO). Our approach is evaluated on Embench, a recent benchmark for interactive embodied tasks, covering both in-domain and out-of-domain scenarios. Experimental results show that our method significantly outperforms models of similar or larger scale, including GPT-4o-mini and 70B+ open-source baselines, and exhibits strong generalization to unseen environments. This work highlights the potential of reinforcement-driven reasoning to advance long-horizon planning in embodied AI.