Bridging VLMs and Embodied Intelligence with Deliberate Practice Policy Optimization

TL;DR

This paper introduces DPPO, a training framework that combines supervised fine-tuning and reinforcement learning to improve embodied intelligence models efficiently from limited data.

Contribution

The paper presents DPPO, a novel meta-learning framework that dynamically balances competence expansion and skill refinement for embodied AI.

Findings

Pelican-VL 1.0 improved by 20.3% with DPPO

Outperforms open-source models at 100B parameters by 10.6%

First systematic framework to address data and resource limitations in embodied AI

Abstract

Developing a universal and versatile embodied intelligence system presents two primary challenges: the critical embodied data bottleneck, where real-world data is scarce and expensive, and the algorithmic inefficiency of existing methods, which are resource-prohibitive. To address these limitations, we introduce Deliberate Practice Policy Optimization (DPPO), a metacognitive ``Metaloop'' training framework that dynamically alternates between supervised fine-tuning (competence expansion) and reinforcement learning (skill refinement). This enables automatic weakness identification and targeted resource allocation, specifically designed to maximize learning efficiency from sparse, finite data. Theoretically, DPPO can be formalised as a unified preference-learning framework. Empirically, training a vision-language embodied model with DPPO, referred to as Pelican-VL 1.0, yields a 20.3% performance improvement over the base model and surpasses open-source models at the 100B-parameter scale by 10.6%. We are open-sourcing both the models and code, providing the first systematic framework that alleviates the data and resource bottleneck and enables the community to build versatile embodied agents efficiently.