DaDu-E: Rethinking the Role of Large Language Model in Robotic Computing Pipeline

TL;DR

DaDu-E introduces a closed-loop planning framework for embodied AI robots that combines lightweight LLMs, feedback, and memory to improve adaptability, efficiency, and robustness in complex environments.

Contribution

It presents a novel closed-loop system integrating lightweight LLMs, feedback, and memory for more efficient and robust robotic planning.

Findings

Achieves task success rates comparable to larger models.

Reduces computational costs by 6.6 times.

Effectively recovers from execution failures.

Abstract

Performing complex tasks in open environments remains challenging for robots, even when using large language models (LLMs) as the core planner. Many LLM-based planners are inefficient due to their large number of parameters and prone to inaccuracies because they operate in open-loop systems. We think the reason is that only applying LLMs as planners is insufficient. In this work, we propose DaDu-E, a robust closed-loop planning framework for embodied AI robots. Specifically, DaDu-E is equipped with a relatively lightweight LLM, a set of encapsulated robot skill instructions, a robust feedback system, and memory augmentation. Together, these components enable DaDu-E to (i) actively perceive and adapt to dynamic environments, (ii) optimize computational costs while maintaining high performance, and (iii) recover from execution failures using its memory and feedback mechanisms. Extensive experiments on real-world and simulated tasks show that DaDu-E achieves task success rates comparable to embodied AI robots with larger models as planners like COME-Robot, while reducing computational requirements by $6.6 \times$. Users are encouraged to explore our system at: \url{https://rlc-lab.github.io/dadu-e/}.