Think, Act, Learn: A Framework for Autonomous Robotic Agents using Closed-Loop Large Language Models

TL;DR

This paper presents the 'Think, Act, Learn' framework that enables autonomous robots to continuously learn and adapt through a closed-loop interaction cycle with large language models, improving robustness and success in complex tasks.

Contribution

The paper introduces a novel closed-loop architecture integrating LLMs for autonomous learning and adaptation in robotics, surpassing open-loop methods in performance and generalization.

Findings

Achieves over 97% success rate on complex tasks

Converges to stable policies in about 9 trials on average

Outperforms baseline methods including open-loop LLMs and RL

Abstract

The integration of Large Language Models (LLMs) into robotics has unlocked unprecedented capabilities in high-level task planning. However, most current systems operate in an open-loop fashion, where LLMs act as one-shot planners, rendering them brittle and unable to adapt to unforeseen circumstances in dynamic physical environments. To overcome this limitation, this paper introduces the "Think, Act, Learn" (T-A-L) framework, a novel architecture that enables an embodied agent to autonomously learn and refine its policies through continuous interaction. Our framework establishes a closed-loop cycle where an LLM first "thinks" by decomposing high-level commands into actionable plans. The robot then "acts" by executing these plans while gathering rich, multimodal sensory feedback. Critically, the "learn" module processes this feedback to facilitate LLM-driven self-reflection, allowing the agent to perform causal analysis on its failures and generate corrective strategies. These insights are stored in an experiential memory to guide future planning cycles. We demonstrate through extensive experiments in both simulation and the real world that our T-A-L agent significantly outperforms baseline methods, including open-loop LLMs, Behavioral Cloning, and traditional Reinforcement Learning. Our framework achieves over a 97% success rate on complex, long-horizon tasks, converges to a stable policy in an average of just 9 trials, and exhibits remarkable generalization to unseen tasks. This work presents a significant step towards developing more robust, adaptive, and truly autonomous robotic agents.