Closed-Loop Verbal Reinforcement Learning for Task-Level Robotic Planning

TL;DR

This paper introduces a novel verbal reinforcement learning framework for interpretable, task-level robotic planning that iteratively improves policies through natural language feedback and interaction with real robots.

Contribution

It presents a closed-loop VRL framework that refines symbolic policies using language-based feedback, enabling transparent and adaptive robotic task planning.

Findings

Supports explainable policy improvements

Enables closed-loop adaptation to failures

Achieves reliable real-robot deployment

Abstract

We propose a new Verbal Reinforcement Learning (VRL) framework for interpretable task-level planning in mobile robotic systems operating under execution uncertainty. The framework follows a closed-loop architecture that enables iterative policy improvement through interaction with the physical environment. In our framework, executable Behavior Trees are repeatedly refined by a Large Language Model actor using structured natural-language feedback produced by a Vision-Language Model critic that observes the physical robot and execution traces. Unlike conventional reinforcement learning, policy updates in VRL occur directly at the symbolic planning level, without gradient-based optimization. This enables transparent reasoning, explicit causal feedback, and human-interpretable policy evolution. We validate the proposed framework on a real mobile robot performing a multi-stage manipulation and navigation task under execution uncertainty. Experimental results show that the framework supports explainable policy improvements, closed-loop adaptation to execution failures, and reliable deployment on physical robotic systems.