Novelty Adaptation Through Hybrid Large Language Model (LLM)-Symbolic Planning and LLM-guided Reinforcement Learning

TL;DR

This paper introduces a neuro-symbolic framework combining LLMs, symbolic planning, and reinforcement learning to enable autonomous agents to adapt to novelties in dynamic environments, improving operator discovery and learning.

Contribution

The paper presents a novel hybrid architecture that uses LLMs for common sense reasoning to identify and learn new operators, enhancing planning and control in robotic agents.

Findings

Outperforms state-of-the-art in operator discovery

Effective in continuous robotic domains

Enhances adaptability to environmental novelties

Abstract

In dynamic open-world environments, autonomous agents often encounter novelties that hinder their ability to find plans to achieve their goals. Specifically, traditional symbolic planners fail to generate plans when the robot's planning domain lacks the operators that enable it to interact appropriately with novel objects in the environment. We propose a neuro-symbolic architecture that integrates symbolic planning, reinforcement learning, and a large language model (LLM) to learn how to handle novel objects. In particular, we leverage the common sense reasoning capability of the LLM to identify missing operators, generate plans with the symbolic AI planner, and write reward functions to guide the reinforcement learning agent in learning control policies for newly identified operators. Our method outperforms the state-of-the-art methods in operator discovery as well as operator learning in continuous robotic domains.