PEORL: Integrating Symbolic Planning and Hierarchical Reinforcement Learning for Robust Decision-Making

TL;DR

PEORL is a unified framework that combines symbolic planning with hierarchical reinforcement learning to enable robust and efficient decision-making in dynamic, uncertain environments, improving planning and learning processes.

Contribution

It introduces a novel integration of symbolic planning with hierarchical reinforcement learning to enhance robustness and efficiency in complex, uncertain domains.

Findings

Enables rapid policy search in benchmark domains.

Produces robust symbolic plans under domain uncertainties.

Improves decision-making in dynamic environments.

Abstract

Reinforcement learning and symbolic planning have both been used to build intelligent autonomous agents. Reinforcement learning relies on learning from interactions with real world, which often requires an unfeasibly large amount of experience. Symbolic planning relies on manually crafted symbolic knowledge, which may not be robust to domain uncertainties and changes. In this paper we present a unified framework {\em PEORL} that integrates symbolic planning with hierarchical reinforcement learning (HRL) to cope with decision-making in a dynamic environment with uncertainties. Symbolic plans are used to guide the agent's task execution and learning, and the learned experience is fed back to symbolic knowledge to improve planning. This method leads to rapid policy search and robust symbolic plans in complex domains. The framework is tested on benchmark domains of HRL.