# A Joint Planning and Learning Framework for Human-Aided Decision-Making

## TL;DR

This paper introduces PACMAN, a unified framework combining knowledge-based planning, reinforcement learning, and human feedback to accelerate and improve decision-making in agents, especially in early learning stages.

## Contribution

It presents the first integrated framework that jointly leverages planning, RL, and human teaching for policy learning, enhancing speed and robustness.

## Key findings

- PACMAN achieves rapid early-stage learning.
- It converges faster with less variance.
- The framework is robust to inconsistent and misleading feedback.

## Abstract

Conventional reinforcement learning (RL) allows an agent to learn policies via environmental rewards only, with a long and slow learning curve, especially at the beginning stage. On the contrary, human learning is usually much faster because prior and general knowledge and multiple information resources are utilized. In this paper, we propose a \textbf{P}lanner-\textbf{A}ctor-\textbf{C}ritic architecture for hu\textbf{MAN}-centered planning and learning (\textbf{PACMAN}), where an agent uses prior, high-level, deterministic symbolic knowledge to plan for goal-directed actions. PACMAN integrates Actor-Critic algorithm of RL to fine-tune its behavior towards both environmental rewards and human feedback. To the best our knowledge, This is the first unified framework where knowledge-based planning, RL, and human teaching jointly contribute to the policy learning of an agent. Our experiments demonstrate that PACMAN leads to a significant jump-start at the early stage of learning, converges rapidly and with small variance, and is robust to inconsistent, infrequent, and misleading feedback.