A Framework for Constrained and Adaptive Behavior-Based Agents

TL;DR

This paper introduces a framework integrating Reinforcement Learning nodes into Behavior Trees to enable adaptive learning in constrained agents, ensuring reliable execution and convergence.

Contribution

It presents a novel framework combining Behavior Trees with Reinforcement Learning, relating to Hierarchical Reinforcement Learning options, and demonstrates empirical stability.

Findings

Learning nodes do not interfere with other nodes' execution.

The framework guarantees convergence of nested learning modules.

Reinforcement Learning enhances agent adaptability without compromising behavior constraints.

Abstract

Behavior Trees are commonly used to model agents for robotics and games, where constrained behaviors must be designed by human experts in order to guarantee that these agents will execute a specific chain of actions given a specific set of perceptions. In such application areas, learning is a desirable feature to provide agents with the ability to adapt and improve interactions with humans and environment, but often discarded due to its unreliability. In this paper, we propose a framework that uses Reinforcement Learning nodes as part of Behavior Trees to address the problem of adding learning capabilities in constrained agents. We show how this framework relates to Options in Hierarchical Reinforcement Learning, ensuring convergence of nested learning nodes, and we empirically show that the learning nodes do not affect the execution of other nodes in the tree.