Creating Hierarchical Dispositions of Needs in an Agent

TL;DR

This paper introduces a hierarchical abstraction method for agents that prioritizes objectives, leading to improved rewards, demonstrated by state-of-the-art results on the Pendulum environment.

Contribution

It proposes a novel hierarchical learning approach using a secondary agent with multiple scalar outputs to encode needs priorities.

Findings

Outperforms baseline methods on Pendulum v1 environment

Derives a hierarchy of needs that guides goal formation

Achieves state-of-the-art results

Abstract

We present a novel method for learning hierarchical abstractions that prioritize competing objectives, leading to improved global expected rewards. Our approach employs a secondary rewarding agent with multiple scalar outputs, each associated with a distinct level of abstraction. The traditional agent then learns to maximize these outputs in a hierarchical manner, conditioning each level on the maximization of the preceding level. We derive an equation that orders these scalar values and the global reward by priority, inducing a hierarchy of needs that informs goal formation. Experimental results on the Pendulum v1 environment demonstrate superior performance compared to a baseline implementation.We achieved state of the art results.