Hierarchical Modular Reinforcement Learning Method and Knowledge Acquisition of State-Action Rule for Multi-target Problem

TL;DR

This paper extends Hierarchical Modular Reinforcement Learning to multi-target scenarios by incorporating distance-based interest estimation and extracting state-action rules with C4.5, demonstrating improved decision-making in complex environments.

Contribution

The paper introduces an expanded HMRL framework with an 'AT field' for multi-target considerations and employs C4.5 for knowledge extraction, enhancing multi-target problem solving.

Findings

Effective multi-target decision-making demonstrated

Knowledge extraction improves action selection

Method outperforms traditional approaches

Abstract

Hierarchical Modular Reinforcement Learning (HMRL), consists of 2 layered learning where Profit Sharing works to plan a prey position in the higher layer and Q-learning method trains the state-actions to the target in the lower layer. In this paper, we expanded HMRL to multi-target problem to take the distance between targets to the consideration. The function, called `AT field', can estimate the interests for an agent according to the distance between 2 agents and the advantage/disadvantage of the other agent. Moreover, the knowledge related to state-action rules is extracted by C4.5. The action under the situation is decided by using the acquired knowledge. To verify the effectiveness of proposed method, some experimental results are reported.