Persistent Rule-based Interactive Reinforcement Learning

TL;DR

This paper introduces a persistent, rule-based interactive reinforcement learning method that retains and reuses human-provided advice, significantly improving learning efficiency and reducing interaction needs.

Contribution

It proposes a novel persistent rule-based approach that retains and generalizes human advice, enhancing agent performance and interaction efficiency.

Findings

Persistent advice improves agent performance

Reduces number of trainer interactions

Rule-based advice matches state-based performance with fewer interactions

Abstract

Interactive reinforcement learning has allowed speeding up the learning process in autonomous agents by including a human trainer providing extra information to the agent in real-time. Current interactive reinforcement learning research has been limited to real-time interactions that offer relevant user advice to the current state only. Additionally, the information provided by each interaction is not retained and instead discarded by the agent after a single-use. In this work, we propose a persistent rule-based interactive reinforcement learning approach, i.e., a method for retaining and reusing provided knowledge, allowing trainers to give general advice relevant to more than just the current state. Our experimental results show persistent advice substantially improves the performance of the agent while reducing the number of interactions required for the trainer. Moreover, rule-based advice shows similar performance impact as state-based advice, but with a substantially reduced interaction count.