Accelerating the Learning of TAMER with Counterfactual Explanations

TL;DR

This paper enhances the TAMER framework for human-in-the-loop reinforcement learning by incorporating counterfactual explanations, significantly speeding up the learning process and improving user experience.

Contribution

It introduces two types of counterfactual explanations into TAMER, a novel approach that accelerates learning in human-in-the-loop RL systems.

Findings

Counterfactual explanations improve learning speed

Enhanced TAMER outperforms baseline methods

User feedback becomes more effective with explanations

Abstract

The capability to interactively learn from human feedback would enable agents in new settings. For example, even novice users could train service robots in new tasks naturally and interactively. Human-in-the-loop Reinforcement Learning (HRL) combines human feedback and Reinforcement Learning (RL) techniques. State-of-the-art interactive learning techniques suffer from slow learning speed, thus leading to a frustrating experience for the human. We approach this problem by extending the HRL framework TAMER for evaluative feedback with the possibility to enhance human feedback with two different types of counterfactual explanations (action and state based). We experimentally show that our extensions improve the speed of learning.