Interactive Imitation Learning in State-Space

TL;DR

This paper introduces TIPS, a novel interactive imitation learning method that uses human feedback in state-space to improve agent behavior, outperforming traditional methods and non-expert demonstrators.

Contribution

The paper presents TIPS, a new approach for interactive imitation learning that leverages human state-space feedback, enhancing agent training beyond conventional techniques.

Findings

Agents trained with TIPS outperform non-expert demonstrators.

TIPS improves learning efficiency through continuous corrective feedback.

State-space feedback is more intuitive for humans than action-space feedback.

Abstract

Imitation Learning techniques enable programming the behavior of agents through demonstrations rather than manual engineering. However, they are limited by the quality of available demonstration data. Interactive Imitation Learning techniques can improve the efficacy of learning since they involve teachers providing feedback while the agent executes its task. In this work, we propose a novel Interactive Learning technique that uses human feedback in state-space to train and improve agent behavior (as opposed to alternative methods that use feedback in action-space). Our method titled Teaching Imitative Policies in State-space~(TIPS) enables providing guidance to the agent in terms of `changing its state' which is often more intuitive for a human demonstrator. Through continuous improvement via corrective feedback, agents trained by non-expert demonstrators using TIPS outperformed the demonstrator and conventional Imitation Learning agents.