InterPReT: Interactive Policy Restructuring and Training Enable Effective Imitation Learning from Laypersons

TL;DR

InterPReT introduces an interactive method allowing laypersons to teach AI agents new skills through demonstrations and instructions, resulting in more robust policies without requiring technical expertise.

Contribution

The paper presents InterPReT, a novel interactive framework that enables non-experts to restructure and train policies through user instructions and demonstrations.

Findings

Laypersons can effectively teach AI agents using InterPReT.

InterPReT produces more robust policies than baseline methods.

User study confirms high usability and improved performance with InterPReT.

Abstract

Imitation learning has shown success in many tasks by learning from expert demonstrations. However, most existing work relies on large-scale demonstrations from technical professionals and close monitoring of the training process. These are challenging for a layperson when they want to teach the agent new skills. To lower the barrier of teaching AI agents, we propose Interactive Policy Restructuring and Training (InterPReT), which takes user instructions to continually update the policy structure and optimize its parameters to fit user demonstrations. This enables end-users to interactively give instructions and demonstrations, monitor the agent's performance, and review the agent's decision-making strategies. A user study (N=34) on teaching an AI agent to drive in a racing game confirms that our approach yields more robust policies without impairing system usability, compared to a generic imitation learning baseline, when a layperson is responsible for both giving demonstrations and determining when to stop. This shows that our method is more suitable for end-users without much technical background in machine learning to train a dependable policy