Closed-loop Teaching via Demonstrations to Improve Policy Transparency

TL;DR

This paper introduces a closed-loop teaching framework that uses demonstrations and real-time belief modeling to enhance policy transparency and reduce human testing regret.

Contribution

It proposes a novel particle filter-based model of human beliefs and a closed-loop curriculum that adapts demonstrations in real time, improving transparency and learning outcomes.

Findings

Reduces human test response regret by 43%.

Demonstrations tailored to current understanding improve learning.

Framework inspired by educational principles like the zone of proximal development.

Abstract

Demonstrations are a powerful way of increasing the transparency of AI policies. Though informative demonstrations may be selected a priori through the machine teaching paradigm, student learning may deviate from the preselected curriculum in situ. This paper thus explores augmenting a curriculum with a closed-loop teaching framework inspired by principles from the education literature, such as the zone of proximal development and the testing effect. We utilize tests accordingly to close to the loop and maintain a novel particle filter model of human beliefs throughout the learning process, allowing us to provide demonstrations that are targeted to the human's current understanding in real time. A user study finds that our proposed closed-loop teaching framework reduces the regret in human test responses by 43% over a baseline.