Policy Optimization with Sparse Global Contrastive Explanations

TL;DR

This paper introduces a reinforcement learning framework that optimizes policies by making minimal, interpretable changes constrained by sparse, global contrastive explanations, demonstrated in discrete and continuous domains.

Contribution

It presents a novel RL approach that incorporates sparse, global contrastive explanations to guide minimal policy modifications for improved interpretability.

Findings

Effective policy improvements with minimal changes

Demonstrated in discrete MDP and continuous navigation domains

Framework enhances interpretability of policy updates

Abstract

We develop a Reinforcement Learning (RL) framework for improving an existing behavior policy via sparse, user-interpretable changes. Our goal is to make minimal changes while gaining as much benefit as possible. We define a minimal change as having a sparse, global contrastive explanation between the original and proposed policy. We improve the current policy with the constraint of keeping that global contrastive explanation short. We demonstrate our framework with a discrete MDP and a continuous 2D navigation domain.