Exploiting Language Instructions for Interpretable and Compositional Reinforcement Learning

TL;DR

This paper introduces a method to interpret and improve the compositionality of reinforcement learning agents by analyzing their latent space with a diagnostic classifier, balancing interpretability and performance on novel instructions.

Contribution

It demonstrates that using a diagnostic classifier enhances interpretability of RL agents' objectives and explores the trade-offs with zero-shot generalization.

Findings

Classification improves interpretability of hidden states.

Interpretable models show a performance shift on novel instructions.

Limited supervision reduces but does not eliminate interpretability benefits.

Abstract

In this work, we present an alternative approach to making an agent compositional through the use of a diagnostic classifier. Because of the need for explainable agents in automated decision processes, we attempt to interpret the latent space from an RL agent to identify its current objective in a complex language instruction. Results show that the classification process causes changes in the hidden states which makes them more easily interpretable, but also causes a shift in zero-shot performance to novel instructions. Lastly, we limit the supervisory signal on the classification, and observe a similar but less notable effect.