Explaining Transition Systems through Program Induction

TL;DR

This paper introduces the $$-machine, a novel architecture that learns interpretable LISP-like programs from data traces, enabling better understanding of dynamical systems, AI agent behavior, and human-robot interactions.

Contribution

The paper presents the $$-machine, a new program induction method using backpropagation and A* search for extracting high-level models from observed data.

Findings

Efficient induction of interpretable programs from data traces.

Successful application to dynamical systems and AI behavior explanation.

Demonstrated effectiveness in human-robot interaction scenarios.

Abstract

Explaining and reasoning about processes which underlie observed black-box phenomena enables the discovery of causal mechanisms, derivation of suitable abstract representations and the formulation of more robust predictions. We propose to learn high level functional programs in order to represent abstract models which capture the invariant structure in the observed data. We introduce the $\pi$-machine (program-induction machine) -- an architecture able to induce interpretable LISP-like programs from observed data traces. We propose an optimisation procedure for program learning based on backpropagation, gradient descent and A* search. We apply the proposed method to three problems: system identification of dynamical systems, explaining the behaviour of a DQN agent and learning by demonstration in a human-robot interaction scenario. Our experimental results show that the $\pi$-machine can efficiently induce interpretable programs from individual data traces.