Few-Shot Bayesian Imitation Learning with Logical Program Policies

TL;DR

This paper introduces a Bayesian imitation learning approach that uses logical program policies, enabling efficient learning from very few demonstrations and generalizing well across diverse task instances.

Contribution

It presents a novel policy representation using logical combinations of programs, a probabilistic grammar prior, and an approximate Bayesian inference algorithm for few-shot learning.

Findings

Achieves 20-1,000x data efficiency over convolutional policies.

Generalizes effectively to new game instances with limited demonstrations.

Much more computationally efficient than vanilla program induction.

Abstract

Humans can learn many novel tasks from a very small number (1--5) of demonstrations, in stark contrast to the data requirements of nearly tabula rasa deep learning methods. We propose an expressive class of policies, a strong but general prior, and a learning algorithm that, together, can learn interesting policies from very few examples. We represent policies as logical combinations of programs drawn from a domain-specific language (DSL), define a prior over policies with a probabilistic grammar, and derive an approximate Bayesian inference algorithm to learn policies from demonstrations. In experiments, we study five strategy games played on a 2D grid with one shared DSL. After a few demonstrations of each game, the inferred policies generalize to new game instances that differ substantially from the demonstrations. Our policy learning is 20--1,000x more data efficient than convolutional and fully convolutional policy learning and many orders of magnitude more computationally efficient than vanilla program induction. We argue that the proposed method is an apt choice for tasks that have scarce training data and feature significant, structured variation between task instances.