Inductive Generalization in Reinforcement Learning from Specifications

TL;DR

This paper introduces a new inductive generalization framework for reinforcement learning that uses logical specifications to enable zero-shot policy generation for related tasks, improving adaptability in complex environments.

Contribution

It proposes a novel method to learn a policy generator leveraging inductive structures, allowing zero-shot adaptation to unseen tasks in RL environments.

Findings

Successful generalization to unseen policies in control benchmarks

Effective zero-shot policy generation for long-horizon tasks

Demonstrated promise in complex RL environments

Abstract

We present a novel inductive generalization framework for RL from logical specifications. Many interesting tasks in RL environments have a natural inductive structure. These inductive tasks have similar overarching goals but they differ inductively in low-level predicates and distributions. We present a generalization procedure that leverages this inductive relationship to learn a higher-order function, a policy generator, that generates appropriately adapted policies for instances of an inductive task in a zero-shot manner. An evaluation of the proposed approach on a set of challenging control benchmarks demonstrates the promise of our framework in generalizing to unseen policies for long-horizon tasks.