Inductive Synthesis of Finite-State Controllers for POMDPs

TL;DR

This paper introduces a new inductive synthesis framework for creating finite-state controllers for POMDPs, which is effective for indefinite-horizon tasks and can handle multi-objective specifications.

Contribution

The paper presents a novel, two-stage inductive synthesis approach for FSCs in POMDPs, improving over existing methods in size and multi-objective handling.

Findings

Competitive with belief-based methods for indefinite-horizon properties

Produces smaller FSCs than existing approaches

Handles multi-objective specifications naturally

Abstract

We present a novel learning framework to obtain finite-state controllers (FSCs) for partially observable Markov decision processes and illustrate its applicability for indefinite-horizon specifications. Our framework builds on oracle-guided inductive synthesis to explore a design space compactly representing available FSCs. The inductive synthesis approach consists of two stages: The outer stage determines the design space, i.e., the set of FSC candidates, while the inner stage efficiently explores the design space. This framework is easily generalisable and shows promising results when compared to existing approaches. Experiments indicate that our technique is (i) competitive to state-of-the-art belief-based approaches for indefinite-horizon properties, (ii) yields smaller FSCs than existing methods for several models, and (iii) naturally treats multi-objective specifications.