In Search of Trees: Decision-Tree Policy Synthesis for Black-Box Systems via Search

TL;DR

This paper introduces a novel search-based method for synthesizing optimal decision-tree control policies for black-box systems, providing guarantees on policy size and performance without requiring access to the system's internal model.

Contribution

It presents a specialized search algorithm with a trace-based pruning mechanism to efficiently find optimal decision-tree policies in black-box environments.

Findings

Efficiently synthesizes small, optimal decision trees for black-box systems.

Provides guarantees on policy size and optimality.

Reduces search space significantly with trace-based pruning.

Abstract

Decision trees, owing to their interpretability, are attractive as control policies for (dynamical) systems. Unfortunately, constructing, or synthesising, such policies is a challenging task. Previous approaches do so by imitating a neural-network policy, approximating a tabular policy obtained via formal synthesis, employing reinforcement learning, or modelling the problem as a mixed-integer linear program. However, these works may require access to a hard-to-obtain accurate policy or a formal model of the environment (within reach of formal synthesis), and may not provide guarantees on the quality or size of the final tree policy. In contrast, we present an approach to synthesise optimal decision-tree policies given a deterministic black-box environment and specification, a discretisation of the tree predicates, and an initial set of states, where optimality is defined with respect to the number of steps to achieve the goal. Our approach is a specialised search algorithm which systematically explores the (exponentially large) space of decision trees under the given discretisation. The key component is a novel trace-based pruning mechanism that significantly reduces the search space. Our approach represents a conceptually novel way of synthesising small decision-tree policies with optimality guarantees even for black-box environments with black-box specifications.