Learning Symbolic Persistent Macro-Actions for POMDP Solving Over Time

TL;DR

This paper introduces a method combining temporal logic and POMDPs to create persistent macro-actions that improve decision-making interpretability and efficiency, demonstrated on benchmark problems with significant computational gains.

Contribution

It presents a novel approach to learning persistent macro-actions using temporal logic and ILP, reducing reliance on manual heuristics and enhancing POMDP solving over time.

Findings

Macro-actions improve decision interpretability and robustness.

Significant reduction in inference time in benchmarks.

Enhanced expressiveness and generality of learned macro-actions.

Abstract

This paper proposes an integration of temporal logical reasoning and Partially Observable Markov Decision Processes (POMDPs) to achieve interpretable decision-making under uncertainty with macro-actions. Our method leverages a fragment of Linear Temporal Logic (LTL) based on Event Calculus (EC) to generate \emph{persistent} (i.e., constant) macro-actions, which guide Monte Carlo Tree Search (MCTS)-based POMDP solvers over a time horizon, significantly reducing inference time while ensuring robust performance. Such macro-actions are learnt via Inductive Logic Programming (ILP) from a few traces of execution (belief-action pairs), thus eliminating the need for manually designed heuristics and requiring only the specification of the POMDP transition model. In the Pocman and Rocksample benchmark scenarios, our learned macro-actions demonstrate increased expressiveness and generality when compared to time-independent heuristics, indeed offering substantial computational efficiency improvements.