Control Synthesis in Partially Observable Environments for Complex Perception-Related Objectives

TL;DR

This paper develops a method for synthesizing optimal policies for complex perception-based tasks in partially observable environments, using a new logic and a scalable Monte Carlo Tree Search approach, demonstrated on a drone case study.

Contribution

It introduces sc-iLTL for specifying complex perception objectives and proposes a scalable MCTS-based control synthesis method for POMDPs.

Findings

Successfully transforms complex perception objectives into reachability problems.

MCTS converges in probability to optimal policies.

Demonstrates effectiveness on a drone-probing case study.

Abstract

Perception-related tasks often arise in autonomous systems operating under partial observability. This work studies the problem of synthesizing optimal policies for complex perception-related objectives in environments modeled by partially observable Markov decision processes. To formally specify such objectives, we introduce \emph{co-safe linear inequality temporal logic} (sc-iLTL), which can define complex tasks that are formed by the logical concatenation of atomic propositions as linear inequalities on the belief space of the POMDPs. Our solution to the control synthesis problem is to transform the \mbox{sc-iLTL} objectives into reachability objectives by constructing the product of the belief MDP and a deterministic finite automaton built from the sc-iLTL objective. To overcome the scalability challenge due to the product, we introduce a Monte Carlo Tree Search (MCTS) method that converges in probability to the optimal policy. Finally, a drone-probing case study demonstrates the applicability of our method.