Transfer Entropy in MDPs with Temporal Logic Specifications

TL;DR

This paper introduces a method for synthesizing policies in MDPs that minimize transfer entropy costs while satisfying high-level temporal logic specifications, demonstrated on a Mars rover navigation scenario.

Contribution

It develops a novel approach combining transfer entropy minimization with temporal logic constraints in MDPs, solved via a modified Arimoto-Blahut algorithm.

Findings

Successfully synthesized policies for Mars rover navigation.

Effectively minimized transfer entropy while satisfying specifications.

Demonstrated practical applicability in autonomous navigation scenarios.

Abstract

Emerging applications in autonomy require control techniques that take into account uncertain environments, communication and sensing constraints, while satisfying highlevel mission specifications. Motivated by this need, we consider a class of Markov decision processes (MDPs), along with a transfer entropy cost function. In this context, we study highlevel mission specifications as co-safe linear temporal logic (LTL) formulae. We provide a method to synthesize a policy that minimizes the weighted sum of the transfer entropy and the probability of failure to satisfy the specification. We derive a set of coupled non-linear equations that an optimal policy must satisfy. We then use a modified Arimoto-Blahut algorithm to solve the non-linear equations. Finally, we demonstrated the proposed method on a navigation and path planning scenario of a Mars rover.