Reinforcement Learning in Partially Observable Markov Decision Processes using Hybrid Probabilistic Logic Programs

TL;DR

This paper introduces a probabilistic logic programming framework for reinforcement learning in POMDPs, integrating domain knowledge, with formal correctness proofs and complexity analysis, and proposes a new action language for factored POMDP representation.

Contribution

It presents a novel hybrid probabilistic logic programming approach for RL in POMDPs, including formal proofs, complexity results, and a new high-level action language.

Findings

NP-completeness of policy finding in the approach

Encoding RL problems as classical logic programs with answer set semantics

Encoding RL problems as SAT problems

Abstract

We present a probabilistic logic programming framework to reinforcement learning, by integrating reinforce-ment learning, in POMDP environments, with normal hybrid probabilistic logic programs with probabilistic answer set seman-tics, that is capable of representing domain-specific knowledge. We formally prove the correctness of our approach. We show that the complexity of finding a policy for a reinforcement learning problem in our approach is NP-complete. In addition, we show that any reinforcement learning problem can be encoded as a classical logic program with answer set semantics. We also show that a reinforcement learning problem can be encoded as a SAT problem. We present a new high level action description language that allows the factored representation of POMDP. Moreover, we modify the original model of POMDP so that it be able to distinguish between knowledge producing actions and actions that change the environment.