Bridging the Gap between Reinforcement Learning and Knowledge Representation: A Logical Off- and On-Policy Framework

TL;DR

This paper introduces a logical framework combining reinforcement learning with knowledge representation using logic programs, enabling complex domain reasoning and encoding RL problems as SAT problems.

Contribution

It presents a novel logic-based knowledge representation framework for reinforcement learning, proving correctness and complexity, and encoding RL problems as SAT.

Findings

The framework is capable of solving complex model-free RL problems.

Finding policies in this framework is NP-complete.

RL problems can be encoded as SAT problems within this framework.

Abstract

Knowledge Representation is important issue in reinforcement learning. In this paper, we bridge the gap between reinforcement learning and knowledge representation, by providing a rich knowledge representation framework, based on normal logic programs with answer set semantics, that is capable of solving model-free reinforcement learning problems for more complex do-mains and exploits the domain-specific knowledge. We prove the correctness of our approach. We show that the complexity of finding an offline and online policy for a model-free reinforcement learning problem in our approach is NP-complete. Moreover, we show that any model-free reinforcement learning problem in MDP environment can be encoded as a SAT problem. The importance of that is model-free reinforcement