Neuro-Symbolic Reinforcement Learning with First-Order Logic

TL;DR

This paper introduces a neuro-symbolic reinforcement learning approach for text-based games that combines first-order logic with neural networks, leading to faster convergence and interpretability of policies.

Contribution

It presents a novel RL method using Logical Neural Networks to extract logical facts from text and external knowledge, improving convergence speed and interpretability.

Findings

Faster convergence compared to state-of-the-art methods

Enhanced interpretability of learned policies

Effective use of external knowledge sources like ConceptNet

Abstract

Deep reinforcement learning (RL) methods often require many trials before convergence, and no direct interpretability of trained policies is provided. In order to achieve fast convergence and interpretability for the policy in RL, we propose a novel RL method for text-based games with a recent neuro-symbolic framework called Logical Neural Network, which can learn symbolic and interpretable rules in their differentiable network. The method is first to extract first-order logical facts from text observation and external word meaning network (ConceptNet), then train a policy in the network with directly interpretable logical operators. Our experimental results show RL training with the proposed method converges significantly faster than other state-of-the-art neuro-symbolic methods in a TextWorld benchmark.