Three Pathways to Neurosymbolic Reinforcement Learning with Interpretable Model and Policy Networks

TL;DR

This paper explores three methods for creating interpretable, differentiable models in neurosymbolic reinforcement learning, balancing logic, simulation, and learning to enhance interpretability and learnability.

Contribution

It introduces three pathways for implementing neurosymbolic models that combine interpretability with differentiability in reinforcement learning settings.

Findings

Differentiability aids learning but reduces interpretability.

Mapping simulation data to logical predicates is complex.

Trade-offs exist between learnability and interpretability.

Abstract

Neurosymbolic AI combines the interpretability, parsimony, and explicit reasoning of classical symbolic approaches with the statistical learning of data-driven neural approaches. Models and policies that are simultaneously differentiable and interpretable may be key enablers of this marriage. This paper demonstrates three pathways to implementing such models and policies in a real-world reinforcement learning setting. Specifically, we study a broad class of neural networks that build interpretable semantics directly into their architecture. We reveal and highlight both the potential and the essential difficulties of combining logic, simulation, and learning. One lesson is that learning benefits from continuity and differentiability, but classical logic is discrete and non-differentiable. The relaxation to real-valued, differentiable representations presents a trade-off; the more learnable, the less interpretable. Another lesson is that using logic in the context of a numerical simulation involves a non-trivial mapping from raw (e.g., real-valued time series) simulation data to logical predicates. Some open questions this note exposes include: What are the limits of rule-based controllers, and how learnable are they? Do the differentiable interpretable approaches discussed here scale to large, complex, uncertain systems? Can we truly achieve interpretability? We highlight these and other themes across the three approaches.