NEUROLOGIC: From Neural Representations to Interpretable Logic Rules

TL;DR

NEUROLOGIC introduces a scalable, flexible framework for extracting interpretable logical rules directly from deep neural networks, including complex architectures like Transformers, enhancing interpretability in domains such as NLP and computer vision.

Contribution

It enables direct rule extraction over neural representations at any layer, supporting richer logic and human priors, overcoming limitations of previous layerwise methods.

Findings

Successfully applied to Transformer-based sentiment analysis

Extracted meaningful, interpretable logic rules from neural models

Demonstrated scalability and broader applicability of the approach

Abstract

Rule-based explanation methods offer rigorous and globally interpretable insights into neural network behavior. However, existing approaches are mostly limited to small fully connected networks and depend on costly layerwise rule extraction and substitution processes. These limitations hinder their generalization to more complex architectures such as Transformers. Moreover, existing methods produce shallow, decision-tree-like rules that fail to capture rich, high-level abstractions in complex domains like computer vision and natural language processing. To address these challenges, we propose NEUROLOGIC, a novel framework that extracts interpretable logical rules directly from deep neural networks. Unlike previous methods, NEUROLOGIC can construct logic rules over hidden predicates derived from neural representations at any chosen layer, in contrast to costly layerwise extraction and rewriting. This flexibility enables broader architectural compatibility and improved scalability. Furthermore, NEUROLOGIC supports richer logical constructs and can incorporate human prior knowledge to ground hidden predicates back to the input space, enhancing interpretability. We validate NEUROLOGIC on Transformer-based sentiment analysis, demonstrating its ability to extract meaningful, interpretable logic rules and provide deeper insights-tasks where existing methods struggle to scale.