A Foundation Model for Zero-Shot Logical Rule Induction

TL;DR

This paper introduces Neural Rule Inducer (NRI), a pretrained model capable of zero-shot logical rule induction using domain-agnostic statistical representations, enabling generalization without retraining.

Contribution

The work presents a novel foundation model for zero-shot logical rule induction that leverages statistical encoding and differentiable rule execution, advancing symbolic reasoning capabilities.

Findings

NRI achieves effective rule recovery and zero-shot transfer.

The model demonstrates robustness to label noise and spurious correlations.

Code and checkpoints are publicly available at the provided GitHub link.

Abstract

Inductive Logic Programming (ILP) learns interpretable logical rules from data. Existing methods are transductive: their learned parameters are bound to specific predicates and require retraining for each new task. We introduce Neural Rule Inducer (NRI), a pretrained model for zero-shot rule induction. Rather than encoding literal identities, NRI represents literals using domain-agnostic statistical properties such as class-conditional rates, entropy, and co-occurrence, which generalize across variable identities and counts without retraining. The model consists of a statistical encoder and a parallel slot-based decoder. Parallel decoding preserves the permutation invariance of logical disjunction; an autoregressive decoder would instead impose an arbitrary clause order. Product T-norm relaxation makes rule execution differentiable, allowing end-to-end training on prediction accuracy alone. We evaluate NRI on rule recovery, robustness to label noise and spurious correlations, and zero-shot transfer to real-world benchmarks, and we believe this work opens up the possibility of foundation models for symbolic reasoning. Code and the reference checkpoint are available at https://github.com/phuayj/neural-rule-inducer.