Knowledge, Rules and Their Embeddings: Two Paths towards Neuro-Symbolic JEPA

TL;DR

This paper introduces a neuro-symbolic framework that combines statistical learning with logical reasoning, enabling more interpretable and robust AI systems through bidirectional rule embedding and continuous rule discovery.

Contribution

It proposes RiJEPA, a novel bidirectional neuro-symbolic architecture that integrates structured inductive biases and differentiable logic for improved rule learning and inference.

Findings

Effective rule embedding via Energy-Based Constraints

Continuous rule discovery using gradient-guided Langevin diffusion

Successful application to synthetic and clinical data

Abstract

Modern self-supervised predictive architectures excel at capturing complex statistical correlations from high-dimensional data but lack mechanisms to internalize verifiable human logic, leaving them susceptible to spurious correlations and shortcut learning. Conversely, traditional rule-based inference systems offer rigorous, interpretable logic but suffer from discrete boundaries and NP-hard combinatorial explosion. To bridge this divide, we propose a bidirectional neuro-symbolic framework centered around Rule-informed Joint-Embedding Predictive Architectures (RiJEPA). In the first direction, we inject structured inductive biases into JEPA training via Energy-Based Constraints (EBC) and a multi-modal dual-encoder architecture. This fundamentally reshapes the representation manifold, replacing arbitrary statistical correlations with geometrically sound logical basins. In the second direction, we demonstrate that by relaxing rigid, discrete symbolic rules into a continuous, differentiable logic, we can bypass traditional combinatorial search for new rule generation. By leveraging gradient-guided Langevin diffusion within the rule energy landscape, we introduce novel paradigms for continuous rule discovery, which enable unconditional joint generation, conditional forward and abductive inference, and marginal predictive translation. Empirical evaluations on both synthetic topological simulations and a high-stakes clinical use case confirm the efficacy of our approach. Ultimately, this framework establishes a powerful foundation for robust, generative, and interpretable neuro-symbolic representation learning.