Reduced Implication-bias Logic Loss for Neuro-Symbolic Learning

TL;DR

This paper identifies a bias in differentiable logic operators used in Neuro-Symbolic learning and proposes RILL, a method to reduce this Implication Bias, leading to improved performance especially with incomplete knowledge bases.

Contribution

The paper reveals the Implication Bias in fuzzy logic-based loss functions and introduces RILL, a novel method to mitigate this bias in Neuro-Symbolic systems.

Findings

RILL significantly improves performance over biased logic loss functions.

RILL maintains robustness with limited labeled data.

The method is especially effective with incomplete knowledge bases.

Abstract

Integrating logical reasoning and machine learning by approximating logical inference with differentiable operators is a widely used technique in Neuro-Symbolic systems. However, some differentiable operators could bring a significant bias during backpropagation and degrade the performance of Neuro-Symbolic learning. In this paper, we reveal that this bias, named \textit{Implication Bias} is common in loss functions derived from fuzzy logic operators. Furthermore, we propose a simple yet effective method to transform the biased loss functions into \textit{Reduced Implication-bias Logic Loss (RILL)} to address the above problem. Empirical study shows that RILL can achieve significant improvements compared with the biased logic loss functions, especially when the knowledge base is incomplete, and keeps more robust than the compared methods when labelled data is insufficient.