T-Norms Driven Loss Functions for Machine Learning

TL;DR

This paper introduces a theoretical framework for neural-symbolic learning using T-norms, unifying loss functions including cross-entropy, and demonstrates faster convergence in experiments.

Contribution

It provides a novel theoretical approach to derive loss functions from T-norms, extending neural-symbolic learning capabilities beyond traditional methods.

Findings

Loss functions can be uniquely determined by T-norm generators.

The framework justifies the use of cross-entropy loss in neural-symbolic learning.

Experimental results show faster convergence rates than previous methods.

Abstract

Neural-symbolic approaches have recently gained popularity to inject prior knowledge into a learner without requiring it to induce this knowledge from data. These approaches can potentially learn competitive solutions with a significant reduction of the amount of supervised data. A large class of neural-symbolic approaches is based on First-Order Logic to represent prior knowledge, relaxed to a differentiable form using fuzzy logic. This paper shows that the loss function expressing these neural-symbolic learning tasks can be unambiguously determined given the selection of a t-norm generator. When restricted to supervised learning, the presented theoretical apparatus provides a clean justification to the popular cross-entropy loss, which has been shown to provide faster convergence and to reduce the vanishing gradient problem in very deep structures. However, the proposed learning formulation extends the advantages of the cross-entropy loss to the general knowledge that can be represented by a neural-symbolic method. Therefore, the methodology allows the development of a novel class of loss functions, which are shown in the experimental results to lead to faster convergence rates than the approaches previously proposed in the literature.