Enhancing Symbolic Machine Learning by Subsymbolic Representations

TL;DR

This paper proposes a method to improve symbolic machine learning by integrating neural embeddings, demonstrating superior performance in real-world domains and suggesting broad applicability beyond the tested scenarios.

Contribution

The paper introduces a novel approach to enhance symbolic learners with neural embeddings, improving efficiency and effectiveness in discriminative tasks.

Findings

Outperforms baseline methods in F1 score across three real-world domains

Embedding-based enhancement is simple yet effective

Potential extensions include analogy and instance similarity applications

Abstract

The goal of neuro-symbolic AI is to integrate symbolic and subsymbolic AI approaches, to overcome the limitations of either. Prominent systems include Logic Tensor Networks (LTN) or DeepProbLog, which offer neural predicates and end-to-end learning. The versatility of systems like LTNs and DeepProbLog, however, makes them less efficient in simpler settings, for instance, for discriminative machine learning, in particular in domains with many constants. Therefore, we follow a different approach: We propose to enhance symbolic machine learning schemes by giving them access to neural embeddings. In the present paper, we show this for TILDE and embeddings of constants used by TILDE in similarity predicates. The approach can be fine-tuned by further refining the embeddings depending on the symbolic theory. In experiments in three real-world domain, we show that this simple, yet effective, approach outperforms all other baseline methods in terms of the F1 score. The approach could be useful beyond this setting: Enhancing symbolic learners in this way could be extended to similarities between instances (effectively working like kernels within a logical language), for analogical reasoning, or for propositionalization.