Project SHADOW: Symbolic Higher-order Associative Deductive reasoning On Wikidata using LM probing

TL;DR

This paper presents SHADOW, a fine-tuned language model designed for symbolic higher-order reasoning, which significantly improves Wikidata triple completion performance by 20% over baseline methods.

Contribution

The paper introduces SHADOW, a novel language model fine-tuned for associative deductive reasoning on knowledge bases, advancing the capabilities of LM-based knowledge extraction.

Findings

SHADOW achieves an F1 score of 68.72% on Wikidata triple completion.

Outperforms baseline solutions by 20% in the LM-KBC 2024 challenge.

Demonstrates the effectiveness of symbolic reasoning in language models.

Abstract

We introduce SHADOW, a fine-tuned language model trained on an intermediate task using associative deductive reasoning, and measure its performance on a knowledge base construction task using Wikidata triple completion. We evaluate SHADOW on the LM-KBC 2024 challenge and show that it outperforms the baseline solution by 20% with a F1 score of 68.72%.