Determining Semantic Textual Similarity using Natural Deduction Proofs

TL;DR

This paper introduces a novel approach to measure semantic textual similarity by integrating shallow features with logical proof-based features derived from natural deduction proofs, improving accuracy over existing logic-based methods.

Contribution

The paper presents a new method combining shallow features with natural deduction proof features from CCG-based semantic representations for better textual similarity assessment.

Findings

Outperforms other logic-based systems in semantic similarity tasks.

Features from natural deduction proofs are effective for learning similarity.

Combines shallow and logical features for improved accuracy.

Abstract

Determining semantic textual similarity is a core research subject in natural language processing. Since vector-based models for sentence representation often use shallow information, capturing accurate semantics is difficult. By contrast, logical semantic representations capture deeper levels of sentence semantics, but their symbolic nature does not offer graded notions of textual similarity. We propose a method for determining semantic textual similarity by combining shallow features with features extracted from natural deduction proofs of bidirectional entailment relations between sentence pairs. For the natural deduction proofs, we use ccg2lambda, a higher-order automatic inference system, which converts Combinatory Categorial Grammar (CCG) derivation trees into semantic representations and conducts natural deduction proofs. Experiments show that our system was able to outperform other logic-based systems and that features derived from the proofs are effective for learning textual similarity.