AMR4NLI: Interpretable and robust NLI measures from semantic graphs

TL;DR

This paper introduces a method for natural language inference that uses semantic graphs and contextualized embeddings to explicitly and robustly determine entailment, enhancing interpretability and performance.

Contribution

It compares semantic structures and embeddings for NLI, demonstrating their complementary benefits and proposing a hybrid approach for improved interpretability and robustness.

Findings

Semantic graphs and embeddings provide complementary signals.

Hybrid models outperform individual methods in NLI tasks.

Semantic structures enhance interpretability of inference measures.

Abstract

The task of natural language inference (NLI) asks whether a given premise (expressed in NL) entails a given NL hypothesis. NLI benchmarks contain human ratings of entailment, but the meaning relationships driving these ratings are not formalized. Can the underlying sentence pair relationships be made more explicit in an interpretable yet robust fashion? We compare semantic structures to represent premise and hypothesis, including sets of contextualized embeddings and semantic graphs (Abstract Meaning Representations), and measure whether the hypothesis is a semantic substructure of the premise, utilizing interpretable metrics. Our evaluation on three English benchmarks finds value in both contextualized embeddings and semantic graphs; moreover, they provide complementary signals, and can be leveraged together in a hybrid model.