Bilinear Fusion of Commonsense Knowledge with Attention-Based NLI Models

TL;DR

This paper introduces BiCAM, a neural framework that effectively incorporates real-world commonsense knowledge into NLI models, improving their accuracy across multiple datasets and knowledge sources.

Contribution

BiCAM is a novel, model-independent neural framework that generalizes commonsense knowledge integration into NLI models using bilinear feature fusion.

Findings

BiCAM significantly improves NLI accuracy on SNLI and SciTail datasets.

BiECAM, an instance of BiCAM, boosts SciTail accuracy by 7-8% with external knowledge.

The approach generalizes across different NLI models and knowledge sources.

Abstract

We consider the task of incorporating real-world commonsense knowledge into deep Natural Language Inference (NLI) models. Existing external knowledge incorporation methods are limited to lexical level knowledge and lack generalization across NLI models, datasets, and commonsense knowledge sources. To address these issues, we propose a novel NLI model-independent neural framework, BiCAM. BiCAM incorporates real-world commonsense knowledge into NLI models. Combined with convolutional feature detectors and bilinear feature fusion, BiCAM provides a conceptually simple mechanism that generalizes well. Quantitative evaluations with two state-of-the-art NLI baselines on SNLI and SciTail datasets in conjunction with ConceptNet and Aristo Tuple KGs show that BiCAM considerably improves the accuracy the incorporated NLI baselines. For example, our BiECAM model, an instance of BiCAM, on the challenging SciTail dataset, improves the accuracy of incorporated baselines by 7.0% with ConceptNet, and 8.0% with Aristo Tuple KG.