Commonsense Knowledge with Negation: A Resource to Enhance Negation Understanding

TL;DR

This paper introduces a new resource of over 2 million negation-augmented commonsense knowledge triples and demonstrates that pre-training LLMs on this data improves their negation understanding.

Contribution

The authors created a large negation-augmented commonsense knowledge corpus and showed that pre-training on it enhances LLMs' ability to understand negation.

Findings

Pre-training on the new corpus improves negation understanding in LLMs.

The corpus contains over 2 million triples with if-then relations involving negation.

Negation-aware commonsense knowledge is challenging for current models.

Abstract

Negation is a common and important semantic feature in natural language, yet Large Language Models (LLMs) struggle when negation is involved in natural language understanding tasks. Commonsense knowledge, on the other hand, despite being a well-studied topic, lacks investigations involving negation. In this work, we show that commonsense knowledge with negation is challenging for models to understand. We present a novel approach to automatically augment existing commonsense knowledge corpora with negation, yielding two new corpora containing over 2M triples with if-then relations. In addition, pre-training LLMs on our corpora benefits negation understanding.