Negated Complementary Commonsense using Large Language Models

TL;DR

This paper investigates how large language models handle negated complementary questions in commonsense reasoning, proposing a methodology that significantly improves their performance in such challenging scenarios.

Contribution

It introduces a model-agnostic approach to enhance large language models' responses to negated complementary questions in commonsense tasks.

Findings

Outperforms GPT-3 few-shot generation by over 11 points.

Highlights importance of studying negated complementary questions.

Provides code, data, and experiments for further research.

Abstract

Larger language models, such as GPT-3, have shown to be excellent in many tasks. However, we demonstrate that out-of-ordinary questions can throw the model off guard. This work focuses on finding answers to negated complementary questions in commonsense scenarios. We illustrate how such questions adversely affect the model responses. We propose a model-agnostic methodology to improve the performance in negated complementary scenarios. Our method outperforms few-shot generation from GPT-3 (by more than 11 points) and, more importantly, highlights the significance of studying the response of large language models in negated complementary questions. The code, data, and experiments are available under: https://github.com/navidre/negated_complementary_commonsense.