Generated Knowledge Prompting for Commonsense Reasoning

TL;DR

This paper introduces generated knowledge prompting, a method that uses language models to generate and incorporate external knowledge into reasoning tasks, significantly enhancing performance on multiple commonsense benchmarks without task-specific supervision.

Contribution

It proposes a novel knowledge prompting approach that leverages large language models for external knowledge generation, improving commonsense reasoning without structured knowledge bases.

Findings

Achieved state-of-the-art results on NumerSense, CommonsenseQA 2.0, and QASC benchmarks.

Demonstrated that generated knowledge improves reasoning performance.

Showed flexibility of large language models as external knowledge sources.

Abstract

It remains an open question whether incorporating external knowledge benefits commonsense reasoning while maintaining the flexibility of pretrained sequence models. To investigate this question, we develop generated knowledge prompting, which consists of generating knowledge from a language model, then providing the knowledge as additional input when answering a question. Our method does not require task-specific supervision for knowledge integration, or access to a structured knowledge base, yet it improves performance of large-scale, state-of-the-art models on four commonsense reasoning tasks, achieving state-of-the-art results on numerical commonsense (NumerSense), general commonsense (CommonsenseQA 2.0), and scientific commonsense (QASC) benchmarks. Generated knowledge prompting highlights large-scale language models as flexible sources of external knowledge for improving commonsense reasoning. Our code is available at https://github.com/liujch1998/GKP