Improving Diversity of Commonsense Generation by Large Language Models via In-Context Learning

TL;DR

This paper introduces a simple method to enhance the diversity of large language model outputs in commonsense reasoning tasks, maintaining quality and improving the range of knowledge used.

Contribution

The paper presents a novel approach to diversify LLM-generated sentences in commonsense reasoning without sacrificing quality, and demonstrates its effectiveness on benchmark datasets.

Findings

Achieves a balance between diversity and quality in LLM outputs

Generated sentences can improve diversity in existing commonsense generators

Method outperforms baseline approaches in benchmark tests

Abstract

Generative Commonsense Reasoning (GCR) requires a model to reason about a situation using commonsense knowledge, while generating coherent sentences. Although the quality of the generated sentences is crucial, the diversity of the generation is equally important because it reflects the model's ability to use a range of commonsense knowledge facts. Large Language Models (LLMs) have shown proficiency in enhancing the generation quality across various tasks through in-context learning (ICL) using given examples without the need for any fine-tuning. However, the diversity aspect in LLM outputs has not been systematically studied before. To address this, we propose a simple method that diversifies the LLM generations, while preserving their quality. Experimental results on three benchmark GCR datasets show that our method achieves an ideal balance between the quality and diversity. Moreover, the sentences generated by our proposed method can be used as training data to improve diversity in existing commonsense generators.