Retrieval Enhanced Model for Commonsense Generation

TL;DR

This paper introduces a retrieval-augmented framework for commonsense generation that improves performance by retrieving prototype sentences and using a trainable retriever, setting new state-of-the-art results on the CommonGen benchmark.

Contribution

It proposes a novel retrieval-based approach to enhance both pre-training and fine-tuning in commonsense generation tasks.

Findings

Achieves new state-of-the-art results on the CommonGen benchmark.

Retrieval-augmented method improves reasoning and generalization.

Utilizing prototype sentences boosts generation quality.

Abstract

Commonsense generation is a challenging task of generating a plausible sentence describing an everyday scenario using provided concepts. Its requirement of reasoning over commonsense knowledge and compositional generalization ability even puzzles strong pre-trained language generation models. We propose a novel framework using retrieval methods to enhance both the pre-training and fine-tuning for commonsense generation. We retrieve prototype sentence candidates by concept matching and use them as auxiliary input. For fine-tuning, we further boost its performance with a trainable sentence retriever. We demonstrate experimentally on the large-scale CommonGen benchmark that our approach achieves new state-of-the-art results.