The SelectGen Challenge: Finding the Best Training Samples for Few-Shot Neural Text Generation

TL;DR

This paper introduces the SelectGen Challenge, focusing on optimizing training sample selection strategies to improve few-shot neural text generation, aiming to enhance model performance and benchmarking accuracy.

Contribution

It highlights the importance of selection strategies over random sampling in few-shot text generation and encourages research on effective sample selection methods.

Findings

Selection strategies significantly impact generation quality

Random sampling is suboptimal for few-shot tasks

Benchmarking can be improved with better sample selection

Abstract

We propose a shared task on training instance selection for few-shot neural text generation. Large-scale pretrained language models have led to dramatic improvements in few-shot text generation. Nonetheless, almost all previous work simply applies random sampling to select the few-shot training instances. Little to no attention has been paid to the selection strategies and how they would affect model performance. The study of the selection strategy can help us to (1) make the most use of our annotation budget in downstream tasks and (2) better benchmark few-shot text generative models. We welcome submissions that present their selection strategies and the effects on the generation quality.