Curriculum-Based Self-Training Makes Better Few-Shot Learners for Data-to-Text Generation

TL;DR

This paper introduces a curriculum-based self-training method that improves few-shot data-to-text generation by effectively leveraging unlabeled data and modeling the relationship between structured data and text.

Contribution

The paper proposes Curriculum-Based Self-Training (CBST), a novel approach that enhances few-shot learning in data-to-text generation by ordering unlabeled data based on difficulty.

Findings

CBST outperforms fine-tuning and task-adaptive pre-training methods.

Achieves state-of-the-art results in few-shot data-to-text generation.

Effectively leverages unlabeled data through curriculum learning.

Abstract

Despite the success of text-to-text pre-trained models in various natural language generation (NLG) tasks, the generation performance is largely restricted by the number of labeled data in downstream tasks, particularly in data-to-text generation tasks. Existing works mostly utilize abundant unlabeled structured data to conduct unsupervised pre-training for task adaption, which fail to model the complex relationship between source structured data and target texts. Thus, we introduce self-training as a better few-shot learner than task-adaptive pre-training, which explicitly captures this relationship via pseudo-labeled data generated by the pre-trained model. To alleviate the side-effect of low-quality pseudo-labeled data during self-training, we propose a novel method called Curriculum-Based Self-Training (CBST) to effectively leverage unlabeled data in a rearranged order determined by the difficulty of text generation. Experimental results show that our method can outperform fine-tuning and task-adaptive pre-training methods, and achieve state-of-the-art performance in the few-shot setting of data-to-text generation.