You Can Generate It Again: Data-to-Text Generation with Verification and Correction Prompting

TL;DR

This paper introduces Verification and Correction Prompting (VCP), a method that improves data-to-text generation in small language models by reducing keyword omission errors through feedback-driven verification and regeneration.

Contribution

The paper presents a novel VCP approach that enhances semantic fidelity in small language models for data-to-text tasks using multi-step verification and correction prompts.

Findings

VCP significantly reduces semantic error rates.

VCP maintains high text quality during correction.

The approach improves keyword coverage in generated texts.

Abstract

Small language models like T5 excel in generating high-quality text for data-to-text tasks, offering adaptability and cost-efficiency compared to Large Language Models (LLMs). However, they frequently miss keywords, which is considered one of the most severe and common errors in this task. In this work, we explore the potential of using feedback systems to enhance semantic fidelity in smaller language models for data-to-text generation tasks, through our Verification and Correction Prompting (VCP) approach. In the inference stage, our approach involves a multi-step process, including generation, verification, and regeneration stages. During the verification stage, we implement a simple rule to check for the presence of every keyword in the prediction. Recognizing that this rule can be inaccurate, we have developed a carefully designed training procedure, which enabling the model to incorporate feedback from the error-correcting prompt effectively, despite its potential inaccuracies. The VCP approach effectively reduces the Semantic Error Rate (SER) while maintaining the text's quality.