Towards Table-to-Text Generation with Pretrained Language Model: A Table Structure Understanding and Text Deliberating Approach

TL;DR

This paper introduces TASD, a novel approach combining table structure understanding and text deliberation with pretrained language models to improve table-to-text generation, achieving more faithful and fluent descriptions.

Contribution

It proposes a three-layered multi-head attention network and multi-pass decoding framework to better leverage table structure and refine generated text in table-to-text tasks.

Findings

Enhanced text faithfulness and fluency demonstrated on public datasets.

Human evaluation confirms improved quality of generated descriptions.

Effective integration of table structure understanding into pretrained models.

Abstract

Although remarkable progress on the neural table-to-text methods has been made, the generalization issues hinder the applicability of these models due to the limited source tables. Large-scale pretrained language models sound like a promising solution to tackle such issues. However, how to effectively bridge the gap between the structured table and the text input by fully leveraging table information to fuel the pretrained model is still not well explored. Besides, another challenge of integrating the deliberation mechanism into the text-to-text pretrained model for solving the table-to-text task remains seldom studied. In this paper, to implement the table-to-text generation with pretrained language model, we propose a table structure understanding and text deliberating approach, namely TASD. Specifically, we devise a three-layered multi-head attention network to realize the table-structure-aware text generation model with the help of the pretrained language model. Furthermore, a multi-pass decoder framework is adopted to enhance the capability of polishing generated text for table descriptions. The empirical studies, as well as human evaluation, on two public datasets, validate that our approach can generate faithful and fluent descriptive texts for different types of tables.