PLOG: Table-to-Logic Pretraining for Logical Table-to-Text Generation

TL;DR

This paper introduces PLOG, a pretraining framework that enhances logical fidelity in table-to-text generation by learning from logical forms, significantly improving performance on logical fidelity benchmarks.

Contribution

Proposes a novel table-to-logic pretraining approach that improves logical inference and fidelity in table-to-text generation tasks.

Findings

PLOG outperforms baselines on LOGICNLG and CONTLOG datasets.

Pretraining on table-to-logic forms enhances logical fidelity.

Large-scale logical form collection improves model training.

Abstract

Logical table-to-text generation is a task that involves generating logically faithful sentences from tables, which requires models to derive logical level facts from table records via logical inference. It raises a new challenge on the logical-level content planning of table-to-text models. However, directly learning the logical inference knowledge from table-text pairs is very difficult for neural models because of the ambiguity of natural language and the scarcity of parallel data. Hence even large-scale pre-trained language models present low logical fidelity on logical table-to-text. In this work, we propose a PLOG (Pretrained Logical Form Generator) framework to improve the generation fidelity. Specifically, PLOG is first pretrained on a table-to-logic-form generation (table-to-logic) task, then finetuned on downstream table-to-text tasks. The formal definition of logical forms enables us to collect large amount of accurate logical forms from tables without human annotation. In addition, PLOG can learn logical inference from table-logic pairs much more definitely than from table-text pairs. To evaluate our model, we further collect a controlled logical table-to-text dataset CONTLOG based on an existing dataset. On two benchmarks, LOGICNLG and CONTLOG, PLOG outperforms strong baselines by a large margin on the logical fidelity, demonstrating the effectiveness of table-to-logic pretraining.