Bootstrapping Generators from Noisy Data

TL;DR

This paper presents a method for bootstrapping data-to-text generators from large, loosely aligned datasets by using multi-instance learning to improve content selection and enhance generation quality.

Contribution

It introduces a novel content selection mechanism using multi-instance learning to better align data and text for training generators from noisy datasets.

Findings

Models with content-specific objectives outperform vanilla models.

Multi-instance learning effectively discovers data-text correspondences.

Enhanced models generate more accurate and relevant text from structured data.

Abstract

A core step in statistical data-to-text generation concerns learning correspondences between structured data representations (e.g., facts in a database) and associated texts. In this paper we aim to bootstrap generators from large scale datasets where the data (e.g., DBPedia facts) and related texts (e.g., Wikipedia abstracts) are loosely aligned. We tackle this challenging task by introducing a special-purpose content selection mechanism. We use multi-instance learning to automatically discover correspondences between data and text pairs and show how these can be used to enhance the content signal while training an encoder-decoder architecture. Experimental results demonstrate that models trained with content-specific objectives improve upon a vanilla encoder-decoder which solely relies on soft attention.