Improving Cross-task Generalization of Unified Table-to-text Models with Compositional Task Configurations

TL;DR

This paper introduces compositional task configurations with prompts to enhance the cross-task generalization of unified table-to-text models, enabling better knowledge sharing and zero-shot task adaptation.

Contribution

The paper proposes explicit prompt-based task configurations that specify task type and input-output formats, improving generalization and zero-shot performance in unified models.

Findings

Outperforms baseline in in-domain settings with +0.5 average improvement.

Achieves +12.6 average improvement in zero-shot settings.

Demonstrates effectiveness across ten table-to-text tasks.

Abstract

There has been great progress in unifying various table-to-text tasks using a single encoder-decoder model trained via multi-task learning (Xie et al., 2022). However, existing methods typically encode task information with a simple dataset name as a prefix to the encoder. This not only limits the effectiveness of multi-task learning, but also hinders the model's ability to generalize to new domains or tasks that were not seen during training, which is crucial for real-world applications. In this paper, we propose compositional task configurations, a set of prompts prepended to the encoder to improve cross-task generalization of unified models. We design the task configurations to explicitly specify the task type, as well as its input and output types. We show that this not only allows the model to better learn shared knowledge across different tasks at training, but also allows us to control the model by composing new configurations that apply novel input-output combinations in a zero-shot manner. We demonstrate via experiments over ten table-to-text tasks that our method outperforms the UnifiedSKG baseline by noticeable margins in both in-domain and zero-shot settings, with average improvements of +0.5 and +12.6 from using a T5-large backbone, respectively.