Linking-Enhanced Pre-Training for Table Semantic Parsing

TL;DR

This paper introduces a novel pre-training framework for table semantic parsing that emphasizes explicit question-schema interactions and schema-aware learning, significantly improving performance on benchmark datasets.

Contribution

It proposes two new pre-training objectives and a schema-aware curriculum learning approach to enhance table semantic parsing models.

Findings

Improved accuracy on Spider and SQUALL benchmarks.

Effective question-schema interaction modeling.

Enhanced learning efficiency with curriculum approach.

Abstract

Recently pre-training models have significantly improved the performance of various NLP tasks by leveraging large-scale text corpora to improve the contextual representation ability of the neural network. The large pre-training language model has also been applied in the area of table semantic parsing. However, existing pre-training approaches have not carefully explored explicit interaction relationships between a question and the corresponding database schema, which is a key ingredient for uncovering their semantic and structural correspondence. Furthermore, the question-aware representation learning in the schema grounding context has received less attention in pre-training objective.To alleviate these issues, this paper designs two novel pre-training objectives to impose the desired inductive bias into the learned representations for table pre-training. We further propose a schema-aware curriculum learning approach to mitigate the impact of noise and learn effectively from the pre-training data in an easy-to-hard manner. We evaluate our pre-trained framework by fine-tuning it on two benchmarks, Spider and SQUALL. The results demonstrate the effectiveness of our pre-training objective and curriculum compared to a variety of baselines.