Semantic-aware Contrastive Learning for More Accurate Semantic Parsing

TL;DR

This paper introduces a semantic-aware contrastive learning method for semantic parsing that improves accuracy by distinguishing fine-grained meaning representations and considering sequence-level semantics, achieving state-of-the-art results.

Contribution

It proposes a novel contrastive learning algorithm with multi-level sampling and semantic-aware similarity functions for more accurate semantic parsing.

Findings

Significant performance improvements over MLE baselines.

Achieves state-of-the-art results on two datasets.

Effective in distinguishing fine-grained semantic representations.

Abstract

Since the meaning representations are detailed and accurate annotations which express fine-grained sequence-level semtantics, it is usually hard to train discriminative semantic parsers via Maximum Likelihood Estimation (MLE) in an autoregressive fashion. In this paper, we propose a semantic-aware contrastive learning algorithm, which can learn to distinguish fine-grained meaning representations and take the overall sequence-level semantic into consideration. Specifically, a multi-level online sampling algorithm is proposed to sample confusing and diverse instances. Three semantic-aware similarity functions are designed to accurately measure the distance between meaning representations as a whole. And a ranked contrastive loss is proposed to pull the representations of the semantic-identical instances together and push negative instances away. Experiments on two standard datasets show that our approach achieves significant improvements over MLE baselines and gets state-of-the-art performances by simply applying semantic-aware contrastive learning on a vanilla Seq2Seq model.