Pair-Level Supervised Contrastive Learning for Natural Language Inference

TL;DR

This paper introduces PairSCL, a novel pair-level supervised contrastive learning method for natural language inference that improves sentence pair representations by jointly learning and distinguishing classes, leading to higher accuracy.

Contribution

The paper proposes a cross attention-based joint representation and a contrastive learning objective for sentence pairs, advancing NLI performance over existing sentence-level contrastive methods.

Findings

PairSCL outperforms other methods by 2.1% on NLI datasets.

Achieves state-of-the-art results on seven transfer text classification tasks.

Demonstrates the effectiveness of pair-level contrastive learning in NLI.

Abstract

Natural language inference (NLI) is an increasingly important task for natural language understanding, which requires one to infer the relationship between the sentence pair (premise and hypothesis). Many recent works have used contrastive learning by incorporating the relationship of the sentence pair from NLI datasets to learn sentence representation. However, these methods only focus on comparisons with sentence-level representations. In this paper, we propose a Pair-level Supervised Contrastive Learning approach (PairSCL). We adopt a cross attention module to learn the joint representations of the sentence pairs. A contrastive learning objective is designed to distinguish the varied classes of sentence pairs by pulling those in one class together and pushing apart the pairs in other classes. We evaluate PairSCL on two public datasets of NLI where the accuracy of PairSCL outperforms other methods by 2.1% on average. Furthermore, our method outperforms the previous state-of-the-art method on seven transfer tasks of text classification.