Constrained Multi-Layer Contrastive Learning for Implicit Discourse Relationship Recognition

TL;DR

This paper introduces a constrained multi-layer contrastive learning method to improve implicit discourse relation recognition, effectively leveraging layer-specific contrastive losses to enhance classification accuracy.

Contribution

It proposes a novel constrained multi-layer contrastive learning approach that enforces hierarchical contrastive loss constraints, improving implicit discourse relation recognition performance.

Findings

Significant performance improvements on PDTB 2.0 and PDTB 3.0 datasets.

Effective enhancement of both multi-class and binary classification tasks.

Demonstrated superiority over existing neural network approaches.

Abstract

Previous approaches to the task of implicit discourse relation recognition (IDRR) generally view it as a classification task. Even with pre-trained language models, like BERT and RoBERTa, IDRR still relies on complicated neural networks with multiple intermediate layers to proper capture the interaction between two discourse units. As a result, the outputs of these intermediate layers may have different capability in discriminating instances of different classes. To this end, we propose to adapt a supervised contrastive learning (CL) method, label- and instance-centered CL, to enhance representation learning. Moreover, we propose a novel constrained multi-layer CL approach to properly impose a constraint that the contrastive loss of higher layers should be smaller than that of lower layers. Experimental results on PDTB 2.0 and PDTB 3.0 show that our approach can significantly improve the performance on both multi-class classification and binary classification.