HNCSE: Advancing Sentence Embeddings via Hybrid Contrastive Learning with Hard Negatives

TL;DR

HNCSE introduces a novel contrastive learning framework that leverages hard negative samples to improve sentence embeddings, demonstrating superior performance on semantic similarity and transfer tasks.

Contribution

It extends SimCSE by effectively incorporating hard negatives, enhancing the semantic richness of sentence representations in unsupervised learning.

Findings

Outperforms existing methods on semantic textual similarity tasks.

Shows improved transfer learning performance.

Validates the effectiveness of hard negatives in sentence embedding.

Abstract

Unsupervised sentence representation learning remains a critical challenge in modern natural language processing (NLP) research. Recently, contrastive learning techniques have achieved significant success in addressing this issue by effectively capturing textual semantics. Many such approaches prioritize the optimization using negative samples. In fields such as computer vision, hard negative samples (samples that are close to the decision boundary and thus more difficult to distinguish) have been shown to enhance representation learning. However, adapting hard negatives to contrastive sentence learning is complex due to the intricate syntactic and semantic details of text. To address this problem, we propose HNCSE, a novel contrastive learning framework that extends the leading SimCSE approach. The hallmark of HNCSE is its innovative use of hard negative samples to enhance the learning of both positive and negative samples, thereby achieving a deeper semantic understanding. Empirical tests on semantic textual similarity and transfer task datasets validate the superiority of HNCSE.