A Simple and Plug-and-play Method for Unsupervised Sentence Representation Enhancement

TL;DR

This paper introduces RepAL, a simple, training-free post-processing method that enhances sentence embeddings by reducing redundancy, improving their effectiveness for semantic matching and retrieval tasks.

Contribution

RepAL is a novel, plug-and-play, unsupervised post-processing technique that enhances sentence representations without additional training.

Findings

RepAL improves sentence embedding quality across multiple models.

It is a training-free, easy-to-apply method.

RepAL consistently boosts performance in semantic tasks.

Abstract

Generating proper embedding of sentences through an unsupervised way is beneficial to semantic matching and retrieval problems in real-world scenarios. This paper presents Representation ALchemy (RepAL), an extremely simple post-processing method that enhances sentence representations. The basic idea in RepAL is to de-emphasize redundant information of sentence embedding generated by pre-trained models. Through comprehensive experiments, we show that RepAL is free of training and is a plug-and-play method that can be combined with most existing unsupervised sentence learning models. We also conducted in-depth analysis to understand RepAL.