Piccolo2: General Text Embedding with Multi-task Hybrid Loss Training

TL;DR

Piccolo2 is a new text embedding model that achieves state-of-the-art performance across multiple tasks by using a multi-task hybrid loss training approach and scalable embedding dimensions.

Contribution

The paper introduces Piccolo2, a novel embedding model that combines multi-task hybrid loss training with scalable vector dimensions for improved performance.

Findings

Surpasses previous models on CMTEB benchmark

Supports flexible vector dimensions with MRL training

Achieves state-of-the-art results across 6 tasks

Abstract

In this report, we introduce Piccolo2, an embedding model that surpasses other models in the comprehensive evaluation over 6 tasks on CMTEB benchmark, setting a new state-of-the-art. Piccolo2 primarily leverages an efficient multi-task hybrid loss training approach, effectively harnessing textual data and labels from diverse downstream tasks. In addition, Piccolo2 scales up the embedding dimension and uses MRL training to support more flexible vector dimensions. The latest information of piccolo models can be accessed via: https://huggingface.co/sensenova/