Arctic-Embed: Scalable, Efficient, and Accurate Text Embedding Models

TL;DR

Arctic-Embed introduces a family of scalable, open-source text embedding models that achieve state-of-the-art retrieval accuracy across various sizes, outperforming some proprietary models.

Contribution

The paper presents a new set of open-source, efficient text embedding models with detailed training recipes and ablation studies, achieving top retrieval performance.

Findings

Models range from 22 to 334 million parameters.

Largest model outperforms some proprietary embedding models.

Training recipe and ablation studies explain performance gains.

Abstract

This report describes the training dataset creation and recipe behind the family of \texttt{arctic-embed} text embedding models (a set of five models ranging from 22 to 334 million parameters with weights open-sourced under an Apache-2 license). At the time of their release, each model achieved state-of-the-art retrieval accuracy for models of their size on the MTEB Retrieval leaderboard, with the largest model, arctic-embed-l outperforming closed source embedding models such as Cohere's embed-v3 and Open AI's text-embed-3-large. In addition to the details of our training recipe, we have provided several informative ablation studies, which we believe are the cause of our model performance.