Nomic Embed: Training a Reproducible Long Context Text Embedder

TL;DR

This paper introduces nomic-embed-text-v1, a fully reproducible open-source long-context text embedding model that surpasses existing models on multiple benchmarks, with complete training data and code released for full transparency.

Contribution

It presents the first fully reproducible open-source long-context text embedding model with open data, code, and weights, outperforming proprietary models on key benchmarks.

Findings

Outperforms OpenAI Ada-002 on benchmarks

Supports 8192 token context length

Fully reproducible with open data and code

Abstract

This technical report describes the training of nomic-embed-text-v1, the first fully reproducible, open-source, open-weights, open-data, 8192 context length English text embedding model that outperforms both OpenAI Ada-002 and OpenAI text-embedding-3-small on the short-context MTEB benchmark and the long context LoCo benchmark. We release the training code and model weights under an Apache 2.0 license. In contrast with other open-source models, we release the full curated training data and code that allows for full replication of nomic-embed-text-v1. You can find code and data to replicate the model at https://github.com/nomic-ai/contrastors.