jina-embeddings-v4: Universal Embeddings for Multimodal Multilingual Retrieval

TL;DR

Jina-embeddings-v4 is a large multimodal embedding model that unifies text and image representations, supporting various retrieval tasks and excelling in processing visually rich content, with a new benchmark for evaluation.

Contribution

The paper introduces jina-embeddings-v4, a 3.8 billion parameter multimodal model with a novel architecture and task-specific adapters, achieving state-of-the-art results in multimodal retrieval.

Findings

Achieves state-of-the-art performance on multimodal retrieval tasks.

Excels in processing visually rich content like tables and diagrams.

Introduces Jina-VDR, a benchmark for visually rich image retrieval.

Abstract

We introduce jina-embeddings-v4, a 3.8 billion parameter multimodal embedding model that unifies text and image representations through a novel architecture supporting both single-vector and multi-vector embeddings in the late interaction style. The model incorporates task-specific Low-Rank Adaptation (LoRA) adapters to optimize performance across diverse retrieval scenarios, including query-document retrieval, semantic text similarity, and code search. Comprehensive evaluations demonstrate that jina-embeddings-v4 achieves state-of-the-art performance on both single-modal and cross-modal retrieval tasks, with particular strength in processing visually rich content such as tables, charts, diagrams, and mixed-media formats. To facilitate evaluation of this capability, we also introduce Jina-VDR, a novel benchmark specifically designed for visually rich image retrieval.