Jina-ColBERT-v2: A General-Purpose Multilingual Late Interaction Retriever

TL;DR

Jina-ColBERT-v2 enhances multilingual information retrieval by improving the ColBERT architecture and training, achieving strong performance across diverse languages with efficient inference.

Contribution

It introduces incremental architectural and training improvements to ColBERT, tailored for multilingual data and efficiency, advancing the state-of-the-art in dense retrieval models.

Findings

Strong performance on multilingual retrieval tasks

Improved efficiency with minimal tradeoffs

Effective handling of heterogeneous multilingual data

Abstract

Multi-vector dense models, such as ColBERT, have proven highly effective in information retrieval. ColBERT's late interaction scoring approximates the joint query-document attention seen in cross-encoders while maintaining inference efficiency closer to traditional dense retrieval models, thanks to its bi-encoder architecture and recent optimizations in indexing and search. In this work we propose a number of incremental improvements to the ColBERT model architecture and training pipeline, using methods shown to work in the more mature single-vector embedding model training paradigm, particularly those that apply to heterogeneous multilingual data or boost efficiency with little tradeoff. Our new model, Jina-ColBERT-v2, demonstrates strong performance across a range of English and multilingual retrieval tasks.