Granite Embedding Models
Parul Awasthy, Aashka Trivedi, Yulong Li, Mihaela Bornea, David Cox,, Abraham Daniels, Martin Franz, Gabe Goodhart, Bhavani Iyer, Vishwajeet Kumar,, Luis Lastras, Scott McCarley, Rudra Murthy, Vignesh P, Sara Rosenthal, Salim, Roukos, Jaydeep Sen, Sukriti Sharma, Avirup Sil

TL;DR
The paper presents the Granite Embedding models, a family of encoder-based retrieval models with multilingual capabilities, that outperform similar-sized models on internal and benchmark retrieval tasks through advanced training techniques.
Contribution
Introduction of highly effective, multilingual encoder-based embedding models with efficient distilled versions, trained with retrieval-specific techniques for improved performance.
Findings
Models outperform publicly available counterparts on internal retrieval tasks.
Models achieve comparable performance to existing benchmarks.
Models are publicly available under open license for research and commercial use.
Abstract
We introduce the Granite Embedding models, a family of encoder-based embedding models designed for retrieval tasks, spanning dense-retrieval and sparse retrieval architectures, with both English and Multilingual capabilities. This report provides the technical details of training these highly effective 12 layer embedding models, along with their efficient 6 layer distilled counterparts. Extensive evaluations show that the models, developed with techniques like retrieval oriented pretraining, contrastive finetuning, knowledge distillation, and model merging significantly outperform publicly available models of similar sizes on both internal IBM retrieval and search tasks, and have equivalent performance on widely used information retrieval benchmarks, while being trained on high-quality data suitable for enterprise use. We publicly release all our Granite Embedding models under the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
- 🤗ibm-granite/granite-embedding-107m-multilingualmodel· 35k dl· ♡ 4935k dl♡ 49
- 🤗ibm-granite/granite-embedding-278m-multilingualmodel· 30k dl· ♡ 7830k dl♡ 78
- 🤗ibm-granite/granite-embedding-125m-englishmodel· 43k dl· ♡ 3543k dl♡ 35
- 🤗ibm-granite/granite-embedding-30m-englishmodel· 83k dl· ♡ 2983k dl♡ 29
- 🤗ibm-granite/granite-embedding-30m-sparsemodel· 61k dl· ♡ 2461k dl♡ 24
- 🤗MagicalAlchemist/granite-embedding-107m-id-en-v6.2model· 5 dl5 dl
- 🤗onnx-community/granite-embedding-30m-english-ONNXmodel· 231 dl231 dl
- 🤗raul3820/granite-30m-sparse-onnxmodel· 5 dl5 dl
- 🤗badmadrad/granite-embedding-125m-english-mlxmodel· 7 dl7 dl
- 🤗seerware/granite-embedding-30m-sparsemodel· 12 dl12 dl
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsInformation Retrieval and Search Behavior · Topic Modeling · Advanced Graph Neural Networks
