Introducing DictaLM -- A Large Generative Language Model for Modern Hebrew

TL;DR

DictaLM is a large-scale Hebrew language model with 7 billion parameters, designed to advance Hebrew NLP research and applications, including modern and Rabbinic Hebrew, by providing open foundation and instruct-tuned models.

Contribution

This paper introduces DictaLM, the first large Hebrew language model with open access, and a specialized Rabbinic Hebrew model, supporting diverse Hebrew NLP tasks.

Findings

Model achieves promising performance on Hebrew NLP tasks.

Open release fosters research and development in Hebrew language processing.

Supports multiple Hebrew dialects and historical forms.

Abstract

We present DictaLM, a large-scale language model tailored for Modern Hebrew. Boasting 7B parameters, this model is predominantly trained on Hebrew-centric data. As a commitment to promoting research and development in the Hebrew language, we release both the foundation model and the instruct-tuned model under a Creative Commons license. Concurrently, we introduce DictaLM-Rab, another foundation model geared towards Rabbinic/Historical Hebrew. These foundation models serve as ideal starting points for fine-tuning various Hebrew-specific tasks, such as instruction, Q&A, sentiment analysis, and more. This release represents a preliminary step, offering an initial Hebrew LLM model for the Hebrew NLP community to experiment with.