Hebrew Diacritics Restoration using Visual Representation

TL;DR

This paper introduces DiVRit, a novel Hebrew diacritization system that uses visual language models to process diacritized candidates as images, achieving high accuracy without relying on explicit linguistic analysis.

Contribution

The work presents a new zero-shot classification approach for Hebrew diacritization using visual language models to embed diacritics directly in vector representations.

Findings

High accuracy in oracle settings with correct diacritized forms among candidates

Effective diacritization without complex linguistic analysis

Significant improvements through architectural and training optimizations

Abstract

Diacritics restoration in Hebrew is a fundamental task for ensuring accurate word pronunciation and disambiguating textual meaning. Despite the language's high degree of ambiguity when unvocalized, recent machine learning approaches have significantly advanced performance on this task. In this work, we present DiVRit, a novel system for Hebrew diacritization that frames the task as a zero-shot classification problem. Our approach operates at the word level, selecting the most appropriate diacritization pattern for each undiacritized word from a dynamically generated candidate set, conditioned on the surrounding textual context. A key innovation of DiVRit is its use of a Hebrew Visual Language Model to process diacritized candidates as images, allowing diacritic information to be embedded directly within their vector representations while the surrounding context remains tokenization-based. Through a comprehensive evaluation across various configurations, we demonstrate that the system effectively performs diacritization without relying on complex, explicit linguistic analysis. Notably, in an ``oracle'' setting where the correct diacritized form is guaranteed to be among the provided candidates, DiVRit achieves a high level of accuracy. Furthermore, strategic architectural enhancements and optimized training methodologies yield significant improvements in the system's overall generalization capabilities. These findings highlight the promising potential of visual representations for accurate and automated Hebrew diacritization.