Bhav-Net: Knowledge Transfer for Cross-Lingual Antonym vs Synonym Distinction via Dual-Space Graph Transformers

TL;DR

Bhav-Net introduces a dual-space graph transformer architecture that effectively distinguishes antonyms from synonyms across multiple languages by transferring knowledge from complex multilingual models to simpler, language-specific models.

Contribution

This work presents Bhav-Net, a novel dual-space architecture combining language-specific BERT encoders with graph transformers for improved cross-lingual antonym-synonym distinction.

Findings

Effective knowledge transfer across eight languages.

Competitive performance against state-of-the-art baselines.

Interpretable semantic representations and cross-lingual generalization.

Abstract

Antonym vs synonym distinction across multiple languages presents unique computational challenges due to the paradoxical nature of antonymous relationships words that share semantic domains while expressing opposite meanings. This work introduces Bhav-Net, a novel dual-space architecture that enables effective knowledge transfer from complex multilingual models to simpler, language-specific architectures while maintaining robust cross-lingual antonym--synonym distinction capabilities. Our approach combines language-specific BERT encoders with graph transformer networks, creating distinct semantic projections where synonymous pairs cluster in one space while antonymous pairs exhibit high similarity in a complementary space. Through comprehensive evaluation across eight languages (English, German, French, Spanish, Italian, Portuguese, Dutch, and Russian), we demonstrate that semantic relationship modeling transfers effectively across languages. The dual-encoder design achieves competitive performance against state-of-the-art baselines while providing interpretable semantic representations and effective cross-lingual generalization.