NBF at SemEval-2025 Task 5: Light-Burst Attention Enhanced System for Multilingual Subject Recommendation

TL;DR

This paper introduces a multilingual subject recommendation system for SemEval 2025 that uses a lightweight self-attention mechanism with bilingual data and negative sampling, achieving competitive recall rates.

Contribution

The paper proposes a novel dimension-as-token self-attention method combined with bilingual training and negative sampling for cross-lingual subject classification.

Findings

Achieved an average recall of 32.24% in general settings.

Demonstrated effective encoding with reduced internal dimensions.

Performed well with minimal GPU resources.

Abstract

We present our system submission for SemEval 2025 Task 5, which focuses on cross-lingual subject classification in the English and German academic domains. Our approach leverages bilingual data during training, employing negative sampling and a margin-based retrieval objective. We demonstrate that a dimension-as-token self-attention mechanism designed with significantly reduced internal dimensions can effectively encode sentence embeddings for subject retrieval. In quantitative evaluation, our system achieved an average recall rate of 32.24% in the general quantitative setting (all subjects), 43.16% and 31.53% of the general qualitative evaluation methods with minimal GPU usage, highlighting their competitive performance. Our results demonstrate that our approach is effective in capturing relevant subject information under resource constraints, although there is still room for improvement.