Approaches to Semantic Textual Similarity in Slovak Language: From Algorithms to Transformers

TL;DR

This paper evaluates various semantic textual similarity methods for Slovak, comparing traditional algorithms, machine learning models, and deep learning tools, highlighting their strengths and trade-offs.

Contribution

It provides a comprehensive comparison of STS approaches in Slovak, including novel use of optimization for feature selection and hyperparameter tuning.

Findings

Traditional algorithms have limited accuracy.

Deep learning models outperform classical methods.

Optimization improves machine learning model performance.

Abstract

Semantic textual similarity (STS) plays a crucial role in many natural language processing tasks. While extensively studied in high-resource languages, STS remains challenging for under-resourced languages such as Slovak. This paper presents a comparative evaluation of sentence-level STS methods applied to Slovak, including traditional algorithms, supervised machine learning models, and third-party deep learning tools. We trained several machine learning models using outputs from traditional algorithms as features, with feature selection and hyperparameter tuning jointly guided by artificial bee colony optimization. Finally, we evaluated several third-party tools, including fine-tuned model by CloudNLP, OpenAI's embedding models, GPT-4 model, and pretrained SlovakBERT model. Our findings highlight the trade-offs between different approaches.