Effects of Pre- and Post-Processing on type-based Embeddings in Lexical Semantic Change Detection

TL;DR

This paper investigates how pre- and post-processing techniques can enhance type-based embeddings for lexical semantic change detection, addressing data limitations and improving model performance.

Contribution

It introduces optimized pre-training and post-processing methods specifically tailored for lexical semantic change detection models.

Findings

Pre-training on large corpora improves detection accuracy.

Post-processing transformations enhance model performance.

Guidelines for applying these techniques across different scenarios.

Abstract

Lexical semantic change detection is a new and innovative research field. The optimal fine-tuning of models including pre- and post-processing is largely unclear. We optimize existing models by (i) pre-training on large corpora and refining on diachronic target corpora tackling the notorious small data problem, and (ii) applying post-processing transformations that have been shown to improve performance on synchronic tasks. Our results provide a guide for the application and optimization of lexical semantic change detection models across various learning scenarios.