A Syllable-based Technique for Word Embeddings of Korean Words

TL;DR

This paper introduces a syllable-based neural network model for Korean word embeddings that captures morphological features and addresses out-of-vocabulary issues more effectively than traditional models.

Contribution

It presents a novel convolutional neural network approach that constructs Korean word embeddings from syllable vectors, improving morphological representation and OOV robustness.

Findings

Produces morphologically meaningful Korean word embeddings

Demonstrates robustness to out-of-vocabulary words

Outperforms traditional Skip-gram embeddings in Korean

Abstract

Word embedding has become a fundamental component to many NLP tasks such as named entity recognition and machine translation. However, popular models that learn such embeddings are unaware of the morphology of words, so it is not directly applicable to highly agglutinative languages such as Korean. We propose a syllable-based learning model for Korean using a convolutional neural network, in which word representation is composed of trained syllable vectors. Our model successfully produces morphologically meaningful representation of Korean words compared to the original Skip-gram embeddings. The results also show that it is quite robust to the Out-of-Vocabulary problem.