Robust and Consistent Estimation of Word Embedding for Bangla Language by fine-tuning Word2Vec Model

TL;DR

This paper fine-tunes the word2vec model to generate robust and consistent word embeddings for the Bangla language, optimizing hyperparameters through intrinsic and extrinsic evaluations.

Contribution

It identifies the most effective hyperparameters for Bangla word embeddings using word2vec, particularly the skip-gram method with 300 dimensions and window size of 4.

Findings

300-dimensional skip-gram embeddings perform best

Fine-tuning hyperparameters improves embedding quality

Embeddings enhance news article classification accuracy

Abstract

Word embedding or vector representation of word holds syntactical and semantic characteristics of a word which can be an informative feature for any machine learning-based models of natural language processing. There are several deep learning-based models for the vectorization of words like word2vec, fasttext, gensim, glove, etc. In this study, we analyze word2vec model for learning word vectors by tuning different hyper-parameters and present the most effective word embedding for Bangla language. For testing the performances of different word embeddings generated by fine-tuning of word2vec model, we perform both intrinsic and extrinsic evaluations. We cluster the word vectors to examine the relational similarity of words for intrinsic evaluation and also use different word embeddings as the feature of news article classifier for extrinsic evaluation. From our experiment, we discover that the word vectors with 300 dimensions, generated from "skip-gram" method of word2vec model using the sliding window size of 4, are giving the most robust vector representations for Bangla language.