Corpora Compared: The Case of the Swedish Gigaword & Wikipedia Corpora

TL;DR

This study compares Swedish language embeddings from Gigaword and Wikipedia corpora, revealing that corpus quality and domain coverage significantly influence embedding performance beyond mere size.

Contribution

It demonstrates that corpus source and content quality impact embedding effectiveness, challenging the assumption that larger corpora always yield better results.

Findings

Wikipedia embeddings outperform Gigaword embeddings in analogy tests

Corpus quality and domain coverage affect embedding performance

Size alone does not determine embedding quality

Abstract

In this work, we show that the difference in performance of embeddings from differently sourced data for a given language can be due to other factors besides data size. Natural language processing (NLP) tasks usually perform better with embeddings from bigger corpora. However, broadness of covered domain and noise can play important roles. We evaluate embeddings based on two Swedish corpora: The Gigaword and Wikipedia, in analogy (intrinsic) tests and discover that the embeddings from the Wikipedia corpus generally outperform those from the Gigaword corpus, which is a bigger corpus. Downstream tests will be required to have a definite evaluation.