Building the Language Resource for a Cebuano-Filipino Neural Machine Translation System

TL;DR

This paper details the creation of a parallel Cebuano-Filipino corpus from biblical texts and Wikipedia, employing correction techniques and topic segmentation, to facilitate neural machine translation with promising BLEU score results.

Contribution

It introduces a novel parallel corpus for Cebuano-Filipino translation, combining correction methods and topic segmentation for low-resource language translation.

Findings

BLEU scores differ between the two corpora

Correction techniques improved translation quality

Topic segmentation aids in sentence extraction

Abstract

Parallel corpus is a critical resource in machine learning-based translation. The task of collecting, extracting, and aligning texts in order to build an acceptable corpus for doing the translation is very tedious most especially for low-resource languages. In this paper, we present the efforts made to build a parallel corpus for Cebuano and Filipino from two different domains: biblical texts and the web. For the biblical resource, subword unit translation for verbs and copy-able approach for nouns were applied to correct inconsistencies in the translation. This correction mechanism was applied as a preprocessing technique. On the other hand, for Wikipedia being the main web resource, commonly occurring topic segments were extracted from both the source and the target languages. These observed topic segments are unique in 4 different categories. The identification of these topic segments may be used for the automatic extraction of sentences. A Recurrent Neural Network was used to implement the translation using OpenNMT sequence modeling tool in TensorFlow. The two different corpora were then evaluated by using them as two separate inputs in the neural network. Results have shown a difference in BLEU scores in both corpora.