Neural Proto-Language Reconstruction

TL;DR

This paper explores advanced neural network models, including Transformers with VAE and data augmentation, to improve the automation and accuracy of proto-language reconstruction, a traditionally manual linguistic task.

Contribution

It introduces novel neural approaches combining VAE and data augmentation to enhance proto-language reconstruction accuracy and training stability.

Findings

VAE-augmented Transformer outperforms previous models on WikiHan dataset.

Data augmentation stabilizes training process.

Neural machine translation models are effective for reconstruction tasks.

Abstract

Proto-form reconstruction has been a painstaking process for linguists. Recently, computational models such as RNN and Transformers have been proposed to automate this process. We take three different approaches to improve upon previous methods, including data augmentation to recover missing reflexes, adding a VAE structure to the Transformer model for proto-to-language prediction, and using a neural machine translation model for the reconstruction task. We find that with the additional VAE structure, the Transformer model has a better performance on the WikiHan dataset, and the data augmentation step stabilizes the training.