Predicting Punctuation in Ancient Chinese Texts: A Multi-Layered LSTM and Attention-Based Approach

TL;DR

This paper introduces a novel multi-layered LSTM with attention mechanism to predict punctuation in ancient Chinese texts, improving understanding and interpretation of texts lacking original punctuation marks.

Contribution

It extends previous work by integrating multi-layered LSTMs and multi-head attention, achieving better performance in punctuation prediction for ancient Chinese texts.

Findings

Multi-layered LSTM with attention outperforms simpler RNN models.

The approach improves punctuation prediction accuracy.

Enhanced understanding of ancient Chinese texts through better segmentation.

Abstract

It was only until the 20th century when the Chinese language began using punctuation. In fact, many ancient Chinese texts contain thousands of lines with no distinct punctuation marks or delimiters in sight. The lack of punctuation in such texts makes it difficult for humans to identify when there pauses or breaks between particular phrases and understand the semantic meaning of the written text (Mogahed, 2012). As a result, unless one was educated in the ancient time period, many readers of ancient Chinese would have significantly different interpretations of the texts. We propose an approach to predict the location (and type) of punctuation in ancient Chinese texts that extends the work of Oh et al (2017) by leveraging a bidirectional multi-layered LSTM with a multi-head attention mechanism as inspired by Luong et al.'s (2015) discussion of attention-based architectures. We find that the use of multi-layered LSTMs and multi-head attention significantly outperforms RNNs that don't incorporate such components when evaluating ancient Chinese texts.