Disentangled Phonetic Representation for Chinese Spelling Correction

TL;DR

This paper introduces a novel approach for Chinese Spelling Correction that disentangles phonetic and textual features, using a pinyin-to-character objective and self-distillation to improve correction accuracy.

Contribution

It proposes a disentangled representation method with a pinyin-to-character objective and self-distillation, enhancing phonetic information utilization in CSC.

Findings

Outperforms existing methods on three benchmarks.

Effective disentanglement of phonetic and textual features.

Improved correction accuracy with phonetic information.

Abstract

Chinese Spelling Correction (CSC) aims to detect and correct erroneous characters in Chinese texts. Although efforts have been made to introduce phonetic information (Hanyu Pinyin) in this task, they typically merge phonetic representations with character representations, which tends to weaken the representation effect of normal texts. In this work, we propose to disentangle the two types of features to allow for direct interaction between textual and phonetic information. To learn useful phonetic representations, we introduce a pinyin-to-character objective to ask the model to predict the correct characters based solely on phonetic information, where a separation mask is imposed to disable attention from phonetic input to text. To avoid overfitting the phonetics, we further design a self-distillation module to ensure that semantic information plays a major role in the prediction. Extensive experiments on three CSC benchmarks demonstrate the superiority of our method in using phonetic information.