The Past Mistake is the Future Wisdom: Error-driven Contrastive Probability Optimization for Chinese Spell Checking
Yinghui Li, Qingyu Zhou, Yangning Li, Zhongli Li, Ruiyang Liu, Rongyi, Sun, Zizhen Wang, Chao Li, Yunbo Cao, Hai-Tao Zheng

TL;DR
This paper introduces ECOPO, an error-driven contrastive probability optimization framework that enhances Chinese Spell Checking by refining PLMs to better distinguish correct characters from common errors, improving accuracy.
Contribution
The paper proposes a novel, model-agnostic ECOPO framework that guides PLMs to avoid common errors, addressing the semantic bias in existing CSC methods.
Findings
ECOPO improves CSC performance on SIGHAN datasets.
ECOPO effectively refines PLM knowledge representations.
ECOPO is compatible with existing CSC methods.
Abstract
Chinese Spell Checking (CSC) aims to detect and correct Chinese spelling errors, which are mainly caused by the phonological or visual similarity. Recently, pre-trained language models (PLMs) promote the progress of CSC task. However, there exists a gap between the learned knowledge of PLMs and the goal of CSC task. PLMs focus on the semantics in text and tend to correct the erroneous characters to semantically proper or commonly used ones, but these aren't the ground-truth corrections. To address this issue, we propose an Error-driven COntrastive Probability Optimization (ECOPO) framework for CSC task. ECOPO refines the knowledge representations of PLMs, and guides the model to avoid predicting these common characters through an error-driven way. Particularly, ECOPO is model-agnostic and it can be combined with existing CSC methods to achieve better performance. Extensive experiments…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
- 🤗Macropodus/macbert4mdcspell_v1model· 40k dl· ♡ 240k dl♡ 2
- 🤗Macropodus/macbert4csc_v2model· 8 dl· ♡ 28 dl♡ 2
- 🤗Macropodus/macbert4csc_v1model· 5 dl· ♡ 15 dl♡ 1
- 🤗Macropodus/bert4csc_v1model· 4 dl· ♡ 14 dl♡ 1
- 🤗Macropodus/relm_v1model· 42 dl· ♡ 142 dl♡ 1
- 🤗Macropodus/macbert4mdcspell_v2model· 283 dl· ♡ 6283 dl♡ 6
- 🤗Macropodus/macbert4mdcspell_v3model· 310 dl· ♡ 1310 dl♡ 1
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNatural Language Processing Techniques · Second Language Acquisition and Learning · Subtitles and Audiovisual Media
