Chinese grammatical error correction based on knowledge distillation

TL;DR

This paper employs knowledge distillation to compress Chinese grammatical error correction models, significantly enhancing their robustness against adversarial attacks while maintaining performance and reducing model size.

Contribution

It introduces a knowledge distillation approach to improve the robustness and efficiency of Chinese grammatical error correction models against adversarial attacks.

Findings

Distilled models maintain performance with fewer parameters

Robustness against attack test sets is significantly improved

Training speed is increased with model compression

Abstract

In view of the poor robustness of existing Chinese grammatical error correction models on attack test sets and large model parameters, this paper uses the method of knowledge distillation to compress model parameters and improve the anti-attack ability of the model. In terms of data, the attack test set is constructed by integrating the disturbance into the standard evaluation data set, and the model robustness is evaluated by the attack test set. The experimental results show that the distilled small model can ensure the performance and improve the training speed under the condition of reducing the number of model parameters, and achieve the optimal effect on the attack test set, and the robustness is significantly improved. Code is available at https://github.com/Richard88888/KD-CGEC.