fastHan: A BERT-based Multi-Task Toolkit for Chinese NLP

TL;DR

fastHan is a compact, multi-task BERT-based toolkit for Chinese NLP that achieves near state-of-the-art results across four core tasks and is highly adaptable for practical use.

Contribution

It introduces a multi-task, pruned BERT model for Chinese NLP, with a smaller 4-layer version, and demonstrates strong performance and transferability.

Findings

Achieves near SOTA in dependency parsing and NER.

Attains SOTA in Chinese word segmentation and POS tagging.

Exhibits strong transferability and user-friendly design.

Abstract

We present fastHan, an open-source toolkit for four basic tasks in Chinese natural language processing: Chinese word segmentation (CWS), Part-of-Speech (POS) tagging, named entity recognition (NER), and dependency parsing. The backbone of fastHan is a multi-task model based on a pruned BERT, which uses the first 8 layers in BERT. We also provide a 4-layer base model compressed from the 8-layer model. The joint-model is trained and evaluated on 13 corpora of four tasks, yielding near state-of-the-art (SOTA) performance in dependency parsing and NER, achieving SOTA performance in CWS and POS. Besides, fastHan's transferability is also strong, performing much better than popular segmentation tools on a non-training corpus. To better meet the need of practical application, we allow users to use their own labeled data to further fine-tune fastHan. In addition to its small size and excellent performance, fastHan is user-friendly. Implemented as a python package, fastHan isolates users from the internal technical details and is convenient to use. The project is released on Github.