Toward a Standardized and More Accurate Indonesian Part-of-Speech Tagging

TL;DR

This paper evaluates various POS tagging techniques for Indonesian, achieving a new state-of-the-art with neural networks and providing a standardized dataset split for future research.

Contribution

It introduces a standardized dataset split for Indonesian POS tagging and demonstrates the effectiveness of neural network models, setting a new performance benchmark.

Findings

Recurrent neural network achieved 97.47 F1 score.

Neural models outperform rule-based and CRF approaches.

Standardized dataset split released for future research.

Abstract

Previous work in Indonesian part-of-speech (POS) tagging are hard to compare as they are not evaluated on a common dataset. Furthermore, in spite of the success of neural network models for English POS tagging, they are rarely explored for Indonesian. In this paper, we explored various techniques for Indonesian POS tagging, including rule-based, CRF, and neural network-based models. We evaluated our models on the IDN Tagged Corpus. A new state-of-the-art of 97.47 F1 score is achieved with a recurrent neural network. To provide a standard for future work, we release the dataset split that we used publicly.