Investigating Bi-LSTM and CRF with POS Tag Embedding for Indonesian Named Entity Tagger

TL;DR

This paper explores deep learning approaches for Indonesian Named Entity recognition, specifically employing Bi-LSTM with POS tag embeddings and comparing Softmax and CRF output layers, demonstrating improved accuracy with POS embeddings.

Contribution

It introduces the use of POS tag embeddings in a Bi-LSTM-based Indonesian NE tagger and compares Softmax and CRF output layers, highlighting their performance differences.

Findings

POS tag embeddings improve NE tagging accuracy

Both Softmax and CRF have weaknesses in classifying NE tags

Deep learning can replace traditional methods without gazetteers

Abstract

Researches on Indonesian named entity (NE) tagger have been conducted since years ago. However, most did not use deep learning and instead employed traditional machine learning algorithms such as association rule, support vector machine, random forest, na\"ive bayes, etc. In those researches, word lists as gazetteers or clue words were provided to enhance the accuracy. Here, we attempt to employ deep learning in our Indonesian NE tagger. We use long short-term memory (LSTM) as the topology since it is the state-of-the-art of NE tagger. By using LSTM, we do not need a word list in order to enhance the accuracy. Basically, there are two main things that we investigate. The first is the output layer of the network: Softmax vs conditional random field (CRF). The second is the usage of part of speech (POS) tag embedding input layer. Using 8400 sentences as the training data and 97 sentences as the evaluation data, we find that using POS tag embedding as additional input improves the performance of our Indonesian NE tagger. As for the comparison between Softmax and CRF, we find that both architectures have a weakness in classifying an NE tag.