Multi-Task Cross-Lingual Sequence Tagging from Scratch

TL;DR

This paper introduces a deep hierarchical neural network for sequence tagging that is task and language independent, achieving state-of-the-art results across multiple languages and tasks through multi-task and cross-lingual training.

Contribution

The paper presents a novel deep recurrent neural network architecture that is feature engineering free and extends to multi-task and cross-lingual training for sequence tagging.

Findings

Achieves state-of-the-art results on POS tagging, chunking, and NER.

Multi-task and cross-lingual training improve performance.

Model is task and language independent, requiring no feature engineering.

Abstract

We present a deep hierarchical recurrent neural network for sequence tagging. Given a sequence of words, our model employs deep gated recurrent units on both character and word levels to encode morphology and context information, and applies a conditional random field layer to predict the tags. Our model is task independent, language independent, and feature engineering free. We further extend our model to multi-task and cross-lingual joint training by sharing the architecture and parameters. Our model achieves state-of-the-art results in multiple languages on several benchmark tasks including POS tagging, chunking, and NER. We also demonstrate that multi-task and cross-lingual joint training can improve the performance in various cases.