Semantic Tagging with Deep Residual Networks

TL;DR

This paper introduces a new multilingual semantic tagging task and presents a ResNet-based tagger that combines word and character features, achieving state-of-the-art results on POS tagging.

Contribution

It is the first to apply deep residual networks to semantic tagging, incorporating a novel residual bypass architecture for improved multilingual semantic parsing.

Findings

ResNet-based tagger outperforms previous models on POS tagging accuracy.

The proposed model effectively combines word and character representations.

Semantic tagging performance is validated both intrinsically and on POS tagging tasks.

Abstract

We propose a novel semantic tagging task, sem-tagging, tailored for the purpose of multilingual semantic parsing, and present the first tagger using deep residual networks (ResNets). Our tagger uses both word and character representations and includes a novel residual bypass architecture. We evaluate the tagset both intrinsically on the new task of semantic tagging, as well as on Part-of-Speech (POS) tagging. Our system, consisting of a ResNet and an auxiliary loss function predicting our semantic tags, significantly outperforms prior results on English Universal Dependencies POS tagging (95.71% accuracy on UD v1.2 and 95.67% accuracy on UD v1.3).