Neural Network based Deep Transfer Learning for Cross-domain Dependency Parsing

TL;DR

This paper presents a neural dependency parser using deep transfer learning and self-attention, effectively adapting to multiple domains for improved cross-domain dependency parsing performance.

Contribution

It introduces a neural network-based deep transfer learning approach with self-attention for cross-domain dependency parsing, applied to three distinct domains.

Findings

Model performs competitively across three target domains.

Utilizes self-attention to enhance word meaning representation.

Transfers pre-trained networks for domain adaptation.

Abstract

In this paper, we describe the details of the neural dependency parser sub-mitted by our team to the NLPCC 2019 Shared Task of Semi-supervised do-main adaptation subtask on Cross-domain Dependency Parsing. Our system is based on the stack-pointer networks(STACKPTR). Considering the im-portance of context, we utilize self-attention mechanism for the representa-tion vectors to capture the meaning of words. In addition, to adapt three dif-ferent domains, we utilize neural network based deep transfer learning which transfers the pre-trained partial network in the source domain to be a part of deep neural network in the three target domains (product comments, product blogs and web fiction) respectively. Results on the three target domains demonstrate that our model performs competitively.