Cross-Domain Multi-Task Learning for Sequential Sentence Classification in Research Papers

TL;DR

This paper introduces a novel multi-task deep learning architecture for cross-domain sequential sentence classification in scientific texts, demonstrating improved performance across multiple datasets and addressing transfer learning challenges.

Contribution

It proposes a new uniform deep learning model and transfer learning methods tailored for cross-domain scientific sentence classification, including semi-automatic class relation identification.

Findings

Models trained on different scientific domains benefit from multi-task learning.

The approach outperforms state-of-the-art on full paper datasets.

Comparable results to state-of-the-art on abstract datasets.

Abstract

Sequential sentence classification deals with the categorisation of sentences based on their content and context. Applied to scientific texts, it enables the automatic structuring of research papers and the improvement of academic search engines. However, previous work has not investigated the potential of transfer learning for sentence classification across different scientific domains and the issue of different text structure of full papers and abstracts. In this paper, we derive seven related research questions and present several contributions to address them: First, we suggest a novel uniform deep learning architecture and multi-task learning for cross-domain sequential sentence classification in scientific texts. Second, we tailor two common transfer learning methods, sequential transfer learning and multi-task learning, to deal with the challenges of the given task. Semantic relatedness of tasks is a prerequisite for successful transfer learning of neural models. Consequently, our third contribution is an approach to semi-automatically identify semantically related classes from different annotation schemes and we present an analysis of four annotation schemes. Comprehensive experimental results indicate that models, which are trained on datasets from different scientific domains, benefit from one another when using the proposed multi-task learning architecture. We also report comparisons with several state-of-the-art approaches. Our approach outperforms the state of the art on full paper datasets significantly while being on par for datasets consisting of abstracts.