# Semi-supervised Multitask Learning for Sequence Labeling

## TL;DR

This paper introduces a semi-supervised multitask learning framework for sequence labeling that uses a language modeling objective to improve accuracy across various NLP tasks without extra data.

## Contribution

It presents a novel training objective that enhances sequence labeling models by incorporating surrounding word prediction, improving performance across multiple tasks.

## Key findings

- Consistent performance improvements on all evaluated benchmarks.
- No additional annotated or unannotated data needed for improvements.
- Effective across diverse sequence labeling tasks.

## Abstract

We propose a sequence labeling framework with a secondary training objective, learning to predict surrounding words for every word in the dataset. This language modeling objective incentivises the system to learn general-purpose patterns of semantic and syntactic composition, which are also useful for improving accuracy on different sequence labeling tasks. The architecture was evaluated on a range of datasets, covering the tasks of error detection in learner texts, named entity recognition, chunking and POS-tagging. The novel language modeling objective provided consistent performance improvements on every benchmark, without requiring any additional annotated or unannotated data.