Variational Sequential Labelers for Semi-Supervised Learning

TL;DR

This paper presents a family of variational models for semi-supervised sequence labeling that combine generative and discriminative approaches, leveraging latent variables to improve performance on multiple datasets.

Contribution

It introduces a novel multitask variational framework with hierarchical latent variables for semi-supervised sequence labeling, outperforming standard baselines.

Findings

Models outperform standard baselines on 8 datasets.

Unlabeled data further improves model performance.

Hierarchical latent structures enhance label and word information integration.

Abstract

We introduce a family of multitask variational methods for semi-supervised sequence labeling. Our model family consists of a latent-variable generative model and a discriminative labeler. The generative models use latent variables to define the conditional probability of a word given its context, drawing inspiration from word prediction objectives commonly used in learning word embeddings. The labeler helps inject discriminative information into the latent space. We explore several latent variable configurations, including ones with hierarchical structure, which enables the model to account for both label-specific and word-specific information. Our models consistently outperform standard sequential baselines on 8 sequence labeling datasets, and improve further with unlabeled data.