Neural Latent Dependency Model for Sequence Labeling

TL;DR

This paper introduces the Neural Latent Dependency Model (NLDM), which captures long-range label dependencies in sequence labeling tasks using a latent tree structure, improving over traditional methods.

Contribution

The paper proposes a novel neural model that efficiently models arbitrary-length label dependencies with a latent tree, along with an end-to-end training and polynomial-time inference algorithm.

Findings

NLDM outperforms strong baselines on synthetic and real datasets.

The model effectively captures long-range dependencies in sequence labeling.

Efficient polynomial-time inference is achieved for the proposed model.

Abstract

Sequence labeling is a fundamental problem in machine learning, natural language processing and many other fields. A classic approach to sequence labeling is linear chain conditional random fields (CRFs). When combined with neural network encoders, they achieve very good performance in many sequence labeling tasks. One limitation of linear chain CRFs is their inability to model long-range dependencies between labels. High order CRFs extend linear chain CRFs by modeling dependencies no longer than their order, but the computational complexity grows exponentially in the order. In this paper, we propose the Neural Latent Dependency Model (NLDM) that models dependencies of arbitrary length between labels with a latent tree structure. We develop an end-to-end training algorithm and a polynomial-time inference algorithm of our model. We evaluate our model on both synthetic and real datasets and show that our model outperforms strong baselines.