An Investigation of Potential Function Designs for Neural CRF

TL;DR

This paper explores various potential function designs for neural CRF models, demonstrating that a decomposed quadrilinear potential function leveraging contextual word representations yields superior sequence labeling performance.

Contribution

It introduces and evaluates a novel quadrilinear potential function that explicitly models interactions between labels and contextual words in neural CRF models.

Findings

Quadrilinear potential function outperforms other designs.

Explicit modeling of contextual words improves accuracy.

Best performance achieved with the proposed potential function.

Abstract

The neural linear-chain CRF model is one of the most widely-used approach to sequence labeling. In this paper, we investigate a series of increasingly expressive potential functions for neural CRF models, which not only integrate the emission and transition functions, but also explicitly take the representations of the contextual words as input. Our extensive experiments show that the decomposed quadrilinear potential function based on the vector representations of two neighboring labels and two neighboring words consistently achieves the best performance.