Linear chain conditional random fields, hidden Markov models, and related classifiers

TL;DR

This paper demonstrates the equivalence between linear-chain CRFs and HMMs and reformulates Bayesian classifiers as discriminative models, challenging the necessity of choosing HMMs over CRFs in NLP.

Contribution

It proves the equivalence of linear-chain CRFs and HMMs and reformulates Bayesian classifiers as discriminative models, providing new insights into model selection.

Findings

LC-CRFs are equivalent to HMMs in terms of posterior distribution

Bayesian classifiers MPM and MAP can be reformulated as discriminative models

Challenging the necessity of using HMMs over CRFs in NLP

Abstract

Practitioners use Hidden Markov Models (HMMs) in different problems for about sixty years. Besides, Conditional Random Fields (CRFs) are an alternative to HMMs and appear in the literature as different and somewhat concurrent models. We propose two contributions. First, we show that basic Linear-Chain CRFs (LC-CRFs), considered as different from the HMMs, are in fact equivalent to them in the sense that for each LC-CRF there exists a HMM - that we specify - whom posterior distribution is identical to the given LC-CRF. Second, we show that it is possible to reformulate the generative Bayesian classifiers Maximum Posterior Mode (MPM) and Maximum a Posteriori (MAP) used in HMMs, as discriminative ones. The last point is of importance in many fields, especially in Natural Language Processing (NLP), as it shows that in some situations dropping HMMs in favor of CRFs was not necessary.