On equivalence between linear-chain conditional random fields and hidden Markov chains

TL;DR

This paper demonstrates that linear-chain conditional random fields and hidden Markov chains are mathematically equivalent, sharing the same posterior distributions despite different parametrizations, challenging the common perception of their differences.

Contribution

The authors explicitly construct an HMC for each CRF, proving their equivalence in processing power and showing they are just differently parametrized models.

Findings

HMCs and CRFs have identical posterior distributions.

Linear-chain CRFs can be explicitly represented as HMCs.

The models are not fundamentally different but differently parametrized.

Abstract

Practitioners successfully use hidden Markov chains (HMCs) in different problems for about sixty years. HMCs belong to the family of generative models and they are often compared to discriminative models, like conditional random fields (CRFs). Authors usually consider CRFs as quite different from HMCs, and CRFs are often presented as interesting alternative to HMCs. In some areas, like natural language processing (NLP), discriminative models have completely supplanted generative models. However, some recent results show that both families of models are not so different, and both of them can lead to identical processing power. In this paper we compare the simple linear-chain CRFs to the basic HMCs. We show that HMCs are identical to CRFs in that for each CRF we explicitly construct an HMC having the same posterior distribution. Therefore, HMCs and linear-chain CRFs are not different but just differently parametrized models.