Towards interpretability of Mixtures of Hidden Markov Models

TL;DR

This paper introduces an information-theoretic measure for interpretability of Mixtures of Hidden Markov Models and proposes an entropy-regularized EM algorithm to enhance interpretability without compromising clustering performance.

Contribution

It presents a novel entropy-based interpretability measure and a regularized EM algorithm to improve the interpretability of MHMMs while maintaining clustering quality.

Findings

Entropy reduction improves interpretability of MHMMs.

Enhanced interpretability does not reduce clustering performance.

The proposed method maintains computational efficiency.

Abstract

Mixtures of Hidden Markov Models (MHMMs) are frequently used for clustering of sequential data. An important aspect of MHMMs, as of any clustering approach, is that they can be interpretable, allowing for novel insights to be gained from the data. However, without a proper way of measuring interpretability, the evaluation of novel contributions is difficult and it becomes practically impossible to devise techniques that directly optimize this property. In this work, an information-theoretic measure (entropy) is proposed for interpretability of MHMMs, and based on that, a novel approach to improve model interpretability is proposed, i.e., an entropy-regularized Expectation Maximization (EM) algorithm. The new approach aims for reducing the entropy of the Markov chains (involving state transition matrices) within an MHMM, i.e., assigning higher weights to common state transitions during clustering. It is argued that this entropy reduction, in general, leads to improved interpretability since the most influential and important state transitions of the clusters can be more easily identified. An empirical investigation shows that it is possible to improve the interpretability of MHMMs, as measured by entropy, without sacrificing (but rather improving) clustering performance and computational costs, as measured by the v-measure and number of EM iterations, respectively.