Detecting Hierarchical Changes in Latent Variable Models

TL;DR

This paper introduces an information-theoretic framework using MDL and DNML for hierarchical change detection in latent variable models, enabling interpretable identification of change levels in data streams.

Contribution

It proposes a novel hierarchical change detection method based on MDL and DNML, with theoretical foundations and empirical validation on stochastic block models.

Findings

Effective detection of hierarchical changes in latent models

Improved interpretability of change causes

Validated on synthetic and benchmark datasets

Abstract

This paper addresses the issue of detecting hierarchical changes in latent variable models (HCDL) from data streams. There are three different levels of changes for latent variable models: 1) the first level is the change in data distribution for fixed latent variables, 2) the second one is that in the distribution over latent variables, and 3) the third one is that in the number of latent variables. It is important to detect these changes because we can analyze the causes of changes by identifying which level a change comes from (change interpretability). This paper proposes an information-theoretic framework for detecting changes of the three levels in a hierarchical way. The key idea to realize it is to employ the MDL (minimum description length) change statistics for measuring the degree of change, in combination with DNML (decomposed normalized maximum likelihood) code-length calculation. We give a theoretical basis for making reliable alarms for changes. Focusing on stochastic block models, we employ synthetic and benchmark datasets to empirically demonstrate the effectiveness of our framework in terms of change interpretability as well as change detection.