Clustering processes

TL;DR

This paper introduces a novel asymptotic notion of consistency for clustering data generated by stationary ergodic processes, proposing simple algorithms that are consistent under broad non-parametric conditions.

Contribution

It defines a new consistency criterion for clustering based on process distribution equivalence and provides simple algorithms that are consistent without parametric or independence assumptions.

Findings

Consistent clustering is achievable under minimal assumptions.

Algorithms work with known or unknown number of clusters.

No parametric or independence assumptions are required.

Abstract

The problem of clustering is considered, for the case when each data point is a sample generated by a stationary ergodic process. We propose a very natural asymptotic notion of consistency, and show that simple consistent algorithms exist, under most general non-parametric assumptions. The notion of consistency is as follows: two samples should be put into the same cluster if and only if they were generated by the same distribution. With this notion of consistency, clustering generalizes such classical statistical problems as homogeneity testing and process classification. We show that, for the case of a known number of clusters, consistency can be achieved under the only assumption that the joint distribution of the data is stationary ergodic (no parametric or Markovian assumptions, no assumptions of independence, neither between nor within the samples). If the number of clusters is unknown, consistency can be achieved under appropriate assumptions on the mixing rates of the processes. (again, no parametric or independence assumptions). In both cases we give examples of simple (at most quadratic in each argument) algorithms which are consistent.