Keeping it Short and Simple: Summarising Complex Event Sequences with Multivariate Patterns

TL;DR

This paper introduces DITTO, an efficient algorithm for summarizing complex multivariate sequential data using rich patterns, enabling concise and interpretable descriptions that reveal underlying correlations and structures.

Contribution

The paper presents DITTO, a novel algorithm that effectively discovers multivariate patterns for data summarization based on the Minimum Description Length principle.

Findings

DITTO accurately detects planted patterns in synthetic data.

It scales well with data length, attribute count, and alphabet size.

On real datasets, DITTO produces interpretable summaries revealing data structure.

Abstract

We study how to obtain concise descriptions of discrete multivariate sequential data. In particular, how to do so in terms of rich multivariate sequential patterns that can capture potentially highly interesting (cor)relations between sequences. To this end we allow our pattern language to span over the domains (alphabets) of all sequences, allow patterns to overlap temporally, as well as allow for gaps in their occurrences. We formalise our goal by the Minimum Description Length principle, by which our objective is to discover the set of patterns that provides the most succinct description of the data. To discover high-quality pattern sets directly from data, we introduce DITTO, a highly efficient algorithm that approximates the ideal result very well. Experiments show that DITTO correctly discovers the patterns planted in synthetic data. Moreover, it scales favourably with the length of the data, the number of attributes, the alphabet sizes. On real data, ranging from sensor networks to annotated text, DITTO discovers easily interpretable summaries that provide clear insight in both the univariate and multivariate structure.