Visual Pattern-Driven Exploration of Big Data

TL;DR

This paper presents a visual analytics pipeline that combines image feature analysis and unsupervised learning to effectively explore and interpret large, complex pattern spaces in big data, aiding analysts in understanding pattern distributions and relevance.

Contribution

It introduces a semi-automatic visual analytics approach that partitions pattern spaces into interpretable clusters using novel quality metrics, enhancing pattern exploration without ground-truth data.

Findings

Effective pattern space partitioning demonstrated in biomedical genomic data case study.

Novel quality metrics guide feature selection and clustering.

Interactive visualization supports drill-down analysis from overview to detail.

Abstract

Pattern extraction algorithms are enabling insights into the ever-growing amount of today's datasets by translating reoccurring data properties into compact representations. Yet, a practical problem arises: With increasing data volumes and complexity also the number of patterns increases, leaving the analyst with a vast result space. Current algorithmic and especially visualization approaches often fail to answer central overview questions essential for a comprehensive understanding of pattern distributions and support, their quality, and relevance to the analysis task. To address these challenges, we contribute a visual analytics pipeline targeted on the pattern-driven exploration of result spaces in a semi-automatic fashion. Specifically, we combine image feature analysis and unsupervised learning to partition the pattern space into interpretable, coherent chunks, which should be given priority in a subsequent in-depth analysis. In our analysis scenarios, no ground-truth is given. Thus, we employ and evaluate novel quality metrics derived from the distance distributions of our image feature vectors and the derived cluster model to guide the feature selection process. We visualize our results interactively, allowing the user to drill down from overview to detail into the pattern space and demonstrate our techniques in a case study on biomedical genomic data.