Validation of cluster analysis results on validation data: A systematic framework

TL;DR

This paper introduces a comprehensive framework for validating clustering results on validation datasets, integrating existing validation methods and formalizing different validation types for improved assessment.

Contribution

It provides a systematic, formalized framework that encompasses most existing validation techniques for clustering results on validation data.

Findings

Framework unifies validation approaches

Formal definitions clarify validation procedures

Examples demonstrate practical application

Abstract

Cluster analysis refers to a wide range of data analytic techniques for class discovery and is popular in many application fields. To judge the quality of a clustering result, different cluster validation procedures have been proposed in the literature. While there is extensive work on classical validation techniques, such as internal and external validation, less attention has been given to validating and replicating a clustering result using a validation dataset. Such a dataset may be part of the original dataset, which is separated before analysis begins, or it could be an independently collected dataset. We present a systematic structured framework for validating clustering results on validation data that includes most existing validation approaches. In particular, we review classical validation techniques such as internal and external validation, stability analysis, hypothesis testing, and visual validation, and show how they can be interpreted in terms of our framework. We precisely define and formalise different types of validation of clustering results on a validation dataset and explain how each type can be implemented in practice. Furthermore, we give examples of how clustering studies from the applied literature that used a validation dataset can be classified into the framework.