Hierarchical Qualitative Clustering: clustering mixed datasets with critical qualitative information

TL;DR

This paper introduces Hierarchical Qualitative Clustering (HQC), a novel method for clustering qualitative data that preserves interpretability and scales well with high-dimensional mixed datasets, demonstrated on music and financial data.

Contribution

The paper presents a new HQC method based on Maximum Mean Discrepancy that effectively clusters qualitative data while maintaining interpretability and scalability.

Findings

Successfully clustered music artists and company industries.

Demonstrated interpretability of qualitative clusters.

Showed scalability to high-dimensional datasets.

Abstract

Clustering can be used to extract insights from data or to verify some of the assumptions held by the domain experts, namely data segmentation. In the literature, few methods can be applied in clustering qualitative values using the context associated with other variables present in the data, without losing interpretability. Moreover, the metrics for calculating dissimilarity between qualitative values often scale poorly for high dimensional mixed datasets. In this study, we propose a novel method for clustering qualitative values, based on Hierarchical Clustering (HQC), and using Maximum Mean Discrepancy. HQC maintains the original interpretability of the qualitative information present in the dataset. We apply HQC to two datasets. Using a mixed dataset provided by Spotify, we showcase how our method can be used for clustering music artists based on the quantitative features of thousands of songs. In addition, using financial features of companies, we cluster company industries, and discuss the implications in investment portfolios diversification.