The Leinster-Cobbold diversity index as a criterion for sub-clustering

TL;DR

This paper introduces an automatic sub-clustering method that uses the Leinster-Cobbold diversity index to iteratively improve classification by maximizing diversity, demonstrated on galaxy spectra.

Contribution

The paper presents a novel sub-clustering algorithm that employs the Leinster-Cobbold diversity index to optimize cluster diversity in large datasets.

Findings

Effective sub-clustering on galaxy spectra datasets

Uses diversity index to guide clustering process

Demonstrates improved classification results

Abstract

An automatic procedure to perform sub-clustering on large samples is presented. At each iteration, the most diverse cluster is sub-clustered, and the global diversity of the new classification is compared to the previous one. The process stops if no improvement is found. The key to our procedure is the use of a quantitative measure of diversity, called the Leinster-Cobbold index, that takes into account the similarity between clusters. While this procedure has been successfully applied on a large sample of spectra of galaxies, we illustrate its efficiency with two examples in this paper.