Large-data determinantal clustering

TL;DR

This paper introduces scalable methods for determinantal consensus clustering on large datasets by using sparse kernel matrices to approximate the original Gram matrix, reducing computational complexity.

Contribution

It proposes two efficient algorithms for large-scale determinantal clustering using approximate DPP sampling with sparse kernels.

Findings

Algorithms perform well on large datasets

Significant reduction in computational time

Maintains clustering diversity and quality

Abstract

Determinantal consensus clustering is a promising and attractive alternative to partitioning about medoids and k-means for ensemble clustering. Based on a determinantal point process or DPP sampling, it ensures that subsets of similar points are less likely to be selected as centroids. It favors more diverse subsets of points. The sampling algorithm of the determinantal point process requires the eigendecomposition of a Gram matrix. This becomes computationally intensive when the data size is very large. This is particularly an issue in consensus clustering, where a given clustering algorithm is run several times in order to produce a final consolidated clustering. We propose two efficient alternatives to carry out determinantal consensus clustering on large datasets. They consist in DPP sampling based on sparse and small kernel matrices whose eigenvalue distributions are close to that of the original Gram matrix.