Unveiling Differences in Generative Models: A Scalable Differential Clustering Approach

TL;DR

This paper introduces FINC, a scalable spectral clustering method using Fourier features, to identify and compare nuanced differences in sample types generated by different models, especially in large-scale datasets.

Contribution

We propose a novel Fourier-based spectral clustering approach, FINC, for differential analysis of generative models, enabling scalable detection of distinct sample types.

Findings

FINC effectively identifies different sample types in large datasets.

The method demonstrates scalability and efficiency in high-dimensional settings.

Numerical results confirm the ability to detect frequency-based differences in generative outputs.

Abstract

A fine-grained comparison of generative models requires the identification of sample types generated differently by each of the involved models. While quantitative scores have been proposed in the literature to rank different generative models, score-based evaluation and ranking do not reveal the nuanced differences between the generative models in producing different sample types. In this work, we propose solving a differential clustering problem to detect sample types generated differently by two generative models. To solve the differential clustering problem, we develop a spectral method called Fourier-based Identification of Novel Clusters (FINC) to identify modes produced by a generative model with a higher frequency in comparison to a reference distribution. FINC provides a scalable algorithm based on random Fourier features to estimate the eigenspace of kernel covariance matrices of two generative models and utilize the principal eigendirections to detect the sample types present more dominantly in each model. We demonstrate the application of the FINC method to large-scale computer vision datasets and generative modeling frameworks. Our numerical results suggest the scalability of the developed Fourier-based method in highlighting the sample types produced with different frequencies by generative models. The project code is available at https://github.com/buyeah1109/FINC.