Comparing Generative Models with the New Physics Learning Machine

TL;DR

This paper evaluates the effectiveness of the New Physics Learning Machine, a generative model assessment tool, by comparing it to other methods within a high-dimensional two-sample testing framework, highlighting efficiency and computational tradeoffs.

Contribution

It provides a comparative analysis of the New Physics Learning Machine against alternative two-sample tests in high-dimensional settings, emphasizing efficiency and practical use cases.

Findings

The New Physics Learning Machine performs competitively in high-dimensional two-sample tests.

Learning-based methods involve significant computational costs.

Different methods offer advantages depending on specific use cases.

Abstract

The rise of generative models for scientific research calls for the development of new methods to evaluate their fidelity. A natural framework for addressing this problem is two-sample hypothesis testing, namely the task of determining whether two data sets are drawn from the same distribution. In large-scale and high-dimensional regimes, machine learning offers a set of tools to push beyond the limitations of standard statistical techniques. In this work, we put this claim to the test by comparing a recent proposal from the high-energy physics literature, the New Physics Learning Machine, to perform a classification-based two-sample test against a number of alternative approaches, following the framework presented in Grossi et al. (2025). We highlight the efficiency tradeoffs of the method and the computational costs that come from adopting learning-based approaches. Finally, we discuss the advantages of the different methods for different use cases.