Learning to Classify from Impure Samples with High-Dimensional Data

TL;DR

This paper demonstrates that high-dimensional classifiers can be trained on impure, weakly supervised data, reducing reliance on simulations and enabling direct analysis of complex data in physics.

Contribution

It introduces methods for training complex, high-dimensional classifiers using weak supervision on impure data, expanding beyond previous low-dimensional approaches.

Findings

High-dimensional classifiers perform comparably to pure samples.

Weak supervision enables training without relying solely on simulations.

The approach facilitates direct physics analysis from data.

Abstract

A persistent challenge in practical classification tasks is that labeled training sets are not always available. In particle physics, this challenge is surmounted by the use of simulations. These simulations accurately reproduce most features of data, but cannot be trusted to capture all of the complex correlations exploitable by modern machine learning methods. Recent work in weakly supervised learning has shown that simple, low-dimensional classifiers can be trained using only the impure mixtures present in data. Here, we demonstrate that complex, high-dimensional classifiers can also be trained on impure mixtures using weak supervision techniques, with performance comparable to what could be achieved with pure samples. Using weak supervision will therefore allow us to avoid relying exclusively on simulations for high-dimensional classification. This work opens the door to a new regime whereby complex models are trained directly on data, providing direct access to probe the underlying physics.