Selfish Evolution: Making Discoveries in Extreme Label Noise with the Help of Overfitting Dynamics

TL;DR

This paper introduces Selfish Evolution, a novel method that leverages overfitting dynamics during training to detect and correct label noise in weakly supervised datasets, demonstrated on astrophysical and standard datasets.

Contribution

The paper presents a new technique that uses model overfitting patterns to identify and correct corrupted labels without prior assumptions, applicable in weak supervision.

Findings

Effective label correction in astrophysical data

Demonstrated success on MNIST dataset

Automatic convergence to cleaner datasets

Abstract

Motivated by the scarcity of proper labels in an astrophysical application, we have developed a novel technique, called Selfish Evolution, which allows for the detection and correction of corrupted labels in a weakly supervised fashion. Unlike methods based on early stopping, we let the model train on the noisy dataset. Only then do we intervene and allow the model to overfit to individual samples. The ``evolution'' of the model during this process reveals patterns with enough information about the noisiness of the label, as well as its correct version. We train a secondary network on these spatiotemporal ``evolution cubes'' to correct potentially corrupted labels. We incorporate the technique in a closed-loop fashion, allowing for automatic convergence towards a mostly clean dataset, without presumptions about the state of the network in which we intervene. We evaluate on the main task of the Supernova-hunting dataset but also demonstrate efficiency on the more standard MNIST dataset.