Using noise resilience for ranking generalization of deep neural networks

TL;DR

This paper introduces measures based on noise resilience to predict the generalization error of deep neural networks, addressing the challenge of understanding why overparameterized models generalize well despite fitting random labels.

Contribution

It proposes novel noise resilience-based measures for predicting neural network generalization, validated through competitive performance at NeurIPS 2020.

Findings

Achieved 5th place in PGDL competition at NeurIPS 2020

Demonstrated effectiveness of noise resilience measures in predicting generalization

Provided insights into the role of noise resilience in neural network generalization

Abstract

Recent papers have shown that sufficiently overparameterized neural networks can perfectly fit even random labels. Thus, it is crucial to understand the underlying reason behind the generalization performance of a network on real-world data. In this work, we propose several measures to predict the generalization error of a network given the training data and its parameters. Using one of these measures, based on noise resilience of the network, we secured 5th position in the predicting generalization in deep learning (PGDL) competition at NeurIPS 2020.