Detecting Learning vs Memorization in Deep Neural Networks using Shared Structure Validation Sets

TL;DR

This paper introduces a permutation-based method to distinguish between learning and memorization in deep neural networks by comparing feature-based classifier performance on shared-structure validation sets, revealing insights into DNN behavior.

Contribution

A novel permutation approach that separates learning from memorization in DNNs by evaluating shallow classifiers trained on features with shared-structure validation sets.

Findings

DNNs can still learn from Gaussian noise inputs.

The method corroborates previous findings on memorization patterns.

Insights into how DNNs memorize over training epochs.

Abstract

The roles played by learning and memorization represent an important topic in deep learning research. Recent work on this subject has shown that the optimization behavior of DNNs trained on shuffled labels is qualitatively different from DNNs trained with real labels. Here, we propose a novel permutation approach that can differentiate memorization from learning in deep neural networks (DNNs) trained as usual (i.e., using the real labels to guide the learning, rather than shuffled labels). The evaluation of weather the DNN has learned and/or memorized, happens in a separate step where we compare the predictive performance of a shallow classifier trained with the features learned by the DNN, against multiple instances of the same classifier, trained on the same input, but using shuffled labels as outputs. By evaluating these shallow classifiers in validation sets that share structure with the training set, we are able to tell apart learning from memorization. Application of our permutation approach to multi-layer perceptrons and convolutional neural networks trained on image data corroborated many findings from other groups. Most importantly, our illustrations also uncovered interesting dynamic patterns about how DNNs memorize over increasing numbers of training epochs, and support the surprising result that DNNs are still able to learn, rather than only memorize, when trained with pure Gaussian noise as input.