DNNs, Dataset Statistics, and Correlation Functions

TL;DR

This paper emphasizes the significance of dataset correlation structures in image recognition, proposing that DNNs learn high-order correlation functions akin to mesoscale phenomena in physics, which may explain their generalization abilities.

Contribution

It introduces a novel perspective linking DNNs' success to their ability to discover mesoscale correlation structures similar to those in condensed matter physics.

Findings

DNNs may be implementing mesoscale correlation functions.

Successful generalization might be due to learning high-order correlations.

Dataset structure critically influences DNN performance.

Abstract

This paper argues that dataset structure is important in image recognition tasks (among other tasks). Specifically, we focus on the nature and genesis of correlational structure in the actual datasets upon which DNNs are trained. We argue that DNNs are implementing a widespread methodology in condensed matter physics and materials science that focuses on mesoscale correlation structures that live between fundamental atomic/molecular scales and continuum scales. Specifically, we argue that DNNs that are successful in image classification must be discovering high order correlation functions. It is well-known that DNNs successfully generalize in apparent contravention of standard statistical learning theory. We consider the implications of our discussion for this puzzle.