Circuit-Based Intrinsic Methods to Detect Overfitting

TL;DR

This paper introduces Counterfactual Simulation (CFS), an intrinsic, model-agnostic method to detect overfitting by analyzing training data flow through models like neural networks and random forests, without relying on test data.

Contribution

The paper proposes CFS, a hyper-parameter-free intrinsic method applicable across various models, providing new insights into neural network generalization and overfitting detection.

Findings

CFS effectively distinguishes models with different overfitting levels.

Neural networks tend to find common patterns rather than brute-force memorization.

Insights into neural network generalization compared to random forests.

Abstract

The focus of this paper is on intrinsic methods to detect overfitting. By intrinsic methods, we mean methods that rely only on the model and the training data, as opposed to traditional methods (we call them extrinsic methods) that rely on performance on a test set or on bounds from model complexity. We propose a family of intrinsic methods, called Counterfactual Simulation (CFS), which analyze the flow of training examples through the model by identifying and perturbing rare patterns. By applying CFS to logic circuits we get a method that has no hyper-parameters and works uniformly across different types of models such as neural networks, random forests and lookup tables. Experimentally, CFS can separate models with different levels of overfit using only their logic circuit representations without any access to the high level structure. By comparing lookup tables, neural networks, and random forests using CFS, we get insight into why neural networks generalize. In particular, we find that stochastic gradient descent in neural nets does not lead to "brute force" memorization, but finds common patterns (whether we train with actual or randomized labels), and neural networks are not unlike forests in this regard. Finally, we identify a limitation with our proposal that makes it unsuitable in an adversarial setting, but points the way to future work on robust intrinsic methods.