Quantifying Overfitting: Evaluating Neural Network Performance through Analysis of Null Space

TL;DR

This paper introduces a novel method to quantify overfitting in neural networks by analyzing the null space of the last layer, enabling overfitting detection solely from testing data without access to training data or accuracy.

Contribution

It presents the first approach to assess overfitting using null space analysis, providing a privacy-preserving and data-efficient overfitting detection method.

Findings

Distinct null space angle patterns correlate with overfitting.

Models with poor generalization show specific null space characteristics.

Method effective across various architectures and datasets.

Abstract

Machine learning models that are overfitted/overtrained are more vulnerable to knowledge leakage, which poses a risk to privacy. Suppose we download or receive a model from a third-party collaborator without knowing its training accuracy. How can we determine if it has been overfitted or overtrained on its training data? It's possible that the model was intentionally over-trained to make it vulnerable during testing. While an overfitted or overtrained model may perform well on testing data and even some generalization tests, we can't be sure it's not over-fitted. Conducting a comprehensive generalization test is also expensive. The goal of this paper is to address these issues and ensure the privacy and generalization of our method using only testing data. To achieve this, we analyze the null space in the last layer of neural networks, which enables us to quantify overfitting without access to training data or knowledge of the accuracy of those data. We evaluated our approach on various architectures and datasets and observed a distinct pattern in the angle of null space when models are overfitted. Furthermore, we show that models with poor generalization exhibit specific characteristics in this space. Our work represents the first attempt to quantify overfitting without access to training data or knowing any knowledge about the training samples.