# Empirical study of extreme overfitting points of neural networks

## TL;DR

This paper investigates the properties and locations of extreme overfitting points in neural networks, where models achieve perfect training accuracy but almost zero test accuracy, challenging assumptions about loss function critical points.

## Contribution

It introduces a method to identify extreme overfitting points and analyzes their characteristics and position on the loss surface of neural networks.

## Key findings

- Extreme overfitting points have high training accuracy but poor test performance.
- Such points are located in specific regions of the loss surface.
- These points challenge the assumption that all critical points generalize well.

## Abstract

In this paper we propose a method of obtaining points of extreme overfitting - parameters of modern neural networks, at which they demonstrate close to 100 % training accuracy, simultaneously with almost zero accuracy on the test sample. Despite the widespread opinion that the overwhelming majority of critical points of the loss function of a neural network have equally good generalizing ability, such points have a huge generalization error. The paper studies the properties of such points and their location on the surface of the loss function of modern neural networks.