# Breaking Inter-Layer Co-Adaptation by Classifier Anonymization

## TL;DR

This paper proposes FOCA, a method to prevent co-adaptation in neural networks by using multiple weak classifiers during training, leading to more robust feature representations.

## Contribution

The paper introduces FOCA, a novel training approach that employs random weak classifiers to avoid co-adaptation between features and classifiers.

## Key findings

- FOCA features form a class-separable, point-like distribution.
- Experimental results support the theoretical proposition.
- FOCA improves generalization by reducing overfitting to specific classifiers.

## Abstract

This study addresses an issue of co-adaptation between a feature extractor and a classifier in a neural network. A naive joint optimization of a feature extractor and a classifier often brings situations in which an excessively complex feature distribution adapted to a very specific classifier degrades the test performance. We introduce a method called Feature-extractor Optimization through Classifier Anonymization (FOCA), which is designed to avoid an explicit co-adaptation between a feature extractor and a particular classifier by using many randomly-generated, weak classifiers during optimization. We put forth a mathematical proposition that states the FOCA features form a point-like distribution within the same class in a class-separable fashion under special conditions. Real-data experiments under more general conditions provide supportive evidences.