# NodeDrop: A Condition for Reducing Network Size without Effect on Output

## TL;DR

NodeDrop is a novel regularization method that identifies and removes redundant nodes in neural networks, significantly reducing model size without sacrificing accuracy, especially useful for resource-constrained systems.

## Contribution

The paper introduces NodeDrop, a new regularization-based approach to eliminate unnecessary features in neural networks while ensuring no loss in output quality.

## Key findings

- Reduces network parameters by 114x on CIFAR10
- Maintains high performance after feature reduction
- Provides a condition to identify non-informative nodes

## Abstract

Determining an appropriate number of features for each layer in a neural network is an important and difficult task. This task is especially important in applications on systems with limited memory or processing power. Many current approaches to reduce network size either utilize iterative procedures, which can extend training time significantly, or require very careful tuning of algorithm parameters to achieve reasonable results. In this paper we propose NodeDrop, a new method for eliminating features in a network. With NodeDrop, we define a condition to identify and guarantee which nodes carry no information, and then use regularization to encourage nodes to meet this condition. We find that NodeDrop drastically reduces the number of features in a network while maintaining high performance, reducing the number of parameters by a factor of 114x for a VGG like network on CIFAR10 without a drop in accuracy.