Long-Tailed Data Classification by Increasing and Decreasing Neurons During Training

TL;DR

This paper introduces a biologically inspired method that dynamically adjusts the number of neurons during training to improve classification accuracy on imbalanced datasets, without increasing final network size.

Contribution

It proposes a novel training approach that periodically adds and removes neurons to better handle class imbalance, maintaining final network size and enhancing performance.

Findings

Outperforms fixed-size networks on multiple datasets.

Enhances minority class recognition accuracy.

Works synergistically with other imbalance techniques.

Abstract

In conventional deep learning, the number of neurons typically remains fixed during training. However, insights from biology suggest that the human hippocampus undergoes continuous neuron generation and pruning of neurons over the course of learning, implying that a flexible allocation of capacity can contribute to enhance performance. Real-world datasets often exhibit class imbalance situations where certain classes have far fewer samples than others, leading to significantly reduce recognition accuracy for minority classes when relying on fixed size networks.To address the challenge, we propose a method that periodically adds and removes neurons during training, thereby boosting representational power for minority classes. By retaining critical features learned from majority classes while selectively increasing neurons for underrepresented classes, our approach dynamically adjusts capacity during training. Importantly, while the number of neurons changes throughout training, the final network size and structure remain unchanged, ensuring efficiency and compatibility with deployment.Furthermore, by experiments on three different datasets and five representative models, we demonstrate that the proposed method outperforms fixed size networks and shows even greater accuracy when combined with other imbalance-handling techniques. Our results underscore the effectiveness of dynamic, biologically inspired network designs in improving performance on class-imbalanced data.