Adaptive Class Emergence Training: Enhancing Neural Network Stability and Generalization through Progressive Target Evolution

TL;DR

This paper introduces a progressive target evolution training method for neural networks that improves stability and generalization by gradually transitioning from null to one-hot vectors during training.

Contribution

It presents a novel training approach inspired by structural equilibrium concepts, enabling smoother adaptation and better performance in complex classification tasks.

Findings

Faster convergence compared to traditional methods

Enhanced accuracy and generalization in noisy data scenarios

Reduced overfitting through progressive target evolution

Abstract

Recent advancements in artificial intelligence, particularly deep neural networks, have pushed the boundaries of what is achievable in complex tasks. Traditional methods for training neural networks in classification problems often rely on static target outputs, such as one-hot encoded vectors, which can lead to unstable optimization and difficulties in handling non-linearities within data. In this paper, we propose a novel training methodology that progressively evolves the target outputs from a null vector to one-hot encoded vectors throughout the training process. This gradual transition allows the network to adapt more smoothly to the increasing complexity of the classification task, maintaining an equilibrium state that reduces the risk of overfitting and enhances generalization. Our approach, inspired by concepts from structural equilibrium in finite element analysis, has been validated through extensive experiments on both synthetic and real-world datasets. The results demonstrate that our method achieves faster convergence, improved accuracy, and better generalization, especially in scenarios with high data complexity and noise. This progressive training framework offers a robust alternative to classical methods, opening new perspectives for more efficient and stable neural network training.