Towards Stable Imbalanced Data Classification via Virtual Big Data Projection

TL;DR

This paper explores how Virtual Big Data (VBD) can improve deep autoencoder training and address class imbalance in data classification through a novel projection-based method, cross-concatenation.

Contribution

It introduces the first projection-based approach, cross-concatenation, leveraging VBD to enhance autoencoder generalization and balance imbalanced datasets without over-sampling.

Findings

VBD significantly reduces autoencoder validation loss.

Cross-concatenation effectively balances skewed class distributions.

VBD improves generalization and mitigates overfitting in autoencoders.

Abstract

Virtual Big Data (VBD) proved to be effective to alleviate mode collapse and vanishing generator gradient as two major problems of Generative Adversarial Neural Networks (GANs) very recently. In this paper, we investigate the capability of VBD to address two other major challenges in Machine Learning including deep autoencoder training and imbalanced data classification. First, we prove that, VBD can significantly decrease the validation loss of autoencoders via providing them a huge diversified training data which is the key to reach better generalization to minimize the over-fitting problem. Second, we use the VBD to propose the first projection-based method called cross-concatenation to balance the skewed class distributions without over-sampling. We prove that, cross-concatenation can solve uncertainty problem of data driven methods for imbalanced classification.