Theoretical Convergence of SMOTE-Generated Samples

TL;DR

This paper offers a rigorous theoretical analysis of SMOTE, demonstrating its convergence properties and providing practical guidance for practitioners, supported by numerical experiments on real and synthetic data.

Contribution

It provides the first theoretical proof of SMOTE's convergence properties and offers insights into how nearest neighbor rank affects convergence speed.

Findings

Z converges in probability to X

Stronger convergence in mean for compact X

Lower nearest neighbor rank accelerates convergence

Abstract

Imbalanced data affects a wide range of machine learning applications, from healthcare to network security. As SMOTE is one of the most popular approaches to addressing this issue, it is imperative to validate it not only empirically but also theoretically. In this paper, we provide a rigorous theoretical analysis of SMOTE's convergence properties. Concretely, we prove that the synthetic random variable Z converges in probability to the underlying random variable X. We further prove a stronger convergence in mean when X is compact. Finally, we show that lower values of the nearest neighbor rank lead to faster convergence offering actionable guidance to practitioners. The theoretical results are supported by numerical experiments using both real-life and synthetic data. Our work provides a foundational understanding that enhances data augmentation techniques beyond imbalanced data scenarios.