DRIFT: Data Reduction via Informative Feature Transformation- Generalization Begins Before Deep Learning starts

TL;DR

DRIFT introduces a physics-inspired preprocessing method that transforms input data into a compact, informative feature set, improving deep learning training stability and generalization by emphasizing essential patterns before learning begins.

Contribution

This paper presents DRIFT, a novel data reduction technique based on vibrational analysis, which enhances deep learning performance by preprocessing data into a physically grounded, low-dimensional feature space.

Findings

DRIFT achieves competitive accuracy with significantly fewer input features.

It improves training stability and robustness against overfitting.

DRIFT shows minimal sensitivity to batch size, architecture, and resolution.

Abstract

Modern deep learning architectures excel at optimization, but only after the data has entered the network. The true bottleneck lies in preparing the right input: minimal, salient, and structured in a way that reflects the essential patterns of the data. We propose DRIFT (Data Reduction via Informative Feature Transformation), a novel preprocessing technique inspired by vibrational analysis in physical systems, to identify and extract the most resonant modes of input data prior to training. Unlike traditional models that attempt to learn amidst both signal and noise, DRIFT mimics physics perception by emphasizing informative features while discarding irrelevant elements. The result is a more compact and interpretable representation that enhances training stability and generalization performance. In DRIFT, images are projected onto a low-dimensional basis formed by spatial vibration mode shapes of plates, offering a physically grounded feature set. This enables neural networks to operate with drastically fewer input dimensions (~ 50 features on MNIST and less than 100 on CIFAR100) while achieving competitive classification accuracy. Extensive experiments across MNIST and CIFAR100 demonstrate DRIFT's superiority over standard pixel-based models and PCA in terms of training stability, resistance to overfitting, and generalization robustness. Notably, DRIFT displays minimal sensitivity to changes in batch size, network architecture, and image resolution, further establishing it as a resilient and efficient data representation strategy. This work shifts the focus from architecture engineering to input curation and underscores the power of physics-driven data transformations in advancing deep learning performance.