How Data Augmentation Shapes Neural Representations

TL;DR

This paper investigates how various data augmentation techniques influence the geometry of neural network representations, revealing shared patterns and predictive insights for model ensembling.

Contribution

It introduces a shape analysis framework to characterize and compare the effects of different data augmentation strategies on neural representations.

Findings

Increasing augmentation strength leads to well-behaved trajectories in shape space.

Different augmentation types steer representations in distinct directions.

Shape-space analysis can predict which representations improve ensembling.

Abstract

Data augmentation is widely recognized for improving generalization in deep networks, yet its impact on the geometry of learned representations remains poorly understood. In this work, we characterize how different data augmentation strategies reshape internal representations in neural networks. Using tools from shape analysis, we embed network hidden representations into a metric space where distance is invariant to scaling, translation, rotation and reflection. We show that increasing augmentation strength leads to well-behaved trajectories in this space, and that different augmentation types steer representations in distinct directions. Moreover, we investigate how neural representation shapes are distorted along data augmentation trajectories, and show that insights from neural geometry can predict which representations provide the most improvement when ensembling models. Our results reveal shared geometric patterns across architectures and seeds, and suggest that analyzing shape-space trajectories offers a principled tool for understanding and comparing data augmentation methods.