Data Augmentation vs. Equivariant Networks: A Theory of Generalization on Dynamics Forecasting

TL;DR

This paper develops a unified theoretical framework to compare data augmentation and equivariant networks, analyzing their effects on generalization in non-stationary dynamical systems with complex temporal dependencies.

Contribution

It derives generalization bounds for both approaches, providing new insights into their roles in improving robustness and performance in dynamic forecasting tasks.

Findings

Unified theory for data augmentation and equivariant networks

Focus on non-stationary dynamics with temporal dependencies

Provides generalization bounds for both methods

Abstract

Exploiting symmetry in dynamical systems is a powerful way to improve the generalization of deep learning. The model learns to be invariant to transformation and hence is more robust to distribution shift. Data augmentation and equivariant networks are two major approaches to injecting symmetry into learning. However, their exact role in improving generalization is not well understood. In this work, we derive the generalization bounds for data augmentation and equivariant networks, characterizing their effect on learning in a unified framework. Unlike most prior theories for the i.i.d. setting, we focus on non-stationary dynamics forecasting with complex temporal dependencies.