Adversarial Data Augmentation for Single Domain Generalization via Lyapunov Exponent-Guided Optimization

TL;DR

This paper introduces LEAwareSGD, a Lyapunov Exponent-guided optimization method that improves single domain generalization by dynamically balancing training stability and adaptability, leading to significant performance gains.

Contribution

It proposes a novel LE-guided optimization approach inspired by dynamical systems theory to enhance model generalization in SDG tasks.

Findings

Achieves up to 9.47% improvement on PACS in low-data regimes.

Demonstrates effectiveness across multiple datasets (PACS, OfficeHome, DomainNet).

Encourages training near the edge of chaos to balance stability and adaptability.

Abstract

Single Domain Generalization (SDG) aims to develop models capable of generalizing to unseen target domains using only one source domain, a task complicated by substantial domain shifts and limited data diversity. Existing SDG approaches primarily rely on data augmentation techniques, which struggle to effectively adapt training dynamics to accommodate large domain shifts. To address this, we propose LEAwareSGD, a novel Lyapunov Exponent (LE)-guided optimization approach inspired by dynamical systems theory. By leveraging LE measurements to modulate the learning rate, LEAwareSGD encourages model training near the edge of chaos, a critical state that optimally balances stability and adaptability. This dynamic adjustment allows the model to explore a wider parameter space and capture more generalizable features, ultimately enhancing the model's generalization capability. Extensive experiments on PACS, OfficeHome, and DomainNet demonstrate that LEAwareSGD yields substantial generalization gains, achieving up to 9.47\% improvement on PACS in low-data regimes. These results underscore the effectiveness of training near the edge of chaos for enhancing model generalization capability in SDG tasks.