Geometrically Regularized Transfer Learning with On-Manifold and Off-Manifold Perturbation

TL;DR

This paper introduces MAADA, a transfer learning framework that decomposes adversarial perturbations into on-manifold and off-manifold components, improving generalization and robustness across domains.

Contribution

It presents a novel geometry-aware adversarial augmentation method with theoretical analysis and demonstrates superior performance on multiple domain adaptation benchmarks.

Findings

Outperforms existing methods on DomainNet, VisDA, and Office-Home datasets.

Enhances cross-domain generalization and structural robustness.

Theoretically reduces hypothesis complexity and smooths decision boundaries.

Abstract

Transfer learning under domain shift remains a fundamental challenge due to the divergence between source and target data manifolds. In this paper, we propose MAADA (Manifold-Aware Adversarial Data Augmentation), a novel framework that decomposes adversarial perturbations into on-manifold and off-manifold components to simultaneously capture semantic variation and model brittleness. We theoretically demonstrate that enforcing on-manifold consistency reduces hypothesis complexity and improves generalization, while off-manifold regularization smooths decision boundaries in low-density regions. Moreover, we introduce a geometry-aware alignment loss that minimizes geodesic discrepancy between source and target manifolds. Experiments on DomainNet, VisDA, and Office-Home show that MAADA consistently outperforms existing adversarial and adaptation methods in both unsupervised and few-shot settings, demonstrating superior structural robustness and cross-domain generalization.