Flatness-aware Curriculum Learning via Adversarial Difficulty

TL;DR

This paper introduces a novel adversarial difficulty measure (ADM) that enhances curriculum learning combined with sharpness-aware minimization (SAM) to improve neural network generalization, especially in flat minima regions.

Contribution

The paper proposes ADM, a new difficulty measure that remains effective in flat regions, enabling better curriculum learning with SAM for improved model robustness.

Findings

Outperforms existing curriculum and flatness-aware training methods.

Effective across image classification, fine-grained recognition, and domain generalization.

Preserves strengths of both CL and SAM in training.

Abstract

Neural networks trained by empirical risk minimization often suffer from overfitting, especially to specific samples or domains, which leads to poor generalization. Curriculum Learning (CL) addresses this issue by selecting training samples based on the difficulty. From the optimization perspective, methods such as Sharpness-Aware Minimization (SAM) improve robustness and generalization by seeking flat minima. However, combining CL with SAM is not straightforward. In flat regions, both the loss values and the gradient norms tend to become uniformly small, which makes it difficult to evaluate sample difficulty and design an effective curriculum. To overcome this problem, we propose the Adversarial Difficulty Measure (ADM), which quantifies adversarial vulnerability by leveraging the robustness properties of models trained toward flat minima. Unlike loss- or gradient-based measures, which become ineffective as training progresses into flatter regions, ADM remains informative by measuring the normalized loss gap between original and adversarial examples. We incorporate ADM into CL-based training with SAM to dynamically assess sample difficulty. We evaluated our approach on image classification tasks, fine-grained recognition, and domain generalization. The results demonstrate that our method preserves the strengths of both CL and SAM while outperforming existing curriculum-based and flatness-aware training strategies.