Spurious Correlation-Aware Embedding Regularization for Worst-Group Robustness
Subeen Park, Joowang Kim, Hakyung Lee, Sunjae Yoo, Kyungwoo Song
TL;DR
This paper introduces SCER, a new regularization method that improves worst-group robustness in deep learning by focusing on core features and reducing reliance on spurious correlations, supported by a theoretical framework.
Contribution
SCER is the first method to theoretically connect embedding space representations with worst-group error and directly regularize features to enhance robustness.
Findings
SCER outperforms previous methods in worst-group accuracy.
Theoretical analysis links classifier reliance on spurious cues to worst-group error.
Empirical results on vision and language tasks demonstrate SCER's effectiveness.
Abstract
Deep learning models achieve strong performance across various domains but often rely on spurious correlations, making them vulnerable to distribution shifts. This issue is particularly severe in subpopulation shift scenarios, where models struggle in underrepresented groups. While existing methods have made progress in mitigating this issue, their performance gains are still constrained. They lack a rigorous theoretical framework connecting the embedding space representations with worst-group error. To address this limitation, we propose Spurious Correlation-Aware Embedding Regularization for Worst-Group Robustness (SCER), a novel approach that directly regularizes feature representations to suppress spurious cues. We show theoretically that worst-group error is influenced by how strongly the classifier relies on spurious versus core directions, identified from differences in…
Peer Reviews
Decision·ICLR 2026 Poster
The paper precisely defines core and spurious mean-embedding differences, aggregates them into global directions, and integrates their alignment with classifier weights into a composite loss term that is transparent and straightforward to implement. The authors conduct comprehensive experiments 6 datasets across vision and language domains, comparing against 19 baseline methods. The settings also cover different spurious correlation strengths and extreme scenarios. Notably, the method substan
When formulating the theorem for error decomposition, the paper is restricted to binary labels and two domains and assumes constant core and spurious differences between domains and classes. The paper does not discuss how violations of these assumptions affect the method's validity, particularly for multi-class problems, which appear in experiments like MultiNLI with 3 classes. It can be argued that correctly specified group labels are required to operationalize SCER in terms of its component
- The method is concisely proposed with theoretical analysis - The results are clearly explained with interpretable evidence
- The core idea is to encourage models to rely less on spurious correlations and more on invariant features. This has been explored in a substantial body of prior work both empirically and theoretically. The contribution here, while sound, does not clearly break new conceptual ground, considering the improvements over baselines appear modest. - I believe this work, as well as similar ones, should be applied to datasets with a larger number of groups. For example, where the number of groups is gr
(S1) The proposed constraints on the embeddings are explicit and look novel to me, especially the correlation terms defined in Eq. (1), which are well supported by Theorem 1. I believe the research directions enlightened by this paper are very promising. Besides, the Introduction and Related Works sections also provide a good discussion about this paper's contributions, which places it well among the literature on spurious correlations. (S2) Comprehensive experimental settings provide strong e
(W1) While a schematic diagram/representation of the embeddings learned from the proposed methods is presented in Figure 1, a comprehensive evaluation of the structure and clusters of the learned embeddings, in comparison to baseline methods, is not included in this paper, which weakens the evaluation of a key contribution of this paper: the explicit constraints on the embeddings. In particular, it would be very helpful for readers to examine the differences between the embeddings learned by the
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsDomain Adaptation and Few-Shot Learning · Face recognition and analysis · Advanced Neural Network Applications