Confounder-Free Continual Learning via Recursive Feature Normalization

TL;DR

This paper introduces Recursive MDN, a novel layer for deep learning models that continually removes confounder effects from features, improving fairness and reducing forgetting in continual learning scenarios.

Contribution

The paper proposes Recursive MDN (R-MDN), a new method that dynamically normalizes features to eliminate confounders in continual learning models.

Findings

R-MDN improves equitable predictions across groups.

R-MDN reduces catastrophic forgetting caused by confounders.

R-MDN can be integrated into various architectures, including vision transformers.

Abstract

Confounders are extraneous variables that affect both the input and the target, resulting in spurious correlations and biased predictions. There are recent advances in dealing with or removing confounders in traditional models, such as metadata normalization (MDN), where the distribution of the learned features is adjusted based on the study confounders. However, in the context of continual learning, where a model learns continuously from new data over time without forgetting, learning feature representations that are invariant to confounders remains a significant challenge. To remove their influence from intermediate feature representations, we introduce the Recursive MDN (R-MDN) layer, which can be integrated into any deep learning architecture, including vision transformers, and at any model stage. R-MDN performs statistical regression via the recursive least squares algorithm to maintain and continually update an internal model state with respect to changing distributions of data and confounding variables. Our experiments demonstrate that R-MDN promotes equitable predictions across population groups, both within static learning and across different stages of continual learning, by reducing catastrophic forgetting caused by confounder effects changing over time.