BRAIN: Bias-Mitigation Continual Learning Approach to Vision-Brain Understanding

TL;DR

This paper introduces BRAIN, a continual learning approach with novel loss functions and forgetting mitigation to improve vision-brain understanding amidst signal bias and decay.

Contribution

The paper presents a new bias-mitigation continual learning framework with innovative loss functions and forgetting strategies for vision-brain understanding.

Findings

Achieves state-of-the-art performance on multiple benchmarks.

Effectively mitigates bias and prevents catastrophic forgetting.

Demonstrates the impact of signal inconsistency on model performance.

Abstract

Memory decay makes it harder for the human brain to recognize visual objects and retain details. Consequently, recorded brain signals become weaker, uncertain, and contain poor visual context over time. This paper presents one of the first vision-learning approaches to address this problem. First, we statistically and experimentally demonstrate the existence of inconsistency in brain signals and its impact on the Vision-Brain Understanding (VBU) model. Our findings show that brain signal representations shift over recording sessions, leading to compounding bias, which poses challenges for model learning and degrades performance. Then, we propose a new Bias-Mitigation Continual Learning (BRAIN) approach to address these limitations. In this approach, the model is trained in a continual learning setup and mitigates the growing bias from each learning step. A new loss function named De-bias Contrastive Learning is also introduced to address the bias problem. In addition, to prevent catastrophic forgetting, where the model loses knowledge from previous sessions, the new Angular-based Forgetting Mitigation approach is introduced to preserve learned knowledge in the model. Finally, the empirical experiments demonstrate that our approach achieves State-of-the-Art (SOTA) performance across various benchmarks, surpassing prior and non-continual learning methods.