LLEDA -- Lifelong Self-Supervised Domain Adaptation

TL;DR

LLEDA is a lifelong domain adaptation framework inspired by brain systems, using latent replay to enable continuous learning and reduce forgetting in neural networks across changing data domains.

Contribution

LLEDA introduces a dual-network system inspired by CLS theory, combining rapid adaptation and gradual generalization with latent replay to improve lifelong domain adaptation.

Findings

Outperforms existing methods in long-term domain adaptation

Reduces catastrophic forgetting in continual learning scenarios

Effective in transferring knowledge across multiple domains

Abstract

Humans and animals have the ability to continuously learn new information over their lifetime without losing previously acquired knowledge. However, artificial neural networks struggle with this due to new information conflicting with old knowledge, resulting in catastrophic forgetting. The complementary learning systems (CLS) theory suggests that the interplay between hippocampus and neocortex systems enables long-term and efficient learning in the mammalian brain, with memory replay facilitating the interaction between these two systems to reduce forgetting. The proposed Lifelong Self-Supervised Domain Adaptation (LLEDA) framework draws inspiration from the CLS theory and mimics the interaction between two networks: a DA network inspired by the hippocampus that quickly adjusts to changes in data distribution and an SSL network inspired by the neocortex that gradually learns domain-agnostic general representations. LLEDA's latent replay technique facilitates communication between these two networks by reactivating and replaying the past memory latent representations to stabilise long-term generalisation and retention without interfering with the previously learned information. Extensive experiments demonstrate that the proposed method outperforms several other methods resulting in a long-term adaptation while being less prone to catastrophic forgetting when transferred to new domains.