Adversarial Learning Networks: Source-free Unsupervised Domain Incremental Learning

TL;DR

This paper introduces an unsupervised, source-free method for incrementally updating deep neural networks in non-stationary environments, using Gaussian prototypes and domain adaptation to avoid storing past data.

Contribution

It proposes a novel approach combining Gaussian prototypes and unsupervised domain adaptation for incremental learning without retaining previous training data.

Findings

Achieved improved performance over state-of-the-art methods.

Minimal forgetting of past knowledge over many iterations.

Effective in applications like sentiment and disease prediction.

Abstract

This work presents an approach for incrementally updating deep neural network (DNN) models in a non-stationary environment. DNN models are sensitive to changes in input data distribution, which limits their application to problem settings with stationary input datasets. In a non-stationary environment, updating a DNN model requires parameter re-training or model fine-tuning. We propose an unsupervised source-free method to update DNN classification models. The contributions of this work are two-fold. First, we use trainable Gaussian prototypes to generate representative samples for future iterations; second, using unsupervised domain adaptation, we incrementally adapt the existing model using unlabelled data. Unlike existing methods, our approach can update a DNN model incrementally for non-stationary source and target tasks without storing past training data. We evaluated our work on incremental sentiment prediction and incremental disease prediction applications and compared our approach to state-of-the-art continual learning, domain adaptation, and ensemble learning methods. Our results show that our approach achieved improved performance compared to existing incremental learning methods. We observe minimal forgetting of past knowledge over many iterations, which can help us develop unsupervised self-learning systems.