EVCL: Elastic Variational Continual Learning with Weight Consolidation

TL;DR

EVCL is a hybrid continual learning model that combines variational inference and weight regularization to effectively prevent forgetting and improve task performance across multiple scenarios.

Contribution

It introduces EVCL, integrating VCL and EWC, to enhance continual learning by leveraging the strengths of both approaches.

Findings

EVCL outperforms existing methods on five discriminative tasks.

It effectively mitigates catastrophic forgetting.

EVCL improves dependency modeling between parameters and data.

Abstract

Continual learning aims to allow models to learn new tasks without forgetting what has been learned before. This work introduces Elastic Variational Continual Learning with Weight Consolidation (EVCL), a novel hybrid model that integrates the variational posterior approximation mechanism of Variational Continual Learning (VCL) with the regularization-based parameter-protection strategy of Elastic Weight Consolidation (EWC). By combining the strengths of both methods, EVCL effectively mitigates catastrophic forgetting and enables better capture of dependencies between model parameters and task-specific data. Evaluated on five discriminative tasks, EVCL consistently outperforms existing baselines in both domain-incremental and task-incremental learning scenarios for deep discriminative models.