Unsupervised Continual Learning in Streaming Environments

TL;DR

This paper introduces ADCN, an unsupervised deep clustering network that autonomously evolves its structure in streaming data environments, effectively extracting features without labeled data and addressing catastrophic forgetting.

Contribution

It presents a novel autonomous deep clustering network that self-evolves its architecture and mitigates forgetting in streaming environments without labeled data.

Findings

ADCN outperforms existing methods in streaming data clustering tasks.

It autonomously adapts its structure based on data, eliminating manual tuning.

The approach effectively prevents catastrophic forgetting in continuous learning.

Abstract

A deep clustering network is desired for data streams because of its aptitude in extracting natural features thus bypassing the laborious feature engineering step. While automatic construction of the deep networks in streaming environments remains an open issue, it is also hindered by the expensive labeling cost of data streams rendering the increasing demand for unsupervised approaches. This paper presents an unsupervised approach of deep clustering network construction on the fly via simultaneous deep learning and clustering termed Autonomous Deep Clustering Network (ADCN). It combines the feature extraction layer and autonomous fully connected layer in which both network width and depth are self-evolved from data streams based on the bias-variance decomposition of reconstruction loss. The self-clustering mechanism is performed in the deep embedding space of every fully connected layer while the final output is inferred via the summation of cluster prediction score. Further, a latent-based regularization is incorporated to resolve the catastrophic forgetting issue. A rigorous numerical study has shown that ADCN produces better performance compared to its counterparts while offering fully autonomous construction of ADCN structure in streaming environments with the absence of any labeled samples for model updates. To support the reproducible research initiative, codes, supplementary material, and raw results of ADCN are made available in \url{https://tinyurl.com/AutonomousDCN}.