Weakly Supervised Deep Learning Approach in Streaming Environments

TL;DR

This paper introduces ParsNet, a self-evolving deep neural network designed for weakly supervised data streams, capable of handling concept drift and delayed labels through a self-labelling strategy and flexible structure adaptation.

Contribution

The paper presents ParsNet, a novel self-evolving deep learning model with a self-correction mechanism for weakly supervised streaming data, addressing concept drift and delayed labels.

Findings

ParsNet outperforms existing methods in high-dimensional data streams.

It effectively handles infinite delay in ground truth access.

ParsNet demonstrates robustness in challenging data stream scenarios.

Abstract

The feasibility of existing data stream algorithms is often hindered by the weakly supervised condition of data streams. A self-evolving deep neural network, namely Parsimonious Network (ParsNet), is proposed as a solution to various weakly-supervised data stream problems. A self-labelling strategy with hedge (SLASH) is proposed in which its auto-correction mechanism copes with \textit{the accumulation of mistakes} significantly affecting the model's generalization. ParsNet is developed from a closed-loop configuration of the self-evolving generative and discriminative training processes exploiting shared parameters in which its structure flexibly grows and shrinks to overcome the issue of concept drift with/without labels. The numerical evaluation has been performed under two challenging problems, namely sporadic access to ground truth and infinitely delayed access to the ground truth. Our numerical study shows the advantage of ParsNet with a substantial margin from its counterparts in the high-dimensional data streams and infinite delay simulation protocol. To support the reproducible research initiative, the source code of ParsNet along with supplementary materials are made available at https://bit.ly/2qNW7p4.