Concept Drift Detection using Ensemble of Integrally Private Models

TL;DR

This paper introduces a novel ensemble method called IPDD for privately detecting concept drift in streaming data using integrally private DNNs, which does not require labels during detection and performs comparably or better than existing methods.

Contribution

The paper proposes the first ensemble approach for concept drift detection that preserves privacy through integrally private DNNs, without needing labels during drift detection.

Findings

IPDD can privately detect concept drift effectively.

IPDD outperforms or matches existing drift detection methods in utility.

The method works on both synthetic and real-world datasets.

Abstract

Deep neural networks (DNNs) are one of the most widely used machine learning algorithm. DNNs requires the training data to be available beforehand with true labels. This is not feasible for many real-world problems where data arrives in the streaming form and acquisition of true labels are scarce and expensive. In the literature, not much focus has been given to the privacy prospect of the streaming data, where data may change its distribution frequently. These concept drifts must be detected privately in order to avoid any disclosure risk from DNNs. Existing privacy models use concept drift detection schemes such ADWIN, KSWIN to detect the drifts. In this paper, we focus on the notion of integrally private DNNs to detect concept drifts. Integrally private DNNs are the models which recur frequently from different datasets. Based on this, we introduce an ensemble methodology which we call 'Integrally Private Drift Detection' (IPDD) method to detect concept drift from private models. Our IPDD method does not require labels to detect drift but assumes true labels are available once the drift has been detected. We have experimented with binary and multi-class synthetic and real-world data. Our experimental results show that our methodology can privately detect concept drift, has comparable utility (even better in some cases) with ADWIN and outperforms utility from different levels of differentially private models. The source code for the paper is available \hyperlink{https://github.com/Ayush-Umu/Concept-drift-detection-Using-Integrally-private-models}{here}.