Detecting Interpretable Subgroup Drifts

TL;DR

This paper introduces methods for detecting and interpreting data distribution drifts at the subgroup level, enabling more precise monitoring of model performance over time.

Contribution

It presents a novel approach for identifying and characterizing subgroup-specific drifts during model deployment, enhancing interpretability and sensitivity compared to global drift detection.

Findings

Subgroup-level drift detection reveals changes missed by global analysis.

The method provides interpretable summaries of subgroup drifts.

Experimental results demonstrate improved detection of subtle data shifts.

Abstract

The ability to detect and adapt to changes in data distributions is crucial to maintain the accuracy and reliability of machine learning models. Detection is generally approached by observing the drift of model performance from a global point of view. However, drifts occurring in (fine-grained) data subgroups may go unnoticed when monitoring global drift. We take a different perspective, and introduce methods for observing drift at the finer granularity of subgroups. Relevant data subgroups are identified during training and monitored efficiently throughout the model's life. Performance drifts in any subgroup are detected, quantified and characterized so as to provide an interpretable summary of the model behavior over time. Experimental results confirm that our subgroup-level drift analysis identifies drifts that do not show at the (coarser) global dataset level. The proposed approach provides a valuable tool for monitoring model performance in dynamic real-world applications, offering insights into the evolving nature of data and ultimately contributing to more robust and adaptive models.