Causal Parametric Drift Simulation: A Digital Twin Framework for Classifier Robustness Evaluation

TL;DR

This paper introduces a causal simulation framework using Structural Causal Models as digital twins to evaluate classifier robustness against concept drift in dynamic environments.

Contribution

It presents a novel causal parametric drift simulation method that enables precise causal interventions for stress-testing classifiers, surpassing traditional evaluation techniques.

Findings

Exposes latent vulnerabilities in classifiers not detected by standard methods.

Uses causal models to simulate realistic concept drift scenarios.

Demonstrates effectiveness on the OSMH dataset.

Abstract

Machine learning classifiers in dynamic environments face concept drift -- changes in the data-generating process that degrade performance. Conventional evaluation via static test sets or noise perturbations fails to preserve causal dependencies in tabular data, often producing causally invalid assessments. Post-hoc tools like SHAP and LIME offer correlational insights that may not reflect the causal mechanisms driving model failure. We propose a framework that complements existing drift detection by leveraging Structural Causal Models as "Digital Twins" of data-generating processes, enabling precise causal interventions while preserving structural dependencies. Our technique, Causal Parametric Drift Simulation, stress-tests classifiers to identify vulnerabilities before deployment. Experiments on the Open Sourcing Mental Illness (OSMH) dataset demonstrate that this approach exposes latent vulnerabilities invisible to standard statistical monitors.