Causal Covariate Shift Correction using Fisher information penalty

TL;DR

This paper introduces Causal Covariate Shift Correction ($C^{3}$), a method that uses Fisher Information to adaptively penalize model loss across distributed, temporally ordered training data, improving accuracy under covariate shift.

Contribution

It proposes a novel Fisher Information-based penalty for covariate shift correction in distributed, temporal data settings, enhancing model accuracy.

Findings

Achieves 12.9% accuracy improvement over baseline.

Reaches 20.3% maximum accuracy gain in batchwise benchmarks.

Attains 5.9% minimum accuracy gain in foldwise benchmarks.

Abstract

Evolving feature densities across batches of training data bias cross-validation, making model selection and assessment unreliable (\cite{sugiyama2012machine}). This work takes a distributed density estimation angle to the training setting where data are temporally distributed. \textit{Causal Covariate Shift Correction ($C^{3}$)}, accumulates knowledge about the data density of a training batch using Fisher Information, and using it to penalize the loss in all subsequent batches. The penalty improves accuracy by $12.9\%$ over the full-dataset baseline, by $20.3\%$ accuracy at maximum in batchwise and $5.9\%$ at minimum in foldwise benchmarks.