An Empirical Examination of Balancing Strategy for Counterfactual Estimation on Time Series

TL;DR

This paper critically evaluates the effectiveness of balancing strategies for counterfactual estimation in time series data, revealing their limitations and prompting a reexamination of their applicability in temporal settings.

Contribution

It provides the first comprehensive empirical analysis of balancing strategies' robustness and effectiveness specifically for time series counterfactual estimation.

Findings

Balancing strategies show limited effectiveness in temporal counterfactual estimation.

Effectiveness varies across different datasets and settings.

Calls for rethinking the use of balancing strategies in time series analysis.

Abstract

Counterfactual estimation from observations represents a critical endeavor in numerous application fields, such as healthcare and finance, with the primary challenge being the mitigation of treatment bias. The balancing strategy aimed at reducing covariate disparities between different treatment groups serves as a universal solution. However, when it comes to the time series data, the effectiveness of balancing strategies remains an open question, with a thorough analysis of the robustness and applicability of balancing strategies still lacking. This paper revisits counterfactual estimation in the temporal setting and provides a brief overview of recent advancements in balancing strategies. More importantly, we conduct a critical empirical examination for the effectiveness of the balancing strategies within the realm of temporal counterfactual estimation in various settings on multiple datasets. Our findings could be of significant interest to researchers and practitioners and call for a reexamination of the balancing strategy in time series settings.