Adaptation Speed Analysis for Fairness-aware Causal Models

TL;DR

This paper investigates how causal models with fairness considerations adapt to domain shifts, analyzing the influence of causal structure and sensitive variables on adaptation speed through theoretical and simulated experiments.

Contribution

It introduces a causal framework for analyzing adaptation speed in fairness-aware models and establishes theoretical links between model adaptation rates under various interventions.

Findings

Causal structure significantly impacts adaptation speed.

Sensitive variables influence the rate of domain adaptation.

Theoretical connection between adaptation speeds of different causal models.

Abstract

For example, in machine translation tasks, to achieve bidirectional translation between two languages, the source corpus is often used as the target corpus, which involves the training of two models with opposite directions. The question of which one can adapt most quickly to a domain shift is of significant importance in many fields. Specifically, consider an original distribution p that changes due to an unknown intervention, resulting in a modified distribution p*. In aligning p with p*, several factors can affect the adaptation rate, including the causal dependencies between variables in p. In real-life scenarios, however, we have to consider the fairness of the training process, and it is particularly crucial to involve a sensitive variable (bias) present between a cause and an effect variable. To explore this scenario, we examine a simple structural causal model (SCM) with a cause-bias-effect structure, where variable A acts as a sensitive variable between cause (X) and effect (Y). The two models, respectively, exhibit consistent and contrary cause-effect directions in the cause-bias-effect SCM. After conducting unknown interventions on variables within the SCM, we can simulate some kinds of domain shifts for analysis. We then compare the adaptation speeds of two models across four shift scenarios. Additionally, we prove the connection between the adaptation speeds of the two models across all interventions.