On the Use of Causal Graphical Models for Designing Experiments in the Automotive Domain

TL;DR

This paper explores how causal graphical models can improve the design and validity assessment of experiments in automotive software, addressing environmental dependencies and transportability issues.

Contribution

It introduces the application of causal graphical models at Volvo Cars for designing automotive experiments and explicitly communicating assumptions.

Findings

Enhanced experiment validity assessment using causal models

Ability to compute direct and indirect causal effects

Reasoning about transportability of causal conclusions

Abstract

Randomized field experiments are the gold standard for evaluating the impact of software changes on customers. In the online domain, randomization has been the main tool to ensure exchangeability. However, due to the different deployment conditions and the high dependence on the surrounding environment, designing experiments for automotive software needs to consider a higher number of restricted variables to ensure conditional exchangeability. In this paper, we show how at Volvo Cars we utilize causal graphical models to design experiments and explicitly communicate the assumptions of experiments. These graphical models are used to further assess the experiment validity, compute direct and indirect causal effects, and reason on the transportability of the causal conclusions.