Causal inference for data-driven debugging and decision making in cloud computing

TL;DR

This paper applies causal inference techniques to improve debugging, control, and privacy-preserving prediction in cloud computing systems, using graphical models and probabilistic formalizations.

Contribution

It introduces a data-driven causal inference framework for debugging and control, and formalizes privacy-preserving cost prediction in cloud environments.

Findings

Counterfactuals can be approximated from stochastic causal models.

Post-interventional probabilities enable privacy-preserving bidding predictions.

Experimental validation on simulated and real cloud data supports the approach.

Abstract

Cloud computing involves complex technical and economical systems and interactions. This brings about various challenges, two of which are: (1) debugging and control to optimize the performance of computing systems, with the help of sandbox experiments, and (2) privacy-preserving prediction of the cost of ``spot'' resources for decision making of cloud clients. In this paper, we formalize debugging by counterfactual probabilities and control by post-(soft-)interventional probabilities. We prove that counterfactuals can approximately be calculated from a ``stochastic'' graphical causal model (while they are originally defined only for ``deterministic'' functional causal models), and based on this sketch a data-driven approach to address problem (1). To address problem (2), we formalize bidding by post-(soft-)interventional probabilities and present a simple mathematical result on approximate integration of ``incomplete'' conditional probability distributions. We show how this can be used by cloud clients to trade off privacy against predictability of the outcome of their bidding actions in a toy scenario. We report experiments on simulated and real data.