# Causal Regularization

## TL;DR

This paper explores how regularization in regression can improve causal inference by reducing confounding effects and provides theoretical bounds on causal generalization under certain confounder models.

## Contribution

It introduces a method for selecting regularization strength based on confounding estimation and proves a causal generalization bound for non-linear models under confounding assumptions.

## Key findings

- Regularization can decrease confounding effects in causal models.
- A method for tuning regularization without cross-validation is proposed.
- A theoretical bound on causal generalization error is established.

## Abstract

I argue that regularizing terms in standard regression methods not only help against overfitting finite data, but sometimes also yield better causal models in the infinite sample regime. I first consider a multi-dimensional variable linearly influencing a target variable with some multi-dimensional unobserved common cause, where the confounding effect can be decreased by keeping the penalizing term in Ridge and Lasso regression even in the population limit. Choosing the size of the penalizing term, is however challenging, because cross validation is pointless. Here it is done by first estimating the strength of confounding via a method proposed earlier, which yielded some reasonable results for simulated and real data. Further, I prove a `causal generalization bound' which states (subject to a particular model of confounding) that the error made by interpreting any non-linear regression as causal model can be bounded from above whenever functions are taken from a not too rich class. In other words, the bound guarantees "generalization" from observational to interventional distributions, which is usually not subject of statistical learning theory (and is only possible due to the underlying symmetries of the confounder model).