Deep Conditional Transformation Models

TL;DR

This paper introduces deep conditional transformation models that unify existing methods to flexibly and interpretably model complex conditional distributions, including high-dimensional data like images and text.

Contribution

It proposes a novel deep neural network architecture for conditional transformation models, enabling flexible, interpretable, and complex modeling of conditional distributions.

Findings

Effective in modeling high-dimensional data

Unifies existing approaches into a single framework

Demonstrates strong performance in experiments

Abstract

Learning the cumulative distribution function (CDF) of an outcome variable conditional on a set of features remains challenging, especially in high-dimensional settings. Conditional transformation models provide a semi-parametric approach that allows to model a large class of conditional CDFs without an explicit parametric distribution assumption and with only a few parameters. Existing estimation approaches within this class are, however, either limited in their complexity and applicability to unstructured data sources such as images or text, lack interpretability, or are restricted to certain types of outcomes. We close this gap by introducing the class of deep conditional transformation models which unifies existing approaches and allows to learn both interpretable (non-)linear model terms and more complex neural network predictors in one holistic framework. To this end we propose a novel network architecture, provide details on different model definitions and derive suitable constraints as well as network regularization terms. We demonstrate the efficacy of our approach through numerical experiments and applications.