Learning Sparse Nonparametric DAGs

TL;DR

This paper introduces a general continuous optimization framework for learning sparse nonparametric DAGs from data, extending algebraic characterizations to a wide class of nonlinear models and loss functions.

Contribution

It extends algebraic DAG characterization to nonparametric SEMs, enabling flexible, model-agnostic DAG learning with continuous optimization.

Findings

Applicable to various nonparametric and semiparametric models

Supports general nonlinear models without specific assumptions

Compatible with black-box optimization routines

Abstract

We develop a framework for learning sparse nonparametric directed acyclic graphs (DAGs) from data. Our approach is based on a recent algebraic characterization of DAGs that led to a fully continuous program for score-based learning of DAG models parametrized by a linear structural equation model (SEM). We extend this algebraic characterization to nonparametric SEM by leveraging nonparametric sparsity based on partial derivatives, resulting in a continuous optimization problem that can be applied to a variety of nonparametric and semiparametric models including GLMs, additive noise models, and index models as special cases. Unlike existing approaches that require specific modeling choices, loss functions, or algorithms, we present a completely general framework that can be applied to general nonlinear models (e.g. without additive noise), general differentiable loss functions, and generic black-box optimization routines. The code is available at https://github.com/xunzheng/notears.