On the Sparse DAG Structure Learning Based on Adaptive Lasso

TL;DR

This paper introduces a data-driven method called NOTEARS-AL for learning sparse Bayesian network structures from observational data, improving upon existing continuous optimization approaches by adaptively penalizing edges to enhance sparsity and accuracy.

Contribution

The paper proposes an adaptive Lasso-based extension to NOTEARS that automatically promotes sparsity without predefined thresholds, with theoretical oracle properties and superior empirical performance.

Findings

NOTEARS-AL outperforms NOTEARS in synthetic experiments.

The method effectively learns sparse DAG structures.

It demonstrates strong results on real-world data.

Abstract

Learning the underlying Bayesian Networks (BNs), represented by directed acyclic graphs (DAGs), of the concerned events from purely-observational data is a crucial part of evidential reasoning. This task remains challenging due to the large and discrete search space. A recent flurry of developments followed NOTEARS[1] recast this combinatorial problem into a continuous optimization problem by leveraging an algebraic equality characterization of acyclicity. However, the continuous optimization methods suffer from obtaining non-spare graphs after the numerical optimization, which leads to the inflexibility to rule out the potentially cycle-inducing edges or false discovery edges with small values. To address this issue, in this paper, we develop a completely data-driven DAG structure learning method without a predefined value to post-threshold small values. We name our method NOTEARS with adaptive Lasso (NOTEARS-AL), which is achieved by applying the adaptive penalty method to ensure the sparsity of the estimated DAG. Moreover, we show that NOTEARS-AL also inherits the oracle properties under some specific conditions. Extensive experiments on both synthetic and a real-world dataset demonstrate that our method consistently outperforms NOTEARS.