Global Optimality in Bivariate Gradient-based DAG Learning

TL;DR

This paper proves that a simple path-following optimization method can globally solve the non-convex problem of learning bivariate DAGs, overcoming the challenge of multiple spurious solutions.

Contribution

It introduces a path-following scheme that guarantees global convergence in the bivariate case, a significant step in DAG learning theory.

Findings

Path-following scheme converges globally in bivariate DAG learning.

Standard first-order methods may get trapped in spurious solutions.

Theoretical proof of global optimality in the population loss.

Abstract

Recently, a new class of non-convex optimization problems motivated by the statistical problem of learning an acyclic directed graphical model from data has attracted significant interest. While existing work uses standard first-order optimization schemes to solve this problem, proving the global optimality of such approaches has proven elusive. The difficulty lies in the fact that unlike other non-convex problems in the literature, this problem is not "benign", and possesses multiple spurious solutions that standard approaches can easily get trapped in. In this paper, we prove that a simple path-following optimization scheme globally converges to the global minimum of the population loss in the bivariate setting.