Behavior Learning (BL): Learning Hierarchical Optimization Structures from Data

TL;DR

Behavior Learning (BL) is a new machine learning framework that learns interpretable hierarchical optimization structures from data, unifying predictive accuracy with interpretability and identifiability across various scientific domains.

Contribution

BL introduces a modular, compositional utility function framework that models hierarchical optimization structures and guarantees interpretability and identifiability, supported by theoretical and empirical validation.

Findings

BL demonstrates strong predictive performance on complex data.

BL achieves intrinsic interpretability and scalability.

Theoretical analysis confirms universal approximation and identifiability.

Abstract

Inspired by behavioral science, we propose Behavior Learning (BL), a novel general-purpose machine learning framework that learns interpretable and identifiable optimization structures from data, ranging from single optimization problems to hierarchical compositions. It unifies predictive performance, intrinsic interpretability, and identifiability, with broad applicability to scientific domains involving optimization. BL parameterizes a compositional utility function built from intrinsically interpretable modular blocks, which induces a data distribution for prediction and generation. Each block represents and can be written in symbolic form as a utility maximization problem (UMP), a foundational paradigm in behavioral science and a universal framework of optimization. BL supports architectures ranging from a single UMP to hierarchical compositions, the latter modeling hierarchical optimization structures. Its smooth and monotone variant (IBL) guarantees identifiability. Theoretically, we establish the universal approximation property of BL, and analyze the M-estimation properties of IBL. Empirically, BL demonstrates strong predictive performance, intrinsic interpretability and scalability to high-dimensional data. Code: https://github.com/MoonYLiang/Behavior-Learning ; install via pip install blnetwork.