Modeling Hierarchical Spaces: A Review and Unified Framework for Surrogate-Based Architecture Design

TL;DR

This paper reviews structured input spaces in simulation-based problems and proposes a unified framework that models hierarchical, conditional, and heterogeneous variables for improved surrogate modeling and optimization.

Contribution

It introduces a comprehensive framework that generalizes existing approaches to model hierarchical and conditional variables, with new concepts like meta and partially-decreed variables, and provides practical tools for complex system design.

Findings

Effective modeling of complex system architectures demonstrated.

Framework enables surrogate modeling on hierarchical domains.

Open-source implementation available in SMT 2.0.

Abstract

Simulation-based problems involving mixed-variable inputs frequently feature domains that are hierarchical, conditional, heterogeneous, or tree-structured. These characteristics pose challenges for data representation, modeling, and optimization. This paper reviews extensive literature on these structured input spaces and proposes a unified framework that generalizes existing approaches. In this framework, input variables may be continuous, integer, or categorical. A variable is described as meta if its value governs the presence of other decreed variables, enabling the modeling of conditional and hierarchical structures. We further introduce the concept of partially-decreed variables, whose activation depends on contextual conditions. To capture these inter-variable hierarchical relationships, we introduce design space graphs, combining principles from feature modeling and graph theory. This allows the definition of general hierarchical domains suitable for describing complex system architectures. Our framework defines hierarchical distances and kernels to enable surrogate modeling and optimization on hierarchical domains. We demonstrate its effectiveness on complex system design problems, including a neural network and a green-aircraft case study. Our methods are available in the open-source Surrogate Modeling Toolbox (SMT 2.0).