Bidirectional Information Flow (BIF) -- A Sample Efficient Hierarchical Gaussian Process for Bayesian Optimization

TL;DR

The paper introduces Bidirectional Information Flow (BIF), a hierarchical Gaussian process framework that enhances Bayesian optimization by enabling mutual top-down and bottom-up information exchange, significantly improving sample efficiency and convergence.

Contribution

BIF is a novel hierarchical Gaussian process method that incorporates bidirectional information flow, improving sample efficiency and robustness in Bayesian optimization tasks.

Findings

BIF achieves up to 4x higher R^2 scores for parent models.

BIF outperforms traditional H-GP methods on synthetic and real-world tasks.

Enhanced convergence and sample efficiency demonstrated in neurostimulation optimization.

Abstract

Hierarchical Gaussian Process (H-GP) models divide problems into different subtasks, allowing for different models to address each part, making them well-suited for problems with inherent hierarchical structure. However, typical H-GP models do not fully take advantage of this structure, only sending information up or down the hierarchy. This one-way coupling limits sample efficiency and slows convergence. We propose Bidirectional Information Flow (BIF), an efficient H-GP framework that establishes bidirectional information exchange between parent and child models in H-GPs for online training. BIF retains the modular structure of hierarchical models - the parent combines subtask knowledge from children GPs - while introducing top-down feedback to continually refine children models during online learning. This mutual exchange improves sample efficiency, enables robust training, and allows modular reuse of learned subtask models. BIF outperforms conventional H-GP Bayesian Optimization methods, achieving up to 4x and 3x higher $R^2$ scores for the parent and children respectively, on synthetic and real-world neurostimulation optimization tasks.