Monte Carlo Tree Descent for Black-Box Optimization

TL;DR

This paper introduces Monte Carlo Tree Descent, a novel black-box optimization method that combines tree search, stochastic descent, and Gaussian Processes to improve search efficiency and outperform existing algorithms on benchmarks.

Contribution

It develops new tree expansion and descent strategies integrating stochastic search and Gaussian Processes, advancing black-box optimization techniques.

Findings

Outperforms state-of-the-art methods on benchmark problems

Efficiently balances exploration and exploitation

Utilizes localized Gaussian Processes for better search guidance

Abstract

The key to Black-Box Optimization is to efficiently search through input regions with potentially widely-varying numerical properties, to achieve low-regret descent and fast progress toward the optima. Monte Carlo Tree Search (MCTS) methods have recently been introduced to improve Bayesian optimization by computing better partitioning of the search space that balances exploration and exploitation. Extending this promising framework, we study how to further integrate sample-based descent for faster optimization. We design novel ways of expanding Monte Carlo search trees, with new descent methods at vertices that incorporate stochastic search and Gaussian Processes. We propose the corresponding rules for balancing progress and uncertainty, branch selection, tree expansion, and backpropagation. The designed search process puts more emphasis on sampling for faster descent and uses localized Gaussian Processes as auxiliary metrics for both exploitation and exploration. We show empirically that the proposed algorithms can outperform state-of-the-art methods on many challenging benchmark problems.