Incorporating domain knowledge into neural-guided search

TL;DR

This paper introduces a formal framework for integrating domain knowledge and constraints into neural-guided search methods, enhancing their effectiveness in tasks like symbolic regression.

Contribution

It formalizes the incorporation of priors and constraints into neural-guided search and demonstrates their benefits in symbolic regression tasks.

Findings

Incorporating domain priors improves search efficiency.

Constraints help enforce valid solutions during search.

Framework is flexible for various priors and constraints.

Abstract

Many AutoML problems involve optimizing discrete objects under a black-box reward. Neural-guided search provides a flexible means of searching these combinatorial spaces using an autoregressive recurrent neural network. A major benefit of this approach is that builds up objects sequentially--this provides an opportunity to incorporate domain knowledge into the search by directly modifying the logits emitted during sampling. In this work, we formalize a framework for incorporating such in situ priors and constraints into neural-guided search, and provide sufficient conditions for enforcing constraints. We integrate several priors and constraints from existing works into this framework, propose several new ones, and demonstrate their efficacy in informing the task of symbolic regression.